[GraphHopper] Graph Storage Types and their correct initialization

[Jürgen Zornig]

>>
>> > do you apply some specific configuration or customization to the 
>> LocationIndexTree?
>>
>> I copied and modified mostly from OSMReader.java for my IdfReader and 
>> from GraphHopper.java for my own GipHopperIdf implementation. 
>> Modifications are mostly simplifying. I removed all parts which are 
>> related to ContractionHierachies for example, because I want to 
>> create a router which can change the graph dynamically while runtime. 
>
> You removed for clarity? Or what was the problem?
Yes for clarity and not making things too complicated on one hand, but 
on the other hand also because I would like to use the graph like an 
search index (Solr or ElasticSearch for example), which means: Changes 
in our graph should be intercepted and produce messages to a GraphHopper 
based Webressource which updates the graph in-situ. Probably fast 
(although its not CH-enabled, its still faster than other commercial 
routers out there ;) ) routing can be done then on the latest graph 
updates everytime. So thats the goal hopefully reached by end of this 
summer.

>
>> RAM and RAM_STORE keep crashing just when I call graph.flush() with 
>> the following ArrayIndexOutOfBoundsException in the RAMDataAccess 
>> Provider.
>
> Another thing I forgot. See below.
>
>> > Are you calling create() before you use the DataAccess files?
>>
>> No, I don't....seems that you've found my second major error here. I 
>> insprected OSMReader for how to do that. So when I have 1034868 
>> nodes, does that mean, that I have to create GraphStorage with a 
>> bytecount of at least 1034868 / 50 = 20698?
> The count parameter is kind of historic, so I should remove it (but 
> still hesitating as some implementations could benefit from such an 
> estimation). So just use 1000 or whatever as the DataAccess objects 
> will increase dynamically via appending new fixed-size segments.

Ok, took my estimate above and will try to integrate that directly into 
the reader with some kind of sophisticated auto-estimate just to be on 
the secure side. Oh, and its working now also with MMAP! (and also 
DAType.RAM stores now a Graph with the same size of MBytes on the 
disc.... have to find out why this is happening. probably I am 
overwriting with RAM_STORE somewhere).

>
>> I am adding nodes to my graph while reading my Idf/Csv File. I only 
>> know how much nodes I have, when I am completely through. So I guess 
>> there is no way to make GraphStorage increase bytecount while 
>> creating nodes automatically and that I have to do an intermediate 
>> read of my file first before filling the GraphStorage (which is 
>> absolutely ok, I just want to know if I choose the right strategy)?
>
> Ah, another thing I forgot, also in the javadocs! The DataAccess stuff 
> is kind of very low level and you need to call incCapacity everytime 
> you are not sure the old size fits. To avoid calling this too much you 
> can do similar stuff I do in GraphHopperStorage.ensureNodeIndex

As far as I can see this only has performance implications while reading 
(2-3 times slower). Graph storing also works when I set ByteCount to 10 
and don't do incCapacity. It seems that growing GraphStorage is done by 
your implementation automagically... or does this do other mean things I 
don't see at the moment?

I am clearly aware that choosing the right capacity before reading has 
great potential to improve the readers performance, but I think I will 
add this at a later stage.