[GraphHopper] Graph Storage Types and their correct initialization

[Peter K]

Hey Jürgen!

> which means: Changes in our graph should be intercepted and produce
> messages to a GraphHopper based Webressource which updates the graph
> in-situ. Probably fast (although its not CH-enabled, its still faster
> than other commercial routers out there ;) ) 

nice to hear this ;)


> routing can be done then on the latest graph updates everytime. So
> thats the goal hopefully reached by end of this summer.

For CH one could do some kind of a background process after updating the
graph, but before this I have to improve the CH stuff so that normal
algorithms also work on a prepared graph ...


>>> > Are you calling create() before you use the DataAccess files?
>>>
>>> No, I don't....seems that you've found my second major error here. I
>>> insprected OSMReader for how to do that. So when I have 1034868
>>> nodes, does that mean, that I have to create GraphStorage with a
>>> bytecount of at least 1034868 / 50 = 20698?
>> The count parameter is kind of historic, so I should remove it (but
>> still hesitating as some implementations could benefit from such an
>> estimation). So just use 1000 or whatever as the DataAccess objects
>> will increase dynamically via appending new fixed-size segments.
>
> Ok, took my estimate above and will try to integrate that directly
> into the reader with some kind of sophisticated auto-estimate just to
> be on the secure side. Oh, and its working now also with MMAP!

Good!

>> Ah, another thing I forgot, also in the javadocs! The DataAccess
>> stuff is kind of very low level and you need to call incCapacity
>> everytime you are not sure the old size fits. To avoid calling this
>> too much you can do similar stuff I do in
>> GraphHopperStorage.ensureNodeIndex
>
> As far as I can see this only has performance implications while
> reading (2-3 times slower). Graph storing also works when I set
> ByteCount to 10 and don't do incCapacity. It seems that growing
> GraphStorage is done by your implementation automagically... or does
> this do other mean things I don't see at the moment?

GraphHopperStorage is doing this for you (ensure* methods). Somehow I
thought you are using a customized graph storage with direct usage of
the DataAccess stuff.


>
> I am clearly aware that choosing the right capacity before reading has
> great potential to improve the readers performance, but I think I will
> add this at a later stage.

Not sure, I'm doing a world wide import very fast (in 1h) and this does
not look like the bottleneck. Although I must admit I have not tried
recently to estimate this up-front :)

Regards,
Peter.