<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Hey Jürgen,<br>
<br>
how much RAM do you assign for this process? (the -Xmx setting)<br>
What version of GraphHopper do you use?<br>
And do you apply some specific configuration or customization to
the LocationIndexTree?<br>
<br>
> I already am aware, that I should not flush() the graph when
I use RAM or RAM_STORE (because everything is kept in Memory I
guess?). <br>
> When I use MMAP, the reader flushes after every 100000 items
which were read and after finishing reading.<br>
<br>
I think flush is only necessary at the end even for MMAP.<br>
<br>
<br>
> I already am aware, that I should not flush() the graph when
I use RAM or RAM_STORE (because everything is kept in Memory I
guess?).<br>
<br>
You should still flush at the end. When you use 'RAM' then simply
nothing happens. <br>
<br>
But when you use RAM_STORE 'flush' writes to disk. The next time
you start GH it then can avoid parsing and it will just load the
data from disk into memory<br>
<br>
<br>
<font face="Courier New, Courier, monospace"><small><small>>
Exception in thread "main" java.lang.NullPointerException<br>
> at
com.graphhopper.storage.MMapDataAccess.newByteBuffer(MMapDataAccess.java:176)</small></small></font><br>
<br>
Are you calling create() before you use the DataAccess files?<br>
Or what is 'null' on line 176 - would you point to the github
source for it?<br>
<br>
> First I have to say, I'm sorry, that I am not really totally
focused on this,<br>
<br>
No problem. Good to have some other usecases to make it robust and
more userfriendly.<br>
<br>
> initialize GraphStorage<br>
<br>
Please see the javadocs of GraphHopperStorage:<br>
Life cycle: (1) object creation, (2) configuration via setters
& getters, (3) create or loadExisting, (4) usage, (5) flush,
(6) close<br>
<br>
Regards,<br>
Peter.<br>
<br>
<br>
</div>
<blockquote cite="mid:53581359.4010002@gmail.com" type="cite">
<meta http-equiv="content-type" content="text/html;
charset=ISO-8859-1">
Hi Peter,<br>
<br>
I can't get my head around storage types in graphhopper, and how
to treat them correctly. I have my own little Idf File reader
which works perfectly with small graphs. Although there is still
much work to do (writing tests, refactor some classes, actually
test) it reads my idf mockup graphs and routes correctly then.<br>
<br>
But when I take a larger amount of data (Graph of austria with
approx. 1 million nodes and 1.2 million links) my 4GB 4 Core Linux
Notebook runs out of memory, when I use DAType.RAM_STORE<br>
<br>
<small><small><font face="Courier New, Courier, monospace">2014-04-23
20:57:20,221 [main] INFO reader.idf.IdfReader - Starting to
read nodes!<br>
</font></small></small><small><small><font face="Courier New,
Courier, monospace">2014-04-23 20:57:24,281 [main] INFO
reader.idf.IdfReader - Read 1034868 records which equals
1034868 rows as expected<br>
2014-04-23 20:57:24,281 [main] INFO reader.idf.IdfReader -
Graph has 1034868 nodes.<br>
2014-04-23 20:57:24,281 [main] INFO reader.idf.IdfReader -
Starting to read links!<br>
2014-04-23 20:57:36,024 [main] INFO reader.idf.IdfReader -
Read 1207004 records which equals 1207004 rows as expected<br>
Exception in thread "main" java.lang.OutOfMemoryError: GC
overhead limit exceeded<br>
at
com.graphhopper.storage.index.LocationIndexTree$InMemTreeEntry.<init>(LocationIndexTree.java:844)<br>
at
com.graphhopper.storage.index.LocationIndexTree$InMemConstructionIndex.addNode(LocationIndexTree.java:428)<br>
at
com.graphhopper.storage.index.LocationIndexTree$InMemConstructionIndex.addNode(LocationIndexTree.java:433)<br>
at
com.graphhopper.storage.index.LocationIndexTree$InMemConstructionIndex.addNode(LocationIndexTree.java:433)<br>
at
com.graphhopper.storage.index.LocationIndexTree$InMemConstructionIndex.addNode(LocationIndexTree.java:433)</font></small></small><br>
<br>
When I use DAType.MMAP I get a NullPointerException immediately
after inserting the very first node, while creating the
"newByteBuffer" in MMapDataAccess.java:<br>
<font face="Courier New, Courier, monospace"><small><small>2014-04-23
21:18:27,264 [main] INFO reader.idf.IdfReader - Starting to
read nodes!<br>
Exception in thread "main" java.lang.NullPointerException<br>
at
com.graphhopper.storage.MMapDataAccess.newByteBuffer(MMapDataAccess.java:176)<br>
at
com.graphhopper.storage.MMapDataAccess.mapIt(MMapDataAccess.java:150)<br>
at
com.graphhopper.storage.MMapDataAccess.incCapacity(MMapDataAccess.java:103)<br>
at
com.graphhopper.storage.GraphHopperStorage.ensureNodeIndex(GraphHopperStorage.java:261)<br>
at
com.graphhopper.storage.GraphHopperStorage.setNode(GraphHopperStorage.java:232)<br>
at
com.graphhopper.reader.idf.IdfReader.loadGraph(IdfReader.java:237)<br>
at
com.graphhopper.reader.idf.IdfReader.doIdf2Graph(IdfReader.java:102)<br>
at
com.graphhopper.GipHopperIdf.importINTREST(GipHopperIdf.java:200)<br>
at
com.graphhopper.GipHopperIdf.process(GipHopperIdf.java:175)<br>
at
com.graphhopper.GipHopperIdf.importOrLoad(GipHopperIdf.java:159)<br>
at
com.graphhopper.GipHopperIdf.main(GipHopperIdf.java:42)</small></small></font><br>
<br>
<br>
I already am aware, that I should not flush() the graph when I use
RAM or RAM_STORE (because everything is kept in Memory I guess?).
When I use MMAP, the reader flushes after every 100000 items which
were read and after finishing reading. In this example where I
load all Nodes and Links from Austria, no graph files are created
in the graph-location folder.<br>
<br>
First I have to say, I'm sorry, that I am not really totally
focused on this, because I develop on this reader really as a
hobby project, while doing ten others. But for now I did a lot of
debugging and research but I think I am completely stuck at the
moment, which is possibly a result of my incomplete understanding
of how to initialize a GraphStorage correctly.<br>
<br>
Any hints appreciated.<br>
<br>
best regards,<br>
<br>
Jürgen<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
GraphHopper mailing list
<a class="moz-txt-link-abbreviated" href="mailto:GraphHopper@openstreetmap.org">GraphHopper@openstreetmap.org</a>
<a class="moz-txt-link-freetext" href="https://lists.openstreetmap.org/listinfo/graphhopper">https://lists.openstreetmap.org/listinfo/graphhopper</a>
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
GraphHopper.com - Fast & Flexible Road Routing</pre>
</body>
</html>