<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-15">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi Peter,<br>
<br>
I can't get my head around storage types in graphhopper, and how to
treat them correctly. I have my own little Idf File reader which
works perfectly with small graphs. Although there is still much work
to do (writing tests, refactor some classes, actually test) it reads
my idf mockup graphs and routes correctly then.<br>
<br>
But when I take a larger amount of data (Graph of austria with
approx. 1 million nodes and 1.2 million links) my 4GB 4 Core Linux
Notebook runs out of memory, when I use DAType.RAM_STORE<br>
<br>
<small><small><font face="Courier New, Courier, monospace">2014-04-23
20:57:20,221 [main] INFO reader.idf.IdfReader - Starting to
read nodes!<br>
</font></small></small><small><small><font face="Courier New,
Courier, monospace"><small><small><font face="Courier New,
Courier, monospace"></font></small></small>2014-04-23
20:57:24,281 [main] INFO reader.idf.IdfReader - Read 1034868
records which equals 1034868 rows as expected<br>
2014-04-23 20:57:24,281 [main] INFO reader.idf.IdfReader -
Graph has 1034868 nodes.<br>
2014-04-23 20:57:24,281 [main] INFO reader.idf.IdfReader -
Starting to read links!<br>
2014-04-23 20:57:36,024 [main] INFO reader.idf.IdfReader -
Read 1207004 records which equals 1207004 rows as expected<br>
Exception in thread "main" java.lang.OutOfMemoryError: GC
overhead limit exceeded<br>
at
com.graphhopper.storage.index.LocationIndexTree$InMemTreeEntry.<init>(LocationIndexTree.java:844)<br>
at
com.graphhopper.storage.index.LocationIndexTree$InMemConstructionIndex.addNode(LocationIndexTree.java:428)<br>
at
com.graphhopper.storage.index.LocationIndexTree$InMemConstructionIndex.addNode(LocationIndexTree.java:433)<br>
at
com.graphhopper.storage.index.LocationIndexTree$InMemConstructionIndex.addNode(LocationIndexTree.java:433)<br>
at
com.graphhopper.storage.index.LocationIndexTree$InMemConstructionIndex.addNode(LocationIndexTree.java:433)</font></small></small><br>
<br>
When I use DAType.MMAP I get a NullPointerException immediately
after inserting the very first node, while creating the
"newByteBuffer" in MMapDataAccess.java:<br>
<font face="Courier New, Courier, monospace"><small><small>2014-04-23
21:18:27,264 [main] INFO reader.idf.IdfReader - Starting to
read nodes!<br>
Exception in thread "main" java.lang.NullPointerException<br>
at
com.graphhopper.storage.MMapDataAccess.newByteBuffer(MMapDataAccess.java:176)<br>
at
com.graphhopper.storage.MMapDataAccess.mapIt(MMapDataAccess.java:150)<br>
at
com.graphhopper.storage.MMapDataAccess.incCapacity(MMapDataAccess.java:103)<br>
at
com.graphhopper.storage.GraphHopperStorage.ensureNodeIndex(GraphHopperStorage.java:261)<br>
at
com.graphhopper.storage.GraphHopperStorage.setNode(GraphHopperStorage.java:232)<br>
at
com.graphhopper.reader.idf.IdfReader.loadGraph(IdfReader.java:237)<br>
at
com.graphhopper.reader.idf.IdfReader.doIdf2Graph(IdfReader.java:102)<br>
at
com.graphhopper.GipHopperIdf.importINTREST(GipHopperIdf.java:200)<br>
at
com.graphhopper.GipHopperIdf.process(GipHopperIdf.java:175)<br>
at
com.graphhopper.GipHopperIdf.importOrLoad(GipHopperIdf.java:159)<br>
at com.graphhopper.GipHopperIdf.main(GipHopperIdf.java:42)</small></small></font><br>
<br>
<br>
I already am aware, that I should not flush() the graph when I use
RAM or RAM_STORE (because everything is kept in Memory I guess?).
When I use MMAP, the reader flushes after every 100000 items which
were read and after finishing reading. In this example where I load
all Nodes and Links from Austria, no graph files are created in the
graph-location folder.<br>
<br>
First I have to say, I'm sorry, that I am not really totally focused
on this, because I develop on this reader really as a hobby project,
while doing ten others. But for now I did a lot of debugging and
research but I think I am completely stuck at the moment, which is
possibly a result of my incomplete understanding of how to
initialize a GraphStorage correctly.<br>
<br>
Any hints appreciated.<br>
<br>
best regards,<br>
<br>
Jürgen<br>
</body>
</html>