[GraphHopper] Running out of heap space when starting webapp with large .osm.pbf

[Nate Atkinson]

Hi Álvaro,

I have -Xmx7g already in the command line. I'll try moving it to before
the -jar.

Due to a mishap with a "*" on the command line, I need to redownload the
.osm.pbf file, but I'll give this a shot in an hour or so when the file
finishes.

Best regards,

Nate Atkinson


On 10/10/2014 08:20 AM, "Álvaro Monares G." wrote:
> Hi Nate, I assume that the issue is that the java virtual machine is
> running out of memory.
> So you can modify the maximum Java heap size with the parameter
> -Xmx
> For instance
> java -Xmx1024m  -jar graphhopper-web-0.3-with-dep.jar -Xmx7g
> jetty.resourcebase=webapp   config=config-example.properties
> osmreader.osm=./north-america-latest.osm.pbf
> 
> So you are giving to the java virtual machine 1024MB (maximum) for
> memory, if you run out of memory again
> you can increase.
> 
> Well, best regards
> Álvaro Monares G.
> 
> El 10-10-2014 10:10, Nate Atkinson escribió:
>> Hi Muhammad,
>>
>> I'd be happy to give this a try. I'm not so familiar with java, and
>> internet searching for application tag / manifest / "largeHeap = true"
>> seems to lead me to Android resources. In this case, I'm trying to use
>> the web application linked from the quickstart guide on a server running
>> Linux. Is this setting still relevant for non-Android systems? If so,
>> which config file should I change?
>>
>> Best Regards,
>>
>> Nate Atkinson
>>
>> On 10/10/2014 07:47 AM, Muhammad Babar wrote:
>>> Have you tried to set largeHeap = true in application tag (manifest)?
>>>
>>> On Fri, Oct 10, 2014 at 5:29 PM, Nate Atkinson <nate at gaia-movement.org
>>> <mailto:nate at gaia-movement.org>> wrote:
>>>
>>>      Hi,
>>>
>>>      I'm evaluating GraphHopper for integration with some software at
>>> work--
>>>      specifically quickly generating geojson path for routes with up
>>> to 40
>>>      stops. Using a small .osm.pbf file downloaded from geofabrik, it
>>> looks
>>>      like GraphHopper will fit our needs well, and I'm excited to use
>>> it.
>>>
>>>      When I try to use a larger .osm.pbf file, like the 6.2 GB file
>>> for North
>>>      America, I get an error when trying to start.
>>>
>>>      Here's what happens:
>>>
>>>      nate at NateTestEnv:~/graphhopper$ java -jar
>>>      graphhopper-web-0.3-with-dep.jar -Xmx7g jetty.resourcebase=webapp
>>>      config=config-example.properties
>>>      osmreader.osm=./north-america-latest.osm.pbf
>>>
>>>      2014-10-09 13:02:38,259 [main] INFO  com.graphhopper.GraphHopper -
>>>      version 0.3|2014-05-13T11:36:30+0200 (3,5,3,2,2)
>>>      2014-10-09 13:02:38,275 [main] INFO  com.graphhopper.GraphHopper
>>> - graph
>>>      LevelGraphStorage|car|MMAP_STORE_SYNC|2D|,,,,, details:edges:0(0),
>>>      nodes:0(0), name: - (0), geo:0(0),
>>>     
>>> bounds:1.7976931348623157E308,-1.7976931348623157E308,1.7976931348623157E308,-1.7976931348623157E308
>>>
>>>      2014-10-09 13:02:38,300 [main] INFO  com.graphhopper.GraphHopper
>>> - start
>>>      creating graph from ./north-america-latest.osm.pbf
>>>      2014-10-09 13:02:38,300 [main] INFO  com.graphhopper.GraphHopper
>>> - using
>>>      LevelGraphStorage|car|MMAP_STORE_SYNC|2D|,,,,, memory:totalMB:120,
>>>      usedMB:8
>>>      2014-10-09 13:10:22,788 [main] INFO  graphhopper.reader.OSMReader -
>>>      500 000 (preprocess), osmIdMap:6 959 244 (86MB) totalMB:723,
>>> usedMB:297
>>>
>>>      ....
>>>
>>>      2014-10-09 13:13:01,288 [main] INFO  graphhopper.reader.OSMReader -
>>>      10 000 000 (preprocess), osmIdMap:137 558 252 (1630MB)
>>> totalMB:1895,
>>>      usedMB:1726
>>>      Exception in thread "pool-1-thread-2"
>>> java.lang.OutOfMemoryError: GC
>>>      overhead limit exceeded
>>>              at
>>>     
>>> gnu.trove.list.array.TLongArrayList.ensureCapacity(TLongArrayList.java:186)
>>>
>>>              at
>>>      gnu.trove.list.array.TLongArrayList.add(TLongArrayList.java:221)
>>>              at
>>>     
>>> com.graphhopper.reader.pbf.PbfBlobDecoder.processWays(PbfBlobDecoder.java:286)
>>>
>>>              at
>>>     
>>> com.graphhopper.reader.pbf.PbfBlobDecoder.processOsmPrimitives(PbfBlobDecoder.java:372)
>>>
>>>              at
>>>     
>>> com.graphhopper.reader.pbf.PbfBlobDecoder.runAndTrapExceptions(PbfBlobDecoder.java:388)
>>>
>>>              at
>>>     
>>> com.graphhopper.reader.pbf.PbfBlobDecoder.run(PbfBlobDecoder.java:406)
>>>      ....
>>>
>>>
>>>      I get a similar message for a second thread. The process stays
>>> running,
>>>      but I see no CPU activity in top.
>>>
>>>      The system is running Debian wheezy. I tested with openjdk 1.6
>>> and 1.7.
>>>
>>>      So, how can I get this running with a large network? I'm sure
>>> there's a
>>>      command line switch or config setting somewhere that I've missed
>>> due to
>>>      my inexperience with java.
>>>
>>>      Best regards,
>>>
>>>      Nate Atkinson
>>>
>>>      _______________________________________________
>>>      GraphHopper mailing list
>>>      GraphHopper at openstreetmap.org
>>> <mailto:GraphHopper at openstreetmap.org>
>>>      https://lists.openstreetmap.org/listinfo/graphhopper
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> GraphHopper mailing list
>>> GraphHopper at openstreetmap.org
>>> https://lists.openstreetmap.org/listinfo/graphhopper
>>>
>> _______________________________________________
>> GraphHopper mailing list
>> GraphHopper at openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/graphhopper
> 
> 
> _______________________________________________
> GraphHopper mailing list
> GraphHopper at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/graphhopper