[OSM-dev] Osmosis crash for large map import to osmbin format

Curt Nowak cnowak at bwl.uni-hildesheim.de
Thu May 28 12:52:53 BST 2009


Great timing!
Thanks a lot, Marcus and Brett. Can't wait to try the new version.


-----Ursprüngliche Nachricht-----
Von: marcus.wolschon at googlemail.com [mailto:marcus.wolschon at googlemail.com] 
Gesendet: Donnerstag, 28. Mai 2009 13:49
An: Brett Henderson
Cc: Curt Nowak; dev at openstreetmap.org
Betreff: Re: [OSM-dev] Osmosis crash for large map import to osmbin format


On Thu, 28 May 2009 17:37:11 +1000, Brett Henderson <brett at bretth.com>
wrote:
> Curt Nowak wrote:
>> Hi all,
>>
>> when trying to import a large map (Germany) into the osmbin format[1] 
>> using osmosis, the program crashes with the output shown at the end 
>> of this mail. I'd like to use the map of Germany for simple routing 
>> problems with the help of libosm[2]. So the osmbin format seemed like 
>> the best idea. I had already filtered the map before, running the 
>> appropriate osmosis tasks on the map. Still, the "Size exceeds 
>> Integer.MAX_VALUE" part makes me wonder whether this is a permanent 
>> issue for large maps (see also [3]).
>> I am not sure where to go next? Is there a possible fix? Should I switch
>> to a Postgres DB for the map data? Is there a way to make libosm work
>> with Postgres?
>> Any hints are most welcome,
>>   
> It looks like the osmbin code is attempting to memory map the file.  
> In
> java, memory mapping is limited to a 2GB file (at least on a 32-bit JVM, 
> not sure about 64-bit).  I don't know enough about the internals of the 
> osmbin tasks to know if this is possible to fix.

I fixed this 2 days ago in the SVN-version of LibOSM in Traveling Salesman. If you build it from the SVN-sources it should work now by using conventional IO not only after IOExceptions but also after IllegalArgumentExceptions in map(). There will be a new binary version next week.

Marcus




More information about the dev mailing list