[OSM-dev] Gosmore

[Robert (Jamie) Munro]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nic Roets wrote:
> I'm sorry that the project has ground to a halt, but I've been on
> vacation (and now sit on a pile of new data that I want to edit and
> upload while I can still picture the scenery).
> 
>> I've found it impossible to get "gosmore --rebuild" to complete a full
> 
> You're running gosmore rebuild.
> 
>> planet with more recent versions of gosmore. It starts of taking lots of
> 
> OSM files generated from TIGER data does not have all the nodes at the
> start. Recent version cope with this through extra hash tables and the
> unused space in them increase the RAM usage.
> 
> The unused space can be reduced by changing MAX_NODES, MAX_SEGMENTS
> and to a lesser extent MAX_NAMED_NODES. I have spent 0 time optimizing
> these variables.

In fact I had reduced those variables to the numbers of nodes and
segments I found in my planet file using grep. Before I did that, just
attempting to run the program would lock my computer up so that I had to
power cycle it to get any response. This kind of implies there is
something wrong with MacOS X virtual memory handling.

> To be sure where the bottleneck occurs look at the message printed out
> (and perhaps the point where bunzip finishes, implying the whole file
> has been read).

All I get is:

$ 7z e -so planet-070620.osm.7z | ./gosmore rebuild

7-Zip 4.33 beta  Copyright (c) 1999-2006 Igor Pavlov  2006-02-05
p7zip Version 4.33 (locale=C,Utf16=off,HugeFiles=on)

Processing archive: planet-070620.osm.7z
./gosmore is in the public domain and comes without warrantee
Reading nodes...

Extracting  planet-070620.osm


That's all it says.

It runs for about 15-20 minutes, during which time 7z and gosmore
alternate having 90%-100% cpu usage, and swapping is reasonable. Then,
at about 9 minutes of gosmore cpu time (according to activity monitor),
CPU usage drops to between 1% and 5%, and swapping goes mad.

Robert (Jamie) Munro
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGlhI3z+aYVHdncI0RAqfPAKDCYkFWo5b3Gu21gilDLfl4tfsRpwCg54tr
cRjVUNvGjsKbqPVXc9SeXZU=
=V0Fm
-----END PGP SIGNATURE-----