[OSM-dev] [Talk-GB] Motorway and Railway progress (simplifying OSM data)

Jon Burgess jburgess at uklinux.net
Mon Nov 6 23:19:30 GMT 2006

On Mon, 2006-11-06 at 10:13 +0000, Nick Burch wrote:

> You should take a look at on of the scripts in 
> http://svn.openstreetmap.org/utils/planet.osm/perl/, eg
> http://svn.openstreetmap.org/utils/planet.osm/perl/planetosm-excerpt-area.pl
> You'll want to use Bit::Vector instead of a regular perl hash wherever you 
> can. You may also find you need to do two passes, one to figure out what 
> nodes/segments/ways you want (populating a nice and efficient 
> Bit::Vector), then a second to actually do your filtering (so you only 
> store lots of data on things of interest)
> I managed to get the memory use on my scripts to drop from 3gb to under a 
> hundred meg, just by switching to Bit::Vector where I could.

I can probably make use of bit vectors in one or two places to save some
memory but I don't think I can save much with my current algorithm. 

The current code relies on more than just knowing whether a node is
present or not. It has to represent a set of closely spaced nodes from
the input as being replaced by a single node in the output. This is
tricky to do without storing at least one ID per node of the input. The
only alternative I can think of is to iterate the OSM data a huge number
of times (perhaps once per node) which isn't practical either.


More information about the dev mailing list