[OSM-talk] Another point in support of ways as ordered lists

[Gabriel Ebner]

On Thu, Sep 21, 2006 at 09:21:30AM +0200, Joerg Ostertag (OSM Munich/Germany) wrote:
> The problem is: how do I take 
>  node 11 ...lat/lon=a/b tag=x
>  node 12 ...lat/lon=c/d tag=y
>  segment 9 ...from=11 to=12 tag=w
>  way ... seg=...,9,... tag=u
> into something like
>  segment 9 ... lat/lon1=a/b lat/lon2=c/d tag=u,w,x,y,
> with the complete planet.osm with 
>  - low memory consumption (<1GB) 
>  - and fast (<10 hours)?

One option is to read the planet into a flat-file database (a simple home-made
one, just lists of nodes/segs/..., lookup using binary search), adding
segment-on-node and way-on-segment data.  This takes around 10min on my dual
opteron (including bunzip2 and UTF8sanitize) and 5M of heap (for buffering).

Then you can traverse the segments in the planet like a graph (storing
traversed segments in a bit vector -- which is just 1M for the current
planet) and save the paths you generate thereby.  This takes around 8min and
1M of heap (for the bit vector).

There's just one caveat:  You need to mmap a 1.8G file, which is likely to get
much bigger in the future.  I might be able to save some space by using 32-bit
coordinates and 32-bit offsets, but this would only delay the point when the
4G barrier on 32-bit machines is reached.

  Gabriel.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://lists.openstreetmap.org/pipermail/talk/attachments/20060922/00249971/attachment.pgp>