[OSM-dev] Daily Diff parsing - tile expiring

Roeland Douma unix at rullzer.com
Thu Aug 21 07:55:08 BST 2008


On Thursday 21 August 2008 01:42:53 Frederik Ramm wrote:
> Hi,
>
> > I think that adding the Lat/Lon tag to node references in the ways is not
> > an option since it would require a huge amount of queries on the database
> > when creating the diffs. The positive side of this would that it would
> > only have to be done once.
>
> If you are happy with supporting only zoom level 16 granularity, you can
> store the tile x and tile y coordinates for each node, that's only 4
> bytes per node.

Zoom level 16 is fine. Or when we are at it level 17 or 18. Since other zoom 
levels can be calculated for that node then. However then of course you would 
need 6 bytes per node. But that is something that can be discussed later or 
maybe even tweaked by settings.

>
> If you create a file that contains four bytes for every node, indexed
> simply by node id (node #1 x at file position 0, node #1 y at 2, node #2
> x at 4 etc.), then your file will have about 1.2 Gig for the current
> planet. When processing a diff you would simply mmap that file,
> effectively loading into memory and granting very fast access to node
> positions.

Sounds good. I'll look into mmap. Never had to use it before. 

> This wastes some memory because it reserves space for deleted nodes as
> well, but when I last looked the number of "deleted" node ids was about
> 20%, and this is an acceptable overhead - otherwise you would have to
> use something like a hashtable to index your node list which would use
> more than that.

This is true. However it is only 1.2Gig right now so a simple/fast setup will 
probably be faster.

Thanks,
Roeland




More information about the dev mailing list