[OSM-dev] Daily Diff parsing - tile expiring
unix at rullzer.com
Thu Aug 21 07:55:08 BST 2008
On Thursday 21 August 2008 01:42:53 Frederik Ramm wrote:
> > I think that adding the Lat/Lon tag to node references in the ways is not
> > an option since it would require a huge amount of queries on the database
> > when creating the diffs. The positive side of this would that it would
> > only have to be done once.
> If you are happy with supporting only zoom level 16 granularity, you can
> store the tile x and tile y coordinates for each node, that's only 4
> bytes per node.
Zoom level 16 is fine. Or when we are at it level 17 or 18. Since other zoom
levels can be calculated for that node then. However then of course you would
need 6 bytes per node. But that is something that can be discussed later or
maybe even tweaked by settings.
> If you create a file that contains four bytes for every node, indexed
> simply by node id (node #1 x at file position 0, node #1 y at 2, node #2
> x at 4 etc.), then your file will have about 1.2 Gig for the current
> planet. When processing a diff you would simply mmap that file,
> effectively loading into memory and granting very fast access to node
Sounds good. I'll look into mmap. Never had to use it before.
> This wastes some memory because it reserves space for deleted nodes as
> well, but when I last looked the number of "deleted" node ids was about
> 20%, and this is an acceptable overhead - otherwise you would have to
> use something like a hashtable to index your node list which would use
> more than that.
This is true. However it is only 1.2Gig right now so a simple/fast setup will
probably be faster.
More information about the dev