[OSM-dev] Daily Diff parsing - tile expiring

Roeland Douma unix at rullzer.com
Wed Aug 20 23:54:16 BST 2008

Hi all,

I'm working on a script to expire tiles bases on (daily) diffs. This so we can 
re-render the tiles with changes and keep the tiles without for a longer 
period of time. Which will allow us to have more up to date maps. And not 
have the server do more work than necessary since tiles that have no changes 
do not have to be re-renderd.

This is easily done for nodes since they contain Lon/Lat tags. So for zoom a 
till b the tiles can be calculated and removed.

However the problem starts when ways are parsed. Since ways only contain refs 
to nodes. So no Lat/Lon info is provided.

The average daily diff contains > 700.000 node refs in way tags. 

I have tried to get all the nodes (1000 nodes at the same time) from the 
OSM-API. This solution provided me straight away with the Lon/Lat for the 
nodes. However this took a very very long time. And it is not very nice to do 
that on a daily basis. (also for the load on the server). 

An other solution I tried was running a local copy of the database. running 
oms2pgsql in slim-mode. However this took a long time. And when running the 
query (with >700.000 nodes) also took a very very long time. Now of course 
the Lat/Lon have to be parsed but that is not the problem.

As you might have guessed both solutions do not work. Or at least not good 
enough for my taste. I would like a simple (or at least fast) solution.

I think that adding the Lat/Lon tag to node references in the ways is not an 
option since it would require a huge amount of queries on the database when 
creating the diffs. The positive side of this would that it would only have 
to be done once.

Something else would be to have some sort of bounding box tag at the ways. 
This is not optimal in tile expiring but is better than it is done now.

Anybody with idea's on this matter?


More information about the dev mailing list