[Talk-GB] Efficient processing of map data for rendering (BOINC).

[Steve Hill]

On Wed, 21 Jan 2009, Chris Andrew wrote:

> I notice that people often mention the delay in map edits being
> applied and made _live_.

On a related note...

For OpenPisteMap, I apply the diffs to the PostGIS DB every minute, so it 
only lags behind the live data by a few minutes.  However, it doesn't 
currently automatically expire any tiles from the cache, so it won't 
re-render a tile after the data has been changed.

I'm currently working on modifying osm2pgsql to create a list of tiles 
that have been changed as it applies the diffs so that they can be removed 
from the cache (and thus re-rendered on the fly when someone requests 
them).

My initial, very simplistic attempt was rather unsuccessful though - I 
made osm2pgsql calculate bounding boxes around every object being deleted 
or created.  However, for some objects the bounding box can be extremely 
large (especially relations) so it expires a very large number of tiles.

I think my next attempt will involve calculating which tiles a LINESTRING
intersects.  However, I'm not sure what to do about POLYGONs - 
technically, every tile within the polygon should be expired, but that 
could be a potentially huge area.  Maybe the answer is simply to put in 
some sanity checks that ignore polygons that cover massive areas.

> With the OSM community growing by the day, this problem can only get
> bigger.  Does anyone know whether anyone has consider using a
> distributed client [1] such as BOINC [2] to do the _number crunching_?

>From my experience, the number crunching doesn't really seem to be the 
limiting factor - database I/O is the biggest overhead for OpenPisteMap 
(although that may be partly down to the massive amount of SRTM contours 
data it has to handle while rendering each tile).

  - Steve
    xmpp:steve at nexusuk.org   sip:steve at nexusuk.org   http://www.nexusuk.org/

      Servatis a periculum, servatis a maleficum - Whisper, Evanescence