[Talk-GB] Efficient processing of map data for rendering (BOINC).

[Matt Amos]

On Thu, Jan 22, 2009 at 10:26 AM, Steve Hill <steve at nexusuk.org> wrote:
> On Wed, 21 Jan 2009, Chris Andrew wrote:
>
>> I notice that people often mention the delay in map edits being
>> applied and made _live_.
>
> On a related note...
>
> For OpenPisteMap, I apply the diffs to the PostGIS DB every minute, so it
> only lags behind the live data by a few minutes.  However, it doesn't
> currently automatically expire any tiles from the cache, so it won't
> re-render a tile after the data has been changed.

this might be helpful
http://svn.openstreetmap.org/applications/utils/export/tile_expiry/

> I'm currently working on modifying osm2pgsql to create a list of tiles
> that have been changed as it applies the diffs so that they can be removed
> from the cache (and thus re-rendered on the fly when someone requests
> them).

the above script expires meta-tiles with minutely updates. see the
blog post here http://blog.cloudmade.com/2009/01/23/nearly-live-tiles/
and the tile server itself http://matt.sandbox.cloudmade.com/

>> With the OSM community growing by the day, this problem can only get
>> bigger.  Does anyone know whether anyone has consider using a
>> distributed client [1] such as BOINC [2] to do the _number crunching_?
>
> From my experience, the number crunching doesn't really seem to be the
> limiting factor - database I/O is the biggest overhead for OpenPisteMap
> (although that may be partly down to the massive amount of SRTM contours
> data it has to handle while rendering each tile).

+1

this is one case where one big raid array is much better than many
distributed disks.

cheers,

matt