[OSM-dev] Deriving Change Sets
Raphaël Jacquot
sxpert at sxpert.org
Sun Jul 1 07:01:01 BST 2007
Brett Henderson wrote:
> Frederik Ramm wrote:
>> This is something I would really love to see get off the ground. (I
>> remember an evening at the Essen meeting where I complained about the
>> weekly dump, and Nick Black said something like "well we could do it
>> daily", and I said "daily is not enough", and he went "well one could
>> do hourly with proper equiment" and I said "dumps, dumps, dumps, I
>> don't want no stupid dumps, I want live data..." - a discussion ensued
>> about what you'd possibly need live data for, but until today I
>> maintain that we should just provide data as live as possible without
>> asking what people want to use it for.)
> At the risk of rambling, I feel the same way. Dumps are extremely
> valuable, simple to implement, and probably the right solution for many
> problems. But they have their limitations such as:
> * Data is always out of date.
> * Attempting to increase dump frequency adds significant load to the
> data source (ie. the database and other osm server infrastructure).
> * Data is re-transmitted every time a new dump is requested adding to
> network utilisation.
> * At the receiving end, the complete dataset must be processed every
> time a new dump is utilised.
+1
> A method of synchronisation avoids the problems above.
>
> A regular synchronisation mechanism enables some of the following
> possibilities:
> * Some end users such as mapnik can produce more up-to-date maps and can
> avoid significant processing by only importing changes.
> * In order to alleviate load from the current API and primary database,
> current users of the API such as tiles at home *may* (this may be
> controversial :-) be able to switch to using a "near live" feed.
> * Tasks can respond to changes more effectively. To use tiles at home as
> an example, the replication task from primary database to rendering
> database could examine the nodes and automatically flag tiles that need
> re-rendering thus eliminating the need to manually request tile re-renders.
> * Read-only tasks without hard real-time requirements can be moved off
> the API (and core database) thus allowing the core infrastructure to
> scale to a larger number of users.
+1
>> But I always thought - as long as "near live" feeds are what one wants
>> - it would be much cheaper in terms of processing power to simply log
>> each change as performed by the API.
> Agree.
that's what I've been proposing in the IRC channel
More information about the dev
mailing list