[OSM-dev] Rantings about API 0.6

[Shaun McDonald]

On 11 Feb 2009, at 08:52, 80n wrote:

> On Wed, Feb 11, 2009 at 2:32 AM, Brett Henderson <brett at bretth.com>  
> wrote:
> Iván Sánchez Ortega wrote:
> > El Miércoles, 11 de Febrero de 2009, Shaun McDonald escribió:
> >
> >> Having a slightly less efficient import that ensures consistency  
> is a
> >> great thing, as it means that the database doesn't grow too  
> quickly.
> >>
> >
> > I fail to see why slower growth is a good thing?
> >
> Slower (perhaps "steady" is a better word) growth has its advantages  
> for
> replication.  Osmosis data extraction is fairly efficient and can
> extract data many times faster than it is currently added, but
> downstream systems can't always process it fast enough.  So while it
> would be great to be able to import data faster, there probably should
> be an upper limit to import speed allow downstream systems to keep up.
>
> This is not a valid reason.
>
> Speaking as one of the downstream consumers of the Osmosis diffs we  
> can take it as fast as you can make it.
>


Thinking about it for a bit longer, wouldn't you like to be able to  
predict when you need to increase the storage capacity, or change an  
algorithm to be able to cope with the increased data. Or what about  
being able to predict when we need to upscale the main database server  
or run another donation drive for more servers?

That's why I wouldn't want to suddenly see a week of the API being  
hammered with maximal data imports.

Shaun

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20090211/dbb9c0c1/attachment.html>