[OSM-dev] OSM with Hadoop

[Frederik Ramm]

Hi,

On 01/14/2015 11:16 AM, Paweł Paprota wrote:
> Main central server with a single multi-TB-sized database
> somehow screams single point of failure to me...

... until you learn that there's actually an active replication, based
on technology that has been tried and proven.

I'm not saying brush away parallel storage, just don't assume from the
outset that it will magically solve problems without incurring others.
Also, we're not the only people operating large relational databases,
and development on that front is also occurring, with the potential of
an efficient multi-master replication system somewhere down the line
leading to a different kind of also-parallel infrastructure. Where the
failure points of each of these architectures lie and how
cost-effective, maintenance intensive, or error prone they are, is
indeed a good subject for analysis.

All I'm saying that it's worth to apply a scientific approach (i.e.
looking at facts) rather than a geek approach (i.e. looking at cool new
technology and shiny lights). And from Stephen's paper I had the
impression that we wouldn't easily be blinded by those shiny lights
which is a good thing.

(In our particular case, the amount of editing that occurs rises far
slower than the amount of data that we have collected, which means that
it is not unlikely that we will be able to work with a "centralised
writing, distributed reading" approach for quite a while still.)

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frederik at remote.org  ##  N49°00'09" E008°23'33"