[OSM-dev] OSM with Hadoop

[Paweł Paprota]

Hi Stephen,

I read your document - it sounds like an interesting project! Even more
so that, as you can see, not much has been done in this area so far.
Pretty much every approach to OSM data storage/handling I've seen is
Postgres + some variation of a database schema optimized towards a
specific data usage (like routing, rendering, some specific types of
read queries).

I'd be very interested to see completely different approaches to this
problem as the database size grows, it becomes extremely hard to build a
service which works with all OSM data, meaning a service with global,
not just local, coverage. Not to mention a service which works with all
*historic* OSM data - that's basically a suicide mission at this point
because (1) you probably won't fit all the data in your local dev
environment and hence (2) if you only have a small part of the data
locally then you can't easily extrapolate from that to how (or rather -
IF) your service will work if you build an infrastructure to support
full dataset.

That is quite a challenge since it really puts heavy brakes on
innovation in OSM and new services being offered.

Paweł

On Thu, Jan 8, 2015, at 16:33, Pieren wrote:
> On Fri, Jan 2, 2015 at 7:38 PM, Stephen Knox <stephenknox73 at gmail.com>
> wrote:
> 
> > So firstly I am wondering if I am missing any previous posts / research on
> > the topic?
> 
> Here are my findings about hadoop quoted in MLs:
> https://lists.openstreetmap.org/pipermail/dev/2009-August/016554.html
> https://lists.openstreetmap.org/pipermail/dev/2010-March/018648.html
> https://lists.openstreetmap.org/pipermail/dev/2010-July/019964.html
> https://lists.openstreetmap.org/pipermail/osmosis-dev/2010-August/000682.html
> https://lists.openstreetmap.org/pipermail/dev/2012-March/024468.html
> https://lists.openstreetmap.org/pipermail/talk/2012-March/062444.html
> https://lists.openstreetmap.org/pipermail/talk/2012-November/065171.html
> 
> Pieren
> 
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/dev