[OSM-dev] Distributed Data Store

[Stefan de Konink]

Hey,

Scott Shawcroft wrote:
> My friend Jason (cced) and I are seniors at the University of Washington 
> in Computer Science and Engineering.  On your FAQ you say people 
> interested in distributing the database across multiple computers should 
> email the list.  Well, here we are.  We are currently in a distributed 
> systems capstone course during which we need to spend the quarter (until 
> mid March) on a single substantial project.

Sounds fun :) There are a lot of 'ideas' here around, geographical 
balancing etc. The standard divide and conquer methods in databases, 
etc. The main problems in OSM:

- Admins don't want to maintain multiple systems
- The fear of anything new not developed by the devs (especially if it 
is not build in Ruby)


Technical problems might be more interesting:

- Synchronization issues, even for a proxy solution; single or multiple 
write databases should distribute their data. Out of sync scenarios etc.
- Especially geo related issues, how to distribute a real geoquery.

> We're interested in trying our hand at creating a better system for 
> storing OSM data.  We're interested in what kind of computing resources 
> to design for (how many machines) and whether we can get access logs in 
> order to test our implementation against.

Related to accesslogs I found a long brick wall, it might be a better 
thing to use a requester that just makes random requests. Sources are 
available for that.

> Also, we'd love to have OSM community members involved since we're new 
> to the organization.
> 
> Lastly, I think we plan to donate our code to the community with the 
> hope that it is useful.
> 
> What do you think?

I love to brainstorm with you :) The next month I want to spend on my 
MSc thesis about improving native geospatial support in MonetDB. And the 
OSM data in it. It would ofcourse be great if the ideas comming out of 
such session can make it to State of the Map 2009.

It would be good to point you at DBslayer (the standard implementation 
or the Cherokee one), it will balance requests but with a better 
balancer could do geobalancing too :)


Yours Sincerely,

Stefan de Konink