[OSM-dev] Embedded databases

Nolan Darilek nolan at thewordnerd.info
Tue Apr 28 14:34:16 BST 2009

I'm slowly hacking on a project that could benefit from an embedded
database, and I'm not smart enough to create my own binary format.
Furthermore, since we aren't usually under as severe constraints with
embedded resources these days, an actual embedded DB makes more sense. I
had a few questions regarding people's experiences, however.

At the moment, I'm hacking on a prototype that uses Tokyo Cabinet's
table store, and geohashes to implement geospatial searches. I'm still
working on refining the import process so haven't experimented with how
well searches work, and of course geohashes fail near the
equator/meridians, so it feels a bit kludgy. Even so, my Ruby code
imports a 13.4 meg OSM file into a 7.9M database in about a minute, so
performance doesn't seem terrible. I'll still have to implement routing,
though, and my geohash hack is about as far as I'll be able to take TC
without implementing a custom geospatial datastore, which should be
possible for someone smarter than I, but then TC's development process
is less than ideal and I don't think it's a good fit.

Has anyone used SQLite for more than rendering? There's one report of it
working well with Mapnik and another suggestion that it might not. Does
anyone have any other experiences to share? How about a working schema
and some example import code?

I've been reading references to MonetDB. I've googled, but its website
seems somewhat disorganized. There appears to be a kernel atop which
several interfaces are built? I've encountered references to geospatial
support, and routing/embedded use was mentioned here recently. Is the
geospatial support confined to a single implementation--only the SQL
support, for instance--or does it work across the board? Is there any
intro documentation anywhere? I mean, of course there are manuals and
tutorials for each component, but what about a tying-it-all-together
"this is what our architecture gives you that others don't" introduction
that helps me understand, say, how the different versions can interact?
How well does it work with Ruby? Again, either my GoogleFu is weak or
it's just not getting used much, because while it reveals the presence
of a Ruby library, I can't seem to find any "here's what's cool about
this and how to start using it" blog posts or articles. Feel free to
toss out URLs to anything I might have missed, as it certainly isn't my
intent to use this list for discussion if there are better resources
more readily available, but I'm just not seeing them. :) I've also
looked through some of the server/SQL/Xquery docs, but am not clear on
how they apply to OSM. Can an XML document be added in its XML form and
queried via SQL somehow? Or is there some intermediate import step that
makes the data available for routing? Or have I just mis-interpreted
what is currently possible? :)

Finally, are there any other embedded solutions I may have missed?
Ideally, whatever format I use should be accessible via Ruby, whether
via existing wrappers or by C libraries I can interface using FFI.

More information about the dev mailing list