[OSM-dev] Embedded databases

Tomas Kolda kolda at web2net.cz
Tue Apr 28 17:12:23 BST 2009

As I wrote in another thread I tried sqlite to do rendering along with 
mapnik. There is no problem to do it even with builtin rtree spatial 
index. I can try to create some SVN repos to share code, but it was just 
for playing (not for production). I tweaked postgress datasource in 
mapnik and make my own implementation of storing GIS geometries, because 
of no dependecy needs. Also look to existing GIS extension for sqlite.


Importing into sqlite is very fast. I'm using simple C++ importer with 
expat parser and insert all data in one transaction. Sqlite has also 
very little size overhead (about 5x size than bzipped OSM).

For my another project I use my own datastore, because it is much more 
faster than universal databases. It uses leveled Z-index. For searching 
I'm using litmax bigmin method. Its performance is great.

So you must choose between speed and simple approach. For routing you 
will need some special data store to do it fast.

Nolan Darilek napsal(a):
> I'm slowly hacking on a project that could benefit from an embedded
> database, and I'm not smart enough to create my own binary format.
> Furthermore, since we aren't usually under as severe constraints with
> embedded resources these days, an actual embedded DB makes more sense. I
> had a few questions regarding people's experiences, however.
> At the moment, I'm hacking on a prototype that uses Tokyo Cabinet's
> table store, and geohashes to implement geospatial searches. I'm still
> working on refining the import process so haven't experimented with how
> well searches work, and of course geohashes fail near the
> equator/meridians, so it feels a bit kludgy. Even so, my Ruby code
> imports a 13.4 meg OSM file into a 7.9M database in about a minute, so
> performance doesn't seem terrible. I'll still have to implement routing,
> though, and my geohash hack is about as far as I'll be able to take TC
> without implementing a custom geospatial datastore, which should be
> possible for someone smarter than I, but then TC's development process
> is less than ideal and I don't think it's a good fit.
> Has anyone used SQLite for more than rendering? There's one report of it
> working well with Mapnik and another suggestion that it might not. Does
> anyone have any other experiences to share? How about a working schema
> and some example import code?
> I've been reading references to MonetDB. I've googled, but its website
> seems somewhat disorganized. There appears to be a kernel atop which
> several interfaces are built? I've encountered references to geospatial
> support, and routing/embedded use was mentioned here recently. Is the
> geospatial support confined to a single implementation--only the SQL
> support, for instance--or does it work across the board? Is there any
> intro documentation anywhere? I mean, of course there are manuals and
> tutorials for each component, but what about a tying-it-all-together
> "this is what our architecture gives you that others don't" introduction
> that helps me understand, say, how the different versions can interact?
> How well does it work with Ruby? Again, either my GoogleFu is weak or
> it's just not getting used much, because while it reveals the presence
> of a Ruby library, I can't seem to find any "here's what's cool about
> this and how to start using it" blog posts or articles. Feel free to
> toss out URLs to anything I might have missed, as it certainly isn't my
> intent to use this list for discussion if there are better resources
> more readily available, but I'm just not seeing them. :) I've also
> looked through some of the server/SQL/Xquery docs, but am not clear on
> how they apply to OSM. Can an XML document be added in its XML form and
> queried via SQL somehow? Or is there some intermediate import step that
> makes the data available for routing? Or have I just mis-interpreted
> what is currently possible? :)
> Finally, are there any other embedded solutions I may have missed?
> Ideally, whatever format I use should be accessible via Ruby, whether
> via existing wrappers or by C libraries I can interface using FFI.
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev

More information about the dev mailing list