[OSM-dev] Fwd: Re: OSM and MongoDB

[Ian Dees]

Yes, one document per node/way/relation.

On Tue, Apr 12, 2011 at 3:47 PM, Steve Coast <steve at asklater.com> wrote:

>  how was the data put in the db though? 1 document per node?
>
>
> On 4/12/2011 1:39 PM, Nolan Darilek wrote:
>
> Oopse, meant for this to go to the whole list.
>
>
>
> -------- Original Message --------  Subject: Re: [OSM-dev] OSM and MongoDB  Date:
> Tue, 12 Apr 2011 15:26:41 -0500  From: Nolan Darilek
> <nolan at thewordnerd.info> <nolan at thewordnerd.info>  To: Ian Dees
> <ian.dees at gmail.com> <ian.dees at gmail.com>
>
> I had/am having a somewhat bad experience storing OSM data in MongoDB.
>
> Initially I stored all map data in MongoDB, but queries took ages. The same
> queries that happen in 100-200 MS now often took nearly a second.
> Additionally, some took upwards of 5, and I even found spots on my map
> sparsely populated with points, but which reliably performed the queries I
> need in 30+ seconds.
>
> I filed a thorough bug in their tracker, including a dataset and queries
> that reliably duplicated the issue. It was marked wontfix, I abandoned
> MongoDB, and it was apparently re-opened and fixed several months later. So
> perhaps it's a non-issue now.
>
> I'm still using MongoDB for part of my current project, user POI storage.
> It does indeed use geohashes, and I'm experiencing strange accuracy issues.
> My platform is pedestrian navigation with many small distance queries.
> Points in the non-MongoDB dataset are reliably detected in a radius roughly
> 100 meters around the traveler. Points in MongoDB queried with the same
> bounding boxes don't appear until they're within 30-40 meters. I recently
> updated from an older version to a new build of 1.8. The older version
> widely varied the detection range. Some points were detected 100 or so
> meters out, while others weren't picked up until 30 or so. It was always the
> same points, too. The point for my apartment remains reliably visible for
> ~100 meters or so, while the corner store and restaurant didn't appear until
> I was very close. 1.8 at least appears to be consistent, always detecting at
> 30 meters or so. I can only assume that this is a geohash oddity that only
> appears for very small differences, something that works out to rounding
> error for larger values.
>
> I like MongoDB for many things, but not for geospatial data more
> complicated than a series of points. I'm working on migrating user/POI
> storage to a geospatial store.
>
>
> On 04/12/2011 01:20 PM, Ian Dees wrote:
>
> Yep, and I think Mongo uses geohashes as their index behind the scenes. One
> of the problems with that, though, is they have some arbitrary length that
> they compute the geohash to and when you have lots of points (as OSM data
> does) the buckets they're searching are very full.
>
> On Tue, Apr 12, 2011 at 1:00 PM, Steve Coast <steve at asklater.com> wrote:
>
>>  bbox queries using the built in spatial indexing presumably? OSM has it's
>> own magical bitmask for that, that may also be as fast in mongo, who knows.
>>
>>
>> On 4/11/2011 5:58 PM, Ian Dees wrote:
>>
>>  On Mon, Apr 11, 2011 at 6:36 PM, Sergey Galuzo <sergal at microsoft.com>wrote:
>>
>>>  Hi,
>>>
>>>
>>>
>>> I am working on evaluation of MongoDB for several storage solutions at
>>> hand. Some of them resemble current OSM editing database. I have heard that
>>> OSM dev is/was evaluating MongoDB also. I was wondering whether it possible
>>> to share the findings?
>>>
>>>
>>>
>>
>>  In my experimentation with MongoDB (seen here:
>> https://github.com/iandees/mongosm/) I found it to be very slow. Inserts
>> were speedy, but bounding-box queries took a long time.
>>
>>  The most recent dev version of MongoDB includes "multi-location
>> documents" support:
>>
>> http://www.mongodb.org/display/DOCS/Geospatial+Indexing#GeospatialIndexing-MultilocationDocuments
>>
>>  This would allow a single way document to be indexed at multiple
>> locations and vastly speed up the map query.
>>
>>
>> _______________________________________________
>> dev mailing list
>> dev at openstreetmap.orghttp://lists.openstreetmap.org/listinfo/dev
>>
>>
>> _______________________________________________
>> dev mailing list
>> dev at openstreetmap.org
>> http://lists.openstreetmap.org/listinfo/dev
>>
>>
>
> _______________________________________________
> dev mailing listdev at openstreetmap.orghttp://lists.openstreetmap.org/listinfo/dev
>
>
>
> _______________________________________________
> dev mailing listdev at openstreetmap.orghttp://lists.openstreetmap.org/listinfo/dev
>
>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20110412/bf19b10b/attachment-0001.html>