[OSM-dev] Fwd: Re: OSM and MongoDB

Ian Dees ian.dees at gmail.com
Tue Apr 12 21:52:14 BST 2011


Yep.

On Tue, Apr 12, 2011 at 3:51 PM, Steve Coast <steve at asklater.com> wrote:

>  and using the builtin spatial index?
>
>
>
> On 4/12/2011 1:50 PM, Ian Dees wrote:
>
> Yes, one document per node/way/relation.
>
> On Tue, Apr 12, 2011 at 3:47 PM, Steve Coast <steve at asklater.com> wrote:
>
>>  how was the data put in the db though? 1 document per node?
>>
>>
>> On 4/12/2011 1:39 PM, Nolan Darilek wrote:
>>
>> Oopse, meant for this to go to the whole list.
>>
>>
>>
>> -------- Original Message --------  Subject: Re: [OSM-dev] OSM and
>> MongoDB  Date: Tue, 12 Apr 2011 15:26:41 -0500  From: Nolan Darilek
>> <nolan at thewordnerd.info> <nolan at thewordnerd.info>  To: Ian Dees
>> <ian.dees at gmail.com> <ian.dees at gmail.com>
>>
>> I had/am having a somewhat bad experience storing OSM data in MongoDB.
>>
>> Initially I stored all map data in MongoDB, but queries took ages. The
>> same queries that happen in 100-200 MS now often took nearly a second.
>> Additionally, some took upwards of 5, and I even found spots on my map
>> sparsely populated with points, but which reliably performed the queries I
>> need in 30+ seconds.
>>
>> I filed a thorough bug in their tracker, including a dataset and queries
>> that reliably duplicated the issue. It was marked wontfix, I abandoned
>> MongoDB, and it was apparently re-opened and fixed several months later. So
>> perhaps it's a non-issue now.
>>
>> I'm still using MongoDB for part of my current project, user POI storage.
>> It does indeed use geohashes, and I'm experiencing strange accuracy issues.
>> My platform is pedestrian navigation with many small distance queries.
>> Points in the non-MongoDB dataset are reliably detected in a radius roughly
>> 100 meters around the traveler. Points in MongoDB queried with the same
>> bounding boxes don't appear until they're within 30-40 meters. I recently
>> updated from an older version to a new build of 1.8. The older version
>> widely varied the detection range. Some points were detected 100 or so
>> meters out, while others weren't picked up until 30 or so. It was always the
>> same points, too. The point for my apartment remains reliably visible for
>> ~100 meters or so, while the corner store and restaurant didn't appear until
>> I was very close. 1.8 at least appears to be consistent, always detecting at
>> 30 meters or so. I can only assume that this is a geohash oddity that only
>> appears for very small differences, something that works out to rounding
>> error for larger values.
>>
>> I like MongoDB for many things, but not for geospatial data more
>> complicated than a series of points. I'm working on migrating user/POI
>> storage to a geospatial store.
>>
>>
>> On 04/12/2011 01:20 PM, Ian Dees wrote:
>>
>> Yep, and I think Mongo uses geohashes as their index behind the scenes.
>> One of the problems with that, though, is they have some arbitrary length
>> that they compute the geohash to and when you have lots of points (as OSM
>> data does) the buckets they're searching are very full.
>>
>> On Tue, Apr 12, 2011 at 1:00 PM, Steve Coast <steve at asklater.com> wrote:
>>
>>>  bbox queries using the built in spatial indexing presumably? OSM has
>>> it's own magical bitmask for that, that may also be as fast in mongo, who
>>> knows.
>>>
>>>
>>> On 4/11/2011 5:58 PM, Ian Dees wrote:
>>>
>>>  On Mon, Apr 11, 2011 at 6:36 PM, Sergey Galuzo <sergal at microsoft.com>wrote:
>>>
>>>>  Hi,
>>>>
>>>>
>>>>
>>>> I am working on evaluation of MongoDB for several storage solutions at
>>>> hand. Some of them resemble current OSM editing database. I have heard that
>>>> OSM dev is/was evaluating MongoDB also. I was wondering whether it possible
>>>> to share the findings?
>>>>
>>>>
>>>>
>>>
>>>  In my experimentation with MongoDB (seen here:
>>> https://github.com/iandees/mongosm/) I found it to be very slow. Inserts
>>> were speedy, but bounding-box queries took a long time.
>>>
>>>  The most recent dev version of MongoDB includes "multi-location
>>> documents" support:
>>>
>>> http://www.mongodb.org/display/DOCS/Geospatial+Indexing#GeospatialIndexing-MultilocationDocuments
>>>
>>>  This would allow a single way document to be indexed at multiple
>>> locations and vastly speed up the map query.
>>>
>>>
>>> _______________________________________________
>>> dev mailing list
>>> dev at openstreetmap.orghttp://lists.openstreetmap.org/listinfo/dev
>>>
>>>
>>> _______________________________________________
>>> dev mailing list
>>> dev at openstreetmap.org
>>> http://lists.openstreetmap.org/listinfo/dev
>>>
>>>
>>
>> _______________________________________________
>> dev mailing listdev at openstreetmap.orghttp://lists.openstreetmap.org/listinfo/dev
>>
>>
>>
>> _______________________________________________
>> dev mailing listdev at openstreetmap.orghttp://lists.openstreetmap.org/listinfo/dev
>>
>>
>> _______________________________________________
>> dev mailing list
>> dev at openstreetmap.org
>> http://lists.openstreetmap.org/listinfo/dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20110412/68d4433f/attachment-0001.html>


More information about the dev mailing list