[OSM-dev] Fwd: Re: OSM and MongoDB

[Steve Coast]

how was the data put in the db though? 1 document per node?

On 4/12/2011 1:39 PM, Nolan Darilek wrote:
> Oopse, meant for this to go to the whole list.
>
>
>
> -------- Original Message --------
> Subject: 	Re: [OSM-dev] OSM and MongoDB
> Date: 	Tue, 12 Apr 2011 15:26:41 -0500
> From: 	Nolan Darilek <nolan at thewordnerd.info>
> To: 	Ian Dees <ian.dees at gmail.com>
>
>
>
> I had/am having a somewhat bad experience storing OSM data in MongoDB.
>
> Initially I stored all map data in MongoDB, but queries took ages. The 
> same queries that happen in 100-200 MS now often took nearly a second. 
> Additionally, some took upwards of 5, and I even found spots on my map 
> sparsely populated with points, but which reliably performed the 
> queries I need in 30+ seconds.
>
> I filed a thorough bug in their tracker, including a dataset and 
> queries that reliably duplicated the issue. It was marked wontfix, I 
> abandoned MongoDB, and it was apparently re-opened and fixed several 
> months later. So perhaps it's a non-issue now.
>
> I'm still using MongoDB for part of my current project, user POI 
> storage. It does indeed use geohashes, and I'm experiencing strange 
> accuracy issues. My platform is pedestrian navigation with many small 
> distance queries. Points in the non-MongoDB dataset are reliably 
> detected in a radius roughly 100 meters around the traveler. Points in 
> MongoDB queried with the same bounding boxes don't appear until 
> they're within 30-40 meters. I recently updated from an older version 
> to a new build of 1.8. The older version widely varied the detection 
> range. Some points were detected 100 or so meters out, while others 
> weren't picked up until 30 or so. It was always the same points, too. 
> The point for my apartment remains reliably visible for ~100 meters or 
> so, while the corner store and restaurant didn't appear until I was 
> very close. 1.8 at least appears to be consistent, always detecting at 
> 30 meters or so. I can only assume that this is a geohash oddity that 
> only appears for very small differences, something that works out to 
> rounding error for larger values.
>
> I like MongoDB for many things, but not for geospatial data more 
> complicated than a series of points. I'm working on migrating user/POI 
> storage to a geospatial store.
>
>
> On 04/12/2011 01:20 PM, Ian Dees wrote:
>> Yep, and I think Mongo uses geohashes as their index behind the 
>> scenes. One of the problems with that, though, is they have some 
>> arbitrary length that they compute the geohash to and when you have 
>> lots of points (as OSM data does) the buckets they're searching are 
>> very full.
>>
>> On Tue, Apr 12, 2011 at 1:00 PM, Steve Coast <steve at asklater.com 
>> <mailto:steve at asklater.com>> wrote:
>>
>>     bbox queries using the built in spatial indexing presumably? OSM
>>     has it's own magical bitmask for that, that may also be as fast
>>     in mongo, who knows.
>>
>>
>>     On 4/11/2011 5:58 PM, Ian Dees wrote:
>>>     On Mon, Apr 11, 2011 at 6:36 PM, Sergey Galuzo
>>>     <sergal at microsoft.com <mailto:sergal at microsoft.com>> wrote:
>>>
>>>         Hi,
>>>
>>>         I am working on evaluation of MongoDB for several storage
>>>         solutions at hand. Some of them resemble current OSM editing
>>>         database. I have heard that OSM dev is/was evaluating
>>>         MongoDB also. I was wondering whether it possible to share
>>>         the findings?
>>>
>>>
>>>     In my experimentation with MongoDB (seen here:
>>>     https://github.com/iandees/mongosm/) I found it to be very slow.
>>>     Inserts were speedy, but bounding-box queries took a long time.
>>>
>>>     The most recent dev version of MongoDB includes "multi-location
>>>     documents" support:
>>>     http://www.mongodb.org/display/DOCS/Geospatial+Indexing#GeospatialIndexing-MultilocationDocuments
>>>
>>>     This would allow a single way document to be indexed at multiple
>>>     locations and vastly speed up the map query.
>>>
>>>
>>>     _______________________________________________
>>>     dev mailing list
>>>     dev at openstreetmap.org <mailto:dev at openstreetmap.org>
>>>     http://lists.openstreetmap.org/listinfo/dev
>>
>>     _______________________________________________
>>     dev mailing list
>>     dev at openstreetmap.org <mailto:dev at openstreetmap.org>
>>     http://lists.openstreetmap.org/listinfo/dev
>>
>>
>>
>> _______________________________________________
>> dev mailing list
>> dev at openstreetmap.org
>> http://lists.openstreetmap.org/listinfo/dev
>
>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20110412/ee03646b/attachment.html>