[OSM-dev] Fwd: Re: OSM and MongoDB

[Steve Coast]

and using the builtin spatial index?


On 4/12/2011 1:50 PM, Ian Dees wrote:
> Yes, one document per node/way/relation.
>
> On Tue, Apr 12, 2011 at 3:47 PM, Steve Coast <steve at asklater.com 
> <mailto:steve at asklater.com>> wrote:
>
>     how was the data put in the db though? 1 document per node?
>
>
>     On 4/12/2011 1:39 PM, Nolan Darilek wrote:
>>     Oopse, meant for this to go to the whole list.
>>
>>
>>
>>     -------- Original Message --------
>>     Subject: 	Re: [OSM-dev] OSM and MongoDB
>>     Date: 	Tue, 12 Apr 2011 15:26:41 -0500
>>     From: 	Nolan Darilek <nolan at thewordnerd.info>
>>     <mailto:nolan at thewordnerd.info>
>>     To: 	Ian Dees <ian.dees at gmail.com> <mailto:ian.dees at gmail.com>
>>
>>
>>
>>     I had/am having a somewhat bad experience storing OSM data in
>>     MongoDB.
>>
>>     Initially I stored all map data in MongoDB, but queries took
>>     ages. The same queries that happen in 100-200 MS now often took
>>     nearly a second. Additionally, some took upwards of 5, and I even
>>     found spots on my map sparsely populated with points, but which
>>     reliably performed the queries I need in 30+ seconds.
>>
>>     I filed a thorough bug in their tracker, including a dataset and
>>     queries that reliably duplicated the issue. It was marked
>>     wontfix, I abandoned MongoDB, and it was apparently re-opened and
>>     fixed several months later. So perhaps it's a non-issue now.
>>
>>     I'm still using MongoDB for part of my current project, user POI
>>     storage. It does indeed use geohashes, and I'm experiencing
>>     strange accuracy issues. My platform is pedestrian navigation
>>     with many small distance queries. Points in the non-MongoDB
>>     dataset are reliably detected in a radius roughly 100 meters
>>     around the traveler. Points in MongoDB queried with the same
>>     bounding boxes don't appear until they're within 30-40 meters. I
>>     recently updated from an older version to a new build of 1.8. The
>>     older version widely varied the detection range. Some points were
>>     detected 100 or so meters out, while others weren't picked up
>>     until 30 or so. It was always the same points, too. The point for
>>     my apartment remains reliably visible for ~100 meters or so,
>>     while the corner store and restaurant didn't appear until I was
>>     very close. 1.8 at least appears to be consistent, always
>>     detecting at 30 meters or so. I can only assume that this is a
>>     geohash oddity that only appears for very small differences,
>>     something that works out to rounding error for larger values.
>>
>>     I like MongoDB for many things, but not for geospatial data more
>>     complicated than a series of points. I'm working on migrating
>>     user/POI storage to a geospatial store.
>>
>>
>>     On 04/12/2011 01:20 PM, Ian Dees wrote:
>>>     Yep, and I think Mongo uses geohashes as their index behind the
>>>     scenes. One of the problems with that, though, is they have some
>>>     arbitrary length that they compute the geohash to and when you
>>>     have lots of points (as OSM data does) the buckets they're
>>>     searching are very full.
>>>
>>>     On Tue, Apr 12, 2011 at 1:00 PM, Steve Coast <steve at asklater.com
>>>     <mailto:steve at asklater.com>> wrote:
>>>
>>>         bbox queries using the built in spatial indexing presumably?
>>>         OSM has it's own magical bitmask for that, that may also be
>>>         as fast in mongo, who knows.
>>>
>>>
>>>         On 4/11/2011 5:58 PM, Ian Dees wrote:
>>>>         On Mon, Apr 11, 2011 at 6:36 PM, Sergey Galuzo
>>>>         <sergal at microsoft.com <mailto:sergal at microsoft.com>> wrote:
>>>>
>>>>             Hi,
>>>>
>>>>             I am working on evaluation of MongoDB for several
>>>>             storage solutions at hand. Some of them resemble
>>>>             current OSM editing database. I have heard that OSM dev
>>>>             is/was evaluating MongoDB also. I was wondering whether
>>>>             it possible to share the findings?
>>>>
>>>>
>>>>         In my experimentation with MongoDB (seen here:
>>>>         https://github.com/iandees/mongosm/) I found it to be very
>>>>         slow. Inserts were speedy, but bounding-box queries took a
>>>>         long time.
>>>>
>>>>         The most recent dev version of MongoDB includes
>>>>         "multi-location documents" support:
>>>>         http://www.mongodb.org/display/DOCS/Geospatial+Indexing#GeospatialIndexing-MultilocationDocuments
>>>>
>>>>         This would allow a single way document to be indexed at
>>>>         multiple locations and vastly speed up the map query.
>>>>
>>>>
>>>>         _______________________________________________
>>>>         dev mailing list
>>>>         dev at openstreetmap.org <mailto:dev at openstreetmap.org>
>>>>         http://lists.openstreetmap.org/listinfo/dev
>>>
>>>         _______________________________________________
>>>         dev mailing list
>>>         dev at openstreetmap.org <mailto:dev at openstreetmap.org>
>>>         http://lists.openstreetmap.org/listinfo/dev
>>>
>>>
>>>
>>>     _______________________________________________
>>>     dev mailing list
>>>     dev at openstreetmap.org  <mailto:dev at openstreetmap.org>
>>>     http://lists.openstreetmap.org/listinfo/dev
>>
>>
>>     _______________________________________________
>>     dev mailing list
>>     dev at openstreetmap.org  <mailto:dev at openstreetmap.org>
>>     http://lists.openstreetmap.org/listinfo/dev
>
>     _______________________________________________
>     dev mailing list
>     dev at openstreetmap.org <mailto:dev at openstreetmap.org>
>     http://lists.openstreetmap.org/listinfo/dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20110412/c411e72a/attachment.html>