<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#ffffff">
Interesting.<br>
<br>
How efficient is the (big)int indexing and/or masking?<br>
<br>
Was this all on a single machine?<br>
<br>
<br>
<br>
On 4/12/2011 1:52 PM, Ian Dees wrote:
<blockquote
cite="mid:BANLkTimFuY1__zSKzBrPr+h5iz2OMZeHwQ@mail.gmail.com"
type="cite">Yep.<br>
<br>
<div class="gmail_quote">On Tue, Apr 12, 2011 at 3:51 PM, Steve
Coast <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:steve@asklater.com">steve@asklater.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt
0.8ex; border-left: 1px solid rgb(204, 204, 204);
padding-left: 1ex;">
<div text="#000000" bgcolor="#ffffff"> and using the builtin
spatial index?
<div>
<div class="h5"><br>
<br>
<br>
On 4/12/2011 1:50 PM, Ian Dees wrote:
<blockquote type="cite">Yes, one document per
node/way/relation.<br>
<br>
<div class="gmail_quote">On Tue, Apr 12, 2011 at 3:47
PM, Steve Coast <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:steve@asklater.com" target="_blank">steve@asklater.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt
0pt 0pt 0.8ex; border-left: 1px solid rgb(204,
204, 204); padding-left: 1ex;">
<div text="#000000" bgcolor="#ffffff"> how was the
data put in the db though? 1 document per node?
<div>
<div><br>
<br>
On 4/12/2011 1:39 PM, Nolan Darilek wrote:
<blockquote type="cite"> Oopse, meant for
this to go to the whole list.<br>
<br>
<br>
<br>
-------- Original Message --------
<table border="0" cellpadding="0"
cellspacing="0">
<tbody>
<tr>
<th valign="BASELINE" align="RIGHT"
nowrap="nowrap">Subject: </th>
<td>Re: [OSM-dev] OSM and MongoDB</td>
</tr>
<tr>
<th valign="BASELINE" align="RIGHT"
nowrap="nowrap">Date: </th>
<td>Tue, 12 Apr 2011 15:26:41 -0500</td>
</tr>
<tr>
<th valign="BASELINE" align="RIGHT"
nowrap="nowrap">From: </th>
<td>Nolan Darilek <a
moz-do-not-send="true"
href="mailto:nolan@thewordnerd.info"
target="_blank"><nolan@thewordnerd.info></a></td>
</tr>
<tr>
<th valign="BASELINE" align="RIGHT"
nowrap="nowrap">To: </th>
<td>Ian Dees <a
moz-do-not-send="true"
href="mailto:ian.dees@gmail.com"
target="_blank"><ian.dees@gmail.com></a></td>
</tr>
</tbody>
</table>
<br>
<br>
I had/am having a somewhat bad experience
storing OSM data in MongoDB.<br>
<br>
Initially I stored all map data in
MongoDB, but queries took ages. The same
queries that happen in 100-200 MS now
often took nearly a second. Additionally,
some took upwards of 5, and I even found
spots on my map sparsely populated with
points, but which reliably performed the
queries I need in 30+ seconds.<br>
<br>
I filed a thorough bug in their tracker,
including a dataset and queries that
reliably duplicated the issue. It was
marked wontfix, I abandoned MongoDB, and
it was apparently re-opened and fixed
several months later. So perhaps it's a
non-issue now.<br>
<br>
I'm still using MongoDB for part of my
current project, user POI storage. It does
indeed use geohashes, and I'm experiencing
strange accuracy issues. My platform is
pedestrian navigation with many small
distance queries. Points in the
non-MongoDB dataset are reliably detected
in a radius roughly 100 meters around the
traveler. Points in MongoDB queried with
the same bounding boxes don't appear until
they're within 30-40 meters. I recently
updated from an older version to a new
build of 1.8. The older version widely
varied the detection range. Some points
were detected 100 or so meters out, while
others weren't picked up until 30 or so.
It was always the same points, too. The
point for my apartment remains reliably
visible for ~100 meters or so, while the
corner store and restaurant didn't appear
until I was very close. 1.8 at least
appears to be consistent, always detecting
at 30 meters or so. I can only assume that
this is a geohash oddity that only appears
for very small differences, something that
works out to rounding error for larger
values.<br>
<br>
I like MongoDB for many things, but not
for geospatial data more complicated than
a series of points. I'm working on
migrating user/POI storage to a geospatial
store.<br>
<br>
<br>
On 04/12/2011 01:20 PM, Ian Dees wrote:
<blockquote type="cite">Yep, and I think
Mongo uses geohashes as their index
behind the scenes. One of the problems
with that, though, is they have some
arbitrary length that they compute the
geohash to and when you have lots of
points (as OSM data does) the buckets
they're searching are very full.<br>
<br>
<div class="gmail_quote">On Tue, Apr 12,
2011 at 1:00 PM, Steve Coast <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:steve@asklater.com"
target="_blank">steve@asklater.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin: 0pt 0pt 0pt 0.8ex;
border-left: 1px solid rgb(204, 204,
204); padding-left: 1ex;">
<div text="#000000"
bgcolor="#ffffff"> bbox queries
using the built in spatial
indexing presumably? OSM has it's
own magical bitmask for that, that
may also be as fast in mongo, who
knows.
<div>
<div><br>
<br>
On 4/11/2011 5:58 PM, Ian Dees
wrote: </div>
</div>
<blockquote type="cite">
<div>
<div>
<div class="gmail_quote">On
Mon, Apr 11, 2011 at 6:36
PM, Sergey Galuzo <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:sergal@microsoft.com" target="_blank">sergal@microsoft.com</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin: 0pt 0pt
0pt 0.8ex; border-left:
1px solid rgb(204, 204,
204); padding-left:
1ex;">
<div link="blue"
vlink="purple"
lang="EN-US">
<div>
<p class="MsoNormal"><span>Hi,</span></p>
<p class="MsoNormal"><span> </span></p>
<p class="MsoNormal"><span>I
am working on
evaluation of
MongoDB for
several storage
solutions at
hand. Some of
them resemble
current OSM
editing
database. I have
heard that OSM
dev is/was
evaluating
MongoDB also. I
was wondering
whether it
possible to
share the
findings?</span></p>
<p class="MsoNormal"><span> </span></p>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>In my experimentation
with MongoDB (seen
here: <a
moz-do-not-send="true"
href="https://github.com/iandees/mongosm/" target="_blank">https://github.com/iandees/mongosm/</a>)
I found it to be very
slow. Inserts were
speedy, but bounding-box
queries took a long
time.</div>
<div><br>
</div>
<div>The most recent dev
version of MongoDB
includes "multi-location
documents" support:</div>
<div> <a
moz-do-not-send="true"
href="http://www.mongodb.org/display/DOCS/Geospatial+Indexing#GeospatialIndexing-MultilocationDocuments"
target="_blank">http://www.mongodb.org/display/DOCS/Geospatial+Indexing#GeospatialIndexing-MultilocationDocuments</a></div>
<div><br>
</div>
<div>This would allow a
single way document to
be indexed at multiple
locations and vastly
speed up the map query.</div>
</div>
</div>
</div>
<pre><fieldset></fieldset>
_______________________________________________
dev mailing list
<div><a moz-do-not-send="true" href="mailto:dev@openstreetmap.org" target="_blank">dev@openstreetmap.org</a>
<a moz-do-not-send="true" href="http://lists.openstreetmap.org/listinfo/dev" target="_blank">http://lists.openstreetmap.org/listinfo/dev</a>
</div></pre>
</blockquote>
</div>
<br>
_______________________________________________<br>
dev mailing list<br>
<a moz-do-not-send="true"
href="mailto:dev@openstreetmap.org"
target="_blank">dev@openstreetmap.org</a><br>
<a moz-do-not-send="true"
href="http://lists.openstreetmap.org/listinfo/dev"
target="_blank">http://lists.openstreetmap.org/listinfo/dev</a><br>
<br>
</blockquote>
</div>
<br>
<pre><fieldset></fieldset>
_______________________________________________
dev mailing list
<a moz-do-not-send="true" href="mailto:dev@openstreetmap.org" target="_blank">dev@openstreetmap.org</a>
<a moz-do-not-send="true" href="http://lists.openstreetmap.org/listinfo/dev" target="_blank">http://lists.openstreetmap.org/listinfo/dev</a>
</pre>
</blockquote>
<br>
<pre><fieldset></fieldset>
_______________________________________________
dev mailing list
<a moz-do-not-send="true" href="mailto:dev@openstreetmap.org" target="_blank">dev@openstreetmap.org</a>
<a moz-do-not-send="true" href="http://lists.openstreetmap.org/listinfo/dev" target="_blank">http://lists.openstreetmap.org/listinfo/dev</a>
</pre>
</blockquote>
</div>
</div>
</div>
<br>
_______________________________________________<br>
dev mailing list<br>
<a moz-do-not-send="true"
href="mailto:dev@openstreetmap.org"
target="_blank">dev@openstreetmap.org</a><br>
<a moz-do-not-send="true"
href="http://lists.openstreetmap.org/listinfo/dev"
target="_blank">http://lists.openstreetmap.org/listinfo/dev</a><br>
<br>
</blockquote>
</div>
<br>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</blockquote>
</body>
</html>