<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
What about rewriting this stuff in C? I have written a MySQL importer
in C some time ago, so what about reusing its XML parsing part?<br>
<br>
If you are interested, I'll put it on Github.<br>
<br>
Andi<br>
<br>
Am 03.07.10 22:43, schrieb Ian Dees:
<blockquote
cite="mid:AANLkTimTs3YRAWkwTxh8YkxOl80qUVVXJgiW8AL4vkHs@mail.gmail.com"
type="cite">
<div class="gmail_quote">On Sat, Jul 3, 2010 at 1:17 PM, Nolan
Darilek <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:nolan@thewordnerd.info">nolan@thewordnerd.info</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">On
07/03/2010 01:09 PM, Nolan Darilek wrote:<br>
<blockquote class="gmail_quote"
style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="im">On 07/02/2010 01:52 PM, Serge Wroclawski wrote:<br>
<blockquote class="gmail_quote"
style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Similarly, Ian Dees and I have written a server using MongoDB, which<br>
also provides functionality such as auto-sharding and built in<br>
map/reduce.<br>
<br>
</blockquote>
<br>
</div>
Is this work available anywhere? How did you find performance to be,
and to what uses did you put it?<br>
<br>
I've done some experiments creating a LibOSM MongoDB backend and found
its performance fairly bad, but I don't have the most optimal server
for it, and probably didn't use MongoDB to its limits. If you
experienced good performance for real-time operations then I'd be very
interested in seeing how you managed it so I might adopt the techniques
and see if I have any better luck.<br>
<br>
It seemed to me that a dump of the entire planet would require a
substantial server to serve up, so I abandoned the work, but would very
much like to revive it if it's at all workable.<br>
<br>
</blockquote>
</blockquote>
<div><br>
</div>
<div>The code is here: <a moz-do-not-send="true"
href="http://github.com/iandees/mongosm">http://github.com/iandees/mongosm</a></div>
<div><br>
</div>
<div>It took several days to import a planet file. The majority of
the CPU time was spent serializing/deserializing BSON in Python and the
Mongo server had very little CPU time so if I use a language with a
faster BSON implementation it might be faster.</div>
<div><br>
</div>
<div>Serge was working on a way to import diff files to maintain
mintutely updates.</div>
<div><br>
</div>
<div>I was working on an HTTP API interface.</div>
</div>
<br>
<pre wrap="">
<fieldset class="mimeAttachmentHeader"></fieldset>
_______________________________________________
dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:dev@openstreetmap.org">dev@openstreetmap.org</a>
<a class="moz-txt-link-freetext" href="http://lists.openstreetmap.org/listinfo/dev">http://lists.openstreetmap.org/listinfo/dev</a>
</pre>
</blockquote>
<br>
</body>
</html>