[OSM-dev] Mass imports (TIGER and AND)

[Martijn van Oosterhout]

On 8/28/07, Dave Hansen <dave at sr71.net> wrote:
> I've been very painfully uploading the TIGER-generated data through
> JOSM.  At the rate I'm going it will probably take 5 or 10 years to
> upload the entire US.  Literally.  I'm uploading one or two counties a
> day, and there are 3,234 counties in the country.

Ah, do you have any idea on the number of requests per second that
approximatly equates to?

> So, I installed the rails port on my laptop, and sicked JOSM on it.  The
> uploads are maybe twice as fast as they are to the main OSM server.  So,
> the round-trip-time actually isn't that _huge_ of a performance
> bottleneck.

Over the last view days I've written a bulk_import.pl which takes a
JOSM or osmChange file and uploads it to the server. It should have
lower overhead than JOSM, but from what you're saying it that it won't
really help. :(

The AND import has > 8 million objects to create, so speed is of concern.

I'm still writing the documentation for it, but I'll be happy to
answer any questions.

Have a nice day,
-- 
Martijn van Oosterhout <kleptog at gmail.com> http://svana.org/kleptog/