[Imports] Florida Water Import

Jason Remillard remillard.jason at gmail.com
Mon Sep 23 23:50:40 UTC 2013


Hi Brian,

On Mon, Sep 23, 2013 at 6:23 PM, Brian May <bmay at mapwise.com> wrote:
> For fully automated - 10k. I experimented with 30k while manually reviewing
> in JOSM, and it ran fine. From observing network traffic looks like it bulk
> transferred everything right away and then there's processing on the OSM
> server that happens for a bit until the upload is done. I know there's docs
> / suggestions that say only do 10k. The API has a limit of 50k. So I'm
> curious about the issues there. There shouldn't be any conflicts with
> existing OSM objects since conflation is being done between OSM data and the
> lakes data within a day or so to eliminate the chance for dupes - and I'm
> not editing existing objects.

The MA building import was 1,000+ change sets at 10,000 elements. We
had about a 5-10% failure rate. The failure rate seemed to be worse on
the weekend. I recently did a smaller import of 150+ change sets, at
2,000 elements with zero failures. I don't have any idea what is
faster.

Regardless of what you select for your change set cutoff, just make
sure your work flow is solid for dealing with failures. It is easy to
miss things during the retry and to do a double upload or skip
something. Sometimes it says if failed, when it actually worked, every
failure needs to be looked at by hand.

The upload scripts we wrote for the MA building import were modified
to log everything, and would check for a log file, and not upload if
it had a log file on the disk.

Thanks
Jason.



More information about the Imports mailing list