[OSM-dev] best approach for mass upload
frederik at remote.org
Sun May 17 00:05:22 BST 2009
Stefano Salvador wrote:
> which is the best approach to do such an upload using the new 0.6 API ? Is
> there a limit on the size of the uploaded file ?
A changeset is limited to 50,000 elements so there's your upper limit
for diff uploads as well.
Two different kinds of overhead play into the diff upload equation. The
smaller your individual HTTP requests, the more processing time and
network latency is wasted in creating the HTTP connection, assigning a
daemon, etc etc; and the larger your uploaded changesets, the more
expensive it becomes for the database to isolate your transaction.
Where exactly the "sweet spot" lies in that equation is hard for me to
tell, and is also likely to be very different on the production machines
compared to the dev server.
My gut feeling would be to try something like this: open a changeset,
then do 40 individual diff uploads of about 1,000 elements each, then
close the changeset, and repeat.
Frederik Ramm ## eMail frederik at remote.org ## N49°00'09" E008°23'33"
More information about the dev