[OSM-dev] best approach for mass upload

Frederik Ramm frederik at remote.org
Sun May 17 00:05:22 BST 2009


Hi,

Stefano Salvador wrote:
> which is the best approach to do such an upload using the new 0.6 API ? Is 
> there a limit on the size of the uploaded file ?

A changeset is limited to 50,000 elements so there's your upper limit 
for diff uploads as well.

Two different kinds of overhead play into the diff upload equation. The 
smaller your individual HTTP requests, the more processing time and 
network latency is wasted in creating the HTTP connection, assigning a 
daemon, etc etc; and the larger your uploaded changesets, the more 
expensive it becomes for the database to isolate your transaction.

Where exactly the "sweet spot" lies in that equation is hard for me to 
tell, and is also likely to be very different on the production machines 
compared to the dev server.

My gut feeling would be to try something like this: open a changeset, 
then do 40 individual diff uploads of about 1,000 elements each, then 
close the changeset, and repeat.

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frederik at remote.org  ##  N49°00'09" E008°23'33"




More information about the dev mailing list