[OSM-dev] buulk_uploads use up all CPU on dev...

Martijn van Oosterhout kleptog at gmail.com
Mon Sep 17 17:06:09 BST 2007


On 9/17/07, spaetz <osm at sspaeth.de> wrote:
> dev's CPU is used up 100%. both your bulk_upload scripts have started using up everything available. So it seems part of your slowness is not the API's slowness, but CPU limitations of dev. Is there a reason why they are so CPU intensive?

Ok, I think daveh fixed his script already.

> Would you guys care to run your uploads somewhat niced too (say at -n 5)? Plus it's weird that it started using so much CPU, I can't recall it using that much before.

Looking at the stats, it costs about 4ms of processing per object to
upload. At the current rate of upload that does work out around the
3-4% CPU usage, which is what ps is showing.

I'll renice my uploader, when it reaches near 100% it's going to start
rereading the file from the beginning to find stuff that's been
missed, and that's going to cost a lot more CPU than just the straight
uploading now.

Have a nice day,
-- 
Martijn van Oosterhout <kleptog at gmail.com> http://svana.org/kleptog/




More information about the dev mailing list