[OSM-dev] rails port import performance

[Brian Cavagnolo]

Hey,

I brought up an empty instance of the rails port to experiment with
some importing.  It's taking about 2.5 minutes to upload a file with
about 1600 nodes and 600 ways.  top reports that ruby is taking about
90% of a CPU and postgres is taking the other 10% on a Intel Core2 Duo
2.40GHz VM with plenty or RAM.  I get similar performance on an Intel
Xeon 2.60GHz non-VM.  I did some quick and dirty profiling on one of
the POSTs to /api/0.6/changeset/<id>/upload.  It seems that much of
the time is spent in the way and node ActiveRecord models.  I tuned
postgres following the guidelines [1] but did not observe any
meaningful change in performance.  Is this sort of performance
typical?  Any recommendations on how to improve it?

If you want to poke around, here are some more details:

-- I use a fork of bulk_upload to perform the upload:
git://github.com/ual/bulkyosm.git

-- Here's the command line to perform the upload:
   time bulk_upload.py -H localhost:3000 -u myname -p mypw -c "testing
upload speed" -i citylots_9.osm

-- The input file and the ruby-prof performance output can be found here:
http://jerusalem.ual.berkeley.edu/performance/

Thanks,
Brian

[1] http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server