[OSM-dev] Osmosis Plans
Jon Burgess
jburgess777 at googlemail.com
Tue Sep 25 20:10:09 BST 2007
On Tue, 2007-09-25 at 19:14 +1000, Brett Henderson wrote:
> spaetz wrote:
> > On Tue, Sep 25, 2007 at 02:31:26PM +1000, Brett Henderson wrote:
> >
> >> 1. Planet dumping.
> >>
> > If the simple planet.c is faster, it might do the job just as well. Don't know
> >
> Yep, no issue here. If planet.c and osmosis use the same query
> mechanism the speed should be very similar but that's probably not the
> case. Osmosis is currently using temp files for way generation to avoid
> query timeouts, there's probably a better way to do this. I've avoided
> using multiple queries per table but given MySQL constraints it may be
> the best approach.
I updated to the latest osmosis code the same timing test as I did on my
C version:
$ time bin/osmosis --read-mysql-current host="localhost" database="openstreetmap" user="openstreetmap" password="openstreetmap" --write-xml file="/tmp/out.osm"
...
FINE: Pipeline complete.
real 3m20.234s
user 3m16.397s
sys 0m4.457s
$ time ./planet > /tmp/new
real 1m14.428s
user 0m26.330s
sys 0m3.868s
Notice how so much more 'user' CPU is spent in osmosis vs the C code. I
don't have the Java profiling skills to be able to tell you where this
CPU time is spent but something is clearly taking a lot more effort to
process.
I think it is good to have competing approaches. It fosters a little
healthy competition and allows ideas about alternative algorithms to be
explored. While developing the C tool I committed 2 updates to improve
the ruby version.
It also builds up a set of example routines for generic OSM processing.
More than half of the C code is common with the other tools I've
developed (planetdiff, osm2pgsql etc). It also provides a stepping stone
towards possibly writing a "mysql2pgsql" tool or whatever else we might
dream up next.
Jon
More information about the dev
mailing list