[osmosis-dev] Odd rate numbers with write-pgsql-dump

[Paul Norman]

I am using the --write-pgsql-dump in preparation for importing the planet
into a pgsnapshot database and have noticed some oddities with ways. 

I am reading from the latest .pbf and using enableBboxBuilder=no
enableLinestringBuilder=yes nodeLocationStoreType=TempFile. I tried InMemory
but 16G of ram is not sufficient with the latest planet.

My --log-progress task reported nodes being processed at about 250k
nodes/second. It has now moved to the ways and the reports are interesting.

It was reporting approximately 150-200 objects/second but when I manually
take compute the ways/second from the way numbers I get figures in the realm
of 5k+ objects/second. I know there may be some differences from deleted
ways skewing the numbers, but this seems beyond what that would account for.

It has taken approximately 20 minutes to reach way 4.5m, about 4k
ways/second. 

iotop reports java reading at 5-10 M/s and writing at 150-400 K/s
iostat reports the array with the .pbf and tempfile at 300 r/s 28 w/s 6000
rkB/s 330 wkB/s, 90 %util.

Although the progress seems okay (day or so to finish) I'm stumped by the
objects/sec number.

As an aside, about a year ago it took 5 hours to do it with InMemory but I'd
have to go up to 24 GB RAM to do that again.