[OSM-dev] Planet dump today delayed (ie not today) :-(

[Tom Hughes]

In message <1187821946.5410.234.camel at localhost.localdomain>
          Jon Burgess <jburgess777 at googlemail.com> wrote:

> Looking at the IO graphs I believe the planet.rb script is moving very
> slowly because DB is completely IO bound. I suspect the 30 minute CPU
> blips occur each time the planet.rb successfully fetches another page of
> 500,000 entries.

Interesting. The logical conclusion is that it is related to the
changes that I made last week - it looks like the increase in InnoDB
buffer space to 2Gb was done immediately after the planet dump last
week. That does seem to have helped the API though, so it is swings
and roundabouts.

> I guess it will start moving more rapidly again when the other DB/API
> users start to drop off after midnight.

I thought Sebastian had stopped it now anyway?

> I'll try to find some time to see if the planet.rb script or SQL queries
> can be made more IO efficient.

They're pretty basic queries, so I doubt it.

Osmosis seems to fare rather better though, as it managed to dump the
database in about 3.5 hours this evening. It also does rather more
complicated queries, but it does use the history tables (which are
all MyISAM) rather than the live tables (which are mostly InnoDB).

Tom

-- 
Tom Hughes (tom at compton.nu)
http://www.compton.nu/