[OSM-dev] Planet dump today delayed (ie not today) :-(

[Tom Hughes]

In message <1187826626.5410.258.camel at localhost.localdomain>
        Jon Burgess <jburgess777 at googlemail.com> wrote:

> On Wed, 2007-08-22 at 23:45 +0100, Tom Hughes wrote:
>> In message <1187821946.5410.234.camel at localhost.localdomain>
>>           Jon Burgess <jburgess777 at googlemail.com> wrote:
>> 
>> > I'll try to find some time to see if the planet.rb script or SQL queries
>> > can be made more IO efficient.
>> 
>> They're pretty basic queries, so I doubt it.
>
> Something must be not quite right though, surely it shouldn't take over
> 12 hours to collect the data. As you say, the job that the script is
> doing is pretty trivial. I'd be tempted to suggest dumping each of the
> relevant tables with mysqldump and then running a script over the SQL to
> extract the data (or just importing into a DB on another machine which
> can run without any other interruptions -- maybe it is the constant R/W
> of the data during the queries which slow it up).

Well the only thing I can think of is that I don't know how efficient
the page based fetching it is doing is, especially when there is a
condition involved.

Unless it is very clever it will probably have to effectively start
reading at the beginning of the table each time and just not return
records until it reaches the starting point.

Tom

-- 
Tom Hughes (tom at compton.nu)
http://www.compton.nu/