[OSM-talk] Create extra Planet files for syncing

[Brett Henderson]

Frederik Ramm wrote:
> Sure, I can do that. I was assuming they would somehow be incomplete 
> as well but if they are ok then I can just switch to using these.
The hourly & minute are definitely more reliable than daily.  I won't be 
surprised to see missing data in daily files.  If you see *any* problems 
with the hourly or minute files I'd like to know about it.

The daily diffs are created using a shell script that doesn't fail 
gracefully and doesn't have the ability to generate multiple files if 
the db has been down for a long period.  The hourly and minute diffs are 
created using the osmosis-mysql-extract application (included in the 
osmosis distribution) which aborts if the db is down (spamming me with 
cron failures every minute ...) and catches up again when the db comes 
up again generating as many files as required to become current again.  
The minute diffs encounter intermittent failures on a regular basis due 
to the db or connectivity being lost for brief periods but always 
recover again.

It would take literally 5 minutes to setup daily diffs with the new 
mechanism.  The gzip format is something people would presumably get 
used to fairly quickly.  But the downside is that the compressed files 
are created directly whereas the daily script extracts to an 
uncompressed file to minimise db locking time then compresses 
separately.  If we eventually switch to all InnoDB tables where locking 
isn't such an issue I'll definitely cut it over.
>
> Bye
> Frederik
>