[OSM-dev] simple tool for finding minimal set of osmosis diff files

[Brett Henderson]

Hi Timo,

There is already a way to do this but I'll leave it up to you to decide 
whether it suits your purposes.

Osmosis has a task named --read-change-interval which will download all 
changes from the last time it ran (it maintains a timestamp file), and 
merge them into a single changeset.  That changeset can then be used to 
patch a file, or imported into a database, etc.

For example, I use it to maintain a local copy of all data in Myanmar.
http://www.bretth.com/myanmar/
The main file of interest in that directory is replicate_osm_file.sh 
which is launched from cron hourly.  configuration.txt defines which set 
of diffs to download, I'm using hourly diffs but minute and day diffs 
will also work.

There is one big problem with your script which should be easy to fix.  
DON'T use the bz2 daily files, they use a different timezone offset 
(BST) to the gz daily files (UTC) so will contain slightly different 
data.  I'm going to disable them very soon in favour of the new gz 
files.  The gz files are a bit larger but fix timezone issues and 
drastically improve the reliability of daily files.

One nice thing about your script is that it uses daily files and hourly 
diffs, osmosis won't do that for you.  It's probably faster to use your 
approach because less files are downloaded, although the total bandwidth 
consumption will be almost identical.

Cheers,
Brett

Timo Juhani Lindfors wrote:
> Hi,
>
> in order to sync my local planet.osm I need to fetch a number of
> osmosis diff files. I could not find a tool for figuring out which
> files are needed so I wrote my own. Please let me know if I just
> reinvented some wheels :-)
>
> http://iki.fi/lindi/planet-sync.pl
>
> # usage example:
> #
> # ./planet-sync.pl 20080625000000 20080629180400
> # http://planet.openstreetmap.org/daily/daily-20080625-20080626.osc.bz2
> # http://planet.openstreetmap.org/daily/daily-20080626-20080627.osc.bz2
> # http://planet.openstreetmap.org/daily/daily-20080627-20080628.osc.bz2
> # http://planet.openstreetmap.org/daily/daily-20080628-20080629.osc.bz2
> # http://planet.openstreetmap.org/hourly/2008062900-2008062901.osc.gz
> # http://planet.openstreetmap.org/hourly/2008062901-2008062902.osc.gz
> # http://planet.openstreetmap.org/hourly/2008062902-2008062903.osc.gz
> # http://planet.openstreetmap.org/hourly/2008062903-2008062904.osc.gz
> # http://planet.openstreetmap.org/hourly/2008062904-2008062905.osc.gz
> # http://planet.openstreetmap.org/hourly/2008062905-2008062906.osc.gz
> # http://planet.openstreetmap.org/hourly/2008062906-2008062907.osc.gz
> # http://planet.openstreetmap.org/hourly/2008062907-2008062908.osc.gz
> # http://planet.openstreetmap.org/hourly/2008062908-2008062909.osc.gz
> # http://planet.openstreetmap.org/hourly/2008062909-2008062910.osc.gz
> # http://planet.openstreetmap.org/hourly/2008062910-2008062911.osc.gz
> # http://planet.openstreetmap.org/hourly/2008062911-2008062912.osc.gz
> # http://planet.openstreetmap.org/hourly/2008062912-2008062913.osc.gz
> # http://planet.openstreetmap.org/hourly/2008062913-2008062914.osc.gz
> # http://planet.openstreetmap.org/hourly/2008062914-2008062915.osc.gz
> # http://planet.openstreetmap.org/hourly/2008062915-2008062916.osc.gz
> # http://planet.openstreetmap.org/hourly/2008062916-2008062917.osc.gz
> # http://planet.openstreetmap.org/hourly/2008062917-2008062918.osc.gz
> # http://planet.openstreetmap.org/minute/200806291800-200806291801.osc.gz
> # http://planet.openstreetmap.org/minute/200806291801-200806291802.osc.gz
> # http://planet.openstreetmap.org/minute/200806291802-200806291803.osc.gz
> # http://planet.openstreetmap.org/minute/200806291803-200806291804.osc.gz
>
> best regards,
> Timo Lindfors
>
>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev
>