[OSM-dev] osm2pgsql can parse diffs now (for testing anyway)

[Brett Henderson]

Andy Allan wrote:
> On Wed, Jul 9, 2008 at 4:43 PM, Martijn van Oosterhout
> <kleptog at gmail.com> wrote:
>   
>> Usage is simple enough. first load your planet dump with --slim mode
>> (if you have the memory, give it a big cache):
>> osm2pgsql --slim my-planet-extract.osm.gz
>> After that you can use:
>> osm2pgsql --slim -a my-daily-diff.osc.gz
>> To apply the patch.
>> Don't forget the -a, or you'll blow away the database you've got.
>>     
Very nice!  Thanks Martijn, I've been looking forward to this for a long 
time.  I look forward to giving this a go.
> Is chaining multiple diffs supported? E.g.
>
> osm2pgsql --slim planet.osm.gz
> osm2pgsql --slim -a thursday.osc.gz
> osm2pgsql --slim -a friday.osc.gz
>   
This would work quite nicely with the Osmosis --read-change-interval task:
http://wiki.openstreetmap.org/index.php/Osmosis#--read-change-interval_.28--rci.29

The purpose of that task is to retrieve all change files since the last 
time it was run (it maintains configuration timestamp state in local 
files), merges all the changes into a single change stream, and writes 
them to the output.  It should be possible to run the whole thing 
something like this:
osmosis --read-change-interval workingDirectory=<my osmosis working 
directory> --write-xml-change /dev/stdout | osm2pgsql --slim -a /dev/stdin

The --read-change-interval can be used with daily, hourly or minute 
changesets by changing the configuration file appropriately.  You need 
to run the --read-change-interval-init task first:
http://wiki.openstreetmap.org/index.php/Osmosis#--read-change-interval-init_.28--rcii.29
Then modify the configuration file it creates.

This can be run as often as you like because it will produce an empty 
changeset if no data is available.  In theory it should be possible now 
to keep a mapnik database within 6 minutes or so of the main API 
although mod_tile might require some serious tweaking to deal with 
this.  Running hourly should work quite nicely however for a small site.