[OSM-dev] simple tool for finding minimal set of osmosis diff files

Brett Henderson brett at bretth.com
Wed Jul 2 23:50:03 BST 2008


Timo Juhani Lindfors wrote:
> Aha, due to java issues I was using the java5 compatible version which
> does not have that option:
>
> $ grep -ri read-change-interval osmosis-0.24.1-java5
> $
>   
Hmm, I haven't updated it for ages.  I should create a new java5 version 
one of these days ...
>   
>> For example, I use it to maintain a local copy of all data in Myanmar.
>> http://www.bretth.com/myanmar/
>> The main file of interest in that directory is replicate_osm_file.sh 
>> which is launched from cron hourly.  configuration.txt defines which set 
>> of diffs to download, I'm using hourly diffs but minute and day diffs 
>> will also work.
>>     
>
> Interesting, I didn't know you could apply diffs if you only have some
> small bounding box of data.
>   
It's a bit of a kludge.  I apply the entire world changes into the file 
then run the result through another bounding box task.  This strips off 
all the data I'm not interested in.  If you're applying to a real 
database you can do something smarter with a bit of effort.  If you do 
apply to a database you should re-order the diff data before import 
using the "--sort-change type=seekable" task which will re-order the 
data to apply all adds, then all modifies, then all deletes thus 
maintaining referential integrity at all times.
>> One nice thing about your script is that it uses daily files and hourly 
>> diffs, osmosis won't do that for you.  It's probably faster to use your 
>> approach because less files are downloaded, although the total bandwidth 
>> consumption will be almost identical.
>>     
>
> Yep, I am experimenting with
>
> http://svn.openstreetmap.org/applications/utils/export/tiledata/import
>
> and trying to make it support minutely diffs so I really need more
> than just hourly diffs. I need to add support for the case where
> starttime is not at midnight.
>   
Cool!

It may not be of use but osmosis does support a pgsql schema which 
provides bounding box extraction support.  There is a script in the 
osmosis scripts directory.
http://svn.openstreetmap.org/applications/utils/osmosis/script/pgsql_simple_schema.sql

There are tasks for bulk loading this data (either direct inserts or via 
bulk load "COPY" files) and a task for applying diffs directly to the 
db.  Jochen has been able to import the whole of Europe in a couple of 
hours, although he said the change application task isn't working 
properly at the moment.

I assume you've already developed your own schema though.

Anyway, good luck.

Cheers,
Brett





More information about the dev mailing list