[OSM-dev] Osmosis: Bounding polygon does not support change data as input?

Andreas Kalsch andreaskalsch at gmx.de
Thu Sep 18 07:59:43 BST 2008


Brett Henderson schrieb:
> Andreas Kalsch wrote:
>> I have set up a OSM database with Germany data. Now I want to update 
>> my data daily. I have not found diff files for Germany so I want to 
>> use the global diff files, which are not too big.
>> To re-import German data every day would be too expensive - it 
>> already took some hours.
>>
>> In Osmosis, I tried this:
>>
>> osmosis/bin/osmosis --read-xml-change data/osm/20080916-20080917.osc 
>> --bounding-polygon data/germany2pts.txt --apply-change --write-mysql 
>> host=localhost database=osm_de user=user password=xxx
>>
>> Result:
>>
>> Sep 17, 2008 10:05:15 PM com.bretth.osmosis.core.Osmosis main
>> INFO: Osmosis Version 0.29
>> Sep 17, 2008 10:05:15 PM com.bretth.osmosis.core.Osmosis main
>> INFO: Preparing pipeline.
>> Sep 17, 2008 10:05:16 PM com.bretth.osmosis.core.Osmosis main
>> SEVERE: Execution aborted.
>> com.bretth.osmosis.core.OsmosisRuntimeException: Task 
>> 2-bounding-polygon does not support data provided by default pipe 
>> stored at level 1 in the default pipe stack.
>>     at 
>> com.bretth.osmosis.core.pipeline.common.PipeTasks.retrieveTask(PipeTasks.java:154) 
>>
>>     at 
>> com.bretth.osmosis.core.pipeline.common.TaskManager.getInputTask(TaskManager.java:164) 
>>
>>     at 
>> com.bretth.osmosis.core.pipeline.v0_5.SinkSourceManager.connect(SinkSourceManager.java:51) 
>>
>>     at 
>> com.bretth.osmosis.core.pipeline.common.Pipeline.connectTasks(Pipeline.java:69) 
>>
>>     at 
>> com.bretth.osmosis.core.pipeline.common.Pipeline.prepare(Pipeline.java:111) 
>>
>>     at com.bretth.osmosis.core.Osmosis.main(Osmosis.java:49)
>>
>> So am I right thinking, that the bounding polygon filter does not 
>> support the input of change data?
>>   
> That's correct.  There is no way of writing a bounding box task for 
> managing changeset data.  Take this scenario for example.
> A way already exists in your database related to several nodes.
> Somebody modifies the way but doesn't touch the nodes.
> The next changeset will contain the updated way within a <modify> 
> element but not the nodes.
> A bounding box task will have no way of knowing whether the way is 
> inside or outside the bounding box.
This is a clear fact, of course. But Osmosis could retrive the nodes 
from the database instead and try to find it out, like you described below.
>
> All is not lost however.
> The simple answer is just to import the diff for the entire world.  
> It's approximately 10MB of compressed data per day so the database 
> will only grow steadily.
> The slightly more complicated answer is to import the entire world 
> diffs but then run an additional query to delete data not inside the 
> bounding box.  This would only have to be run occasionally.
OK, so a solution could be:
- load planet diff
- apply it completely
- make copy of current database with only points within the polygon and 
delete the old database - every n days
- after that triggering changes to my application database which uses 
just a subset of the whole data, so that I use no data which is outlide 
the polygon

This is a compromise that would work for me.
>
> A more complicated solution (involving some coding) could be as follows:
> Before applying a changeset, re-order the changeset to preserve 
> referential integrity.  Osmosis can do this already with the --sort task.
> Only write nodes that are inside the bounding box and check to see if 
> each node already exists in the database.
> For every way write, check the nodes that should by this point already 
> exist in the database to see if the way is inside the bounding box.
> For every relation write, check to see if any members already exist in 
> the database.
I am sure this would take very much time. Sorting a file is surely not a 
very quick task ;)
>
> I don't have any immediate plans to attempt the correct solution above 
> due to lack of time, but it would be neat if somebody could get it 
> working.
>
> Brett
>
>





More information about the dev mailing list