[OSM-dev] Osmosis Output Error

Brett Henderson brett at bretth.com
Wed Dec 10 00:16:44 GMT 2008


Andrew Ayre wrote:
> Karl Newman wrote:
>> On Tue, Dec 9, 2008 at 2:40 PM, Andrew Ayre <andy at britishideas.com 
>> <mailto:andy at britishideas.com>> wrote:
>>
>>     Brett Henderson wrote:
>>
>>         Karl Newman wrote:
>>
>>             So, sounds like the most recent changes are not the issue.
>>             Do you know if SRTM2OSM creates entities with negative ids?
>>             If not, you may run into collisions with the other data file
>>             that you're trying to merge.
>>
>>             Karl
>>
>>         Andrew, it would be great if you could put your files online
>>         somewhere.  I'll take a look this evening if Karl doesn't beat
>>         me to it.
>>
>>         My biggest concern at this point is that you don't appear to be
>>         getting a decent error message.  Perhaps I'm losing an exception
>>         stack trace somewhere.  It makes it hard to diagnose the 
>> problem.
>>
>>         Brett
>>
>>
>>     Karl - I'm not sure if SRTM2OSM produces entities with negative IDs.
>>     I'll try to check into that. But I think other people are doing the
>>     same thing I'm trying to do, making me wonder if I am doing
>>     something wrong here.
>>
>>     Brett - I've uploaded the files to:
>>
>>     http://files.britishideas.com/index.php?dir=osmosis/
>>
>>     Note that I am running this on a set of 26 pairs of tiles. I get
>>     this errors on six of the pairs. The others seem to be OK, although
>>     I haven't rendered them yet to check.
>>
>>     Thanks for the help! Andy
>>
>>
>> Looking at your source data (generated by srtm2osm), there are a 
>> couple potential issues. The first is that the ids are not negative, 
>> although they start at 1E9, so you probably won't run into a 
>> collision (I don't think the new entity ids are anywhere near that 
>> yet). The bigger problem is that it intermingles nodes and ways, and 
>> generally Osmosis works better if all the nodes come first, then all 
>> the ways, then all the relations. For a merge, they have to be 
>> sorted. That's easy enough to do, though--just add a --sort task to 
>> your command line (for both input files). i.e., osmosis --rx 
>> osmfile.osm --sort --rx srtmfile.osm --sort --merge --wx outfile.osm
>>
>> In reality, maybe the problem is entirely with your osm data and not 
>> at all with the SRTM. It's possible some of your osm data files have 
>> items out of order, which is causing a problem with the merge.
>
> Thanks for looking into this. I added --sort after each input file 
> name and it still gives the same error. The OSM map file was generated 
> by osmosis, which I used to split planet.osm.
>
> java -Xmx1560m -jar /home/nav/scripts/osmosis.jar --read-xml 
> file="/home/nav/temp/tile.63255095.osm" --sort --read-xml 
> file="/home/nav/temp/eletile.63255095.osm" --sort --merge --write-xml 
> file="/home/nav/temp/mergedtile.63255095.osm"
>
I had a quick play and think I've found the problem.

Firstly, when you receive an error in osmosis, look further down the 
stack trace.  The real reason is almost always in there somewhere.  
Osmosis is multi-threaded so it is difficult to have the real reason 
show up at the start of the error message.  In your case the first bit 
of error was from the input xml readers detecting that the merge task 
had aborted.  The merge task output was further down the error messages.

In this case there were two problems:
1. The data was unsorted.
2. Some data doesn't appear to have dates attached (I haven't found the 
offending entry yet).

This command line works:
osmosis --rx tile.63255095.osm.gz enableDateParsing=false --sort --rx 
eletile.63255095.osm.gz enableDateParsing=false --sort --merge --wx 
out.osm.gz

It sorts all data before the merge, and ignores all dates in the input 
data (replacing them with current system time).  This is a bit of a 
kludge, ideally the data itself should be fixed to include the correct 
dates but hopefully it points you in the right direction.

Hope that helps.
Brett





More information about the dev mailing list