[OSM-dev] Osmosis Output Error

Karl Newman siliconfiend at gmail.com
Wed Dec 10 00:44:46 GMT 2008


On Tue, Dec 9, 2008 at 4:41 PM, Karl Newman <siliconfiend at gmail.com> wrote:

> On Tue, Dec 9, 2008 at 4:16 PM, Brett Henderson <brett at bretth.com> wrote:
>
>> Andrew Ayre wrote:
>>
>>> Karl Newman wrote:
>>>
>>>> On Tue, Dec 9, 2008 at 2:40 PM, Andrew Ayre <andy at britishideas.com<mailto:
>>>> andy at britishideas.com>> wrote:
>>>>
>>>>    Brett Henderson wrote:
>>>>
>>>>        Karl Newman wrote:
>>>>
>>>>            So, sounds like the most recent changes are not the issue.
>>>>            Do you know if SRTM2OSM creates entities with negative ids?
>>>>            If not, you may run into collisions with the other data file
>>>>            that you're trying to merge.
>>>>
>>>>            Karl
>>>>
>>>>        Andrew, it would be great if you could put your files online
>>>>        somewhere.  I'll take a look this evening if Karl doesn't beat
>>>>        me to it.
>>>>
>>>>        My biggest concern at this point is that you don't appear to be
>>>>        getting a decent error message.  Perhaps I'm losing an exception
>>>>        stack trace somewhere.  It makes it hard to diagnose the problem.
>>>>
>>>>        Brett
>>>>
>>>>
>>>>    Karl - I'm not sure if SRTM2OSM produces entities with negative IDs.
>>>>    I'll try to check into that. But I think other people are doing the
>>>>    same thing I'm trying to do, making me wonder if I am doing
>>>>    something wrong here.
>>>>
>>>>    Brett - I've uploaded the files to:
>>>>
>>>>    http://files.britishideas.com/index.php?dir=osmosis/
>>>>
>>>>    Note that I am running this on a set of 26 pairs of tiles. I get
>>>>    this errors on six of the pairs. The others seem to be OK, although
>>>>    I haven't rendered them yet to check.
>>>>
>>>>    Thanks for the help! Andy
>>>>
>>>>
>>>> Looking at your source data (generated by srtm2osm), there are a couple
>>>> potential issues. The first is that the ids are not negative, although they
>>>> start at 1E9, so you probably won't run into a collision (I don't think the
>>>> new entity ids are anywhere near that yet). The bigger problem is that it
>>>> intermingles nodes and ways, and generally Osmosis works better if all the
>>>> nodes come first, then all the ways, then all the relations. For a merge,
>>>> they have to be sorted. That's easy enough to do, though--just add a --sort
>>>> task to your command line (for both input files). i.e., osmosis --rx
>>>> osmfile.osm --sort --rx srtmfile.osm --sort --merge --wx outfile.osm
>>>>
>>>> In reality, maybe the problem is entirely with your osm data and not at
>>>> all with the SRTM. It's possible some of your osm data files have items out
>>>> of order, which is causing a problem with the merge.
>>>>
>>>
>>> Thanks for looking into this. I added --sort after each input file name
>>> and it still gives the same error. The OSM map file was generated by
>>> osmosis, which I used to split planet.osm.
>>>
>>> java -Xmx1560m -jar /home/nav/scripts/osmosis.jar --read-xml
>>> file="/home/nav/temp/tile.63255095.osm" --sort --read-xml
>>> file="/home/nav/temp/eletile.63255095.osm" --sort --merge --write-xml
>>> file="/home/nav/temp/mergedtile.63255095.osm"
>>>
>>>  I had a quick play and think I've found the problem.
>>
>> Firstly, when you receive an error in osmosis, look further down the stack
>> trace.  The real reason is almost always in there somewhere.  Osmosis is
>> multi-threaded so it is difficult to have the real reason show up at the
>> start of the error message.  In your case the first bit of error was from
>> the input xml readers detecting that the merge task had aborted.  The merge
>> task output was further down the error messages.
>>
>> In this case there were two problems:
>> 1. The data was unsorted.
>> 2. Some data doesn't appear to have dates attached (I haven't found the
>> offending entry yet).
>>
>> This command line works:
>> osmosis --rx tile.63255095.osm.gz enableDateParsing=false --sort --rx
>> eletile.63255095.osm.gz enableDateParsing=false --sort --merge --wx
>> out.osm.gz
>>
>> It sorts all data before the merge, and ignores all dates in the input
>> data (replacing them with current system time).  This is a bit of a kludge,
>> ideally the data itself should be fixed to include the correct dates but
>> hopefully it points you in the right direction.
>>
>> Hope that helps.
>> Brett
>>
>>
> The SRTM data has no timestamps, which is causing the problem. So you only
> need to set enableDateParsing=false for that file. It might be good to show
> a warning if an entity has no timestamp attribute in the XML. I propose
> adding the following lines in BaseElementProcessor.java:
>
> +    private static final Logger log =
> Logger.getLogger(BaseElementProcessor.class.getName());
>
> ...
>
>     protected TimestampContainer createTimestampContainer(String data) {
>         if (enableDateParsing) {
> +            if (data == null || data == "") {
> +                log.warning("Element missing timestamp attribute.");
> +            }
>             return new UnparsedTimestampContainer(timestampFormat, data);
>         } else {
>             return dummyTimestampContainer;
>         }
>     }
>
> That encapsulates it in the lowest possible level, but the downside is that
> it doesn't report anything about the problematic element--Ideally it would
> show the id and whether it's a node, way or relation. That could conceivably
> be solved by passing that information to the createTimestampContainer
> method, but that seems sloppy.
>
> Karl
>

Oops, that should be "".equals(data) or data.length() == 0

Karl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20081209/3bfa7db4/attachment.html>


More information about the dev mailing list