On Tue, Dec 9, 2008 at 4:16 PM, Brett Henderson <span dir="ltr"><<a href="mailto:brett@bretth.com">brett@bretth.com</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div><div></div><div class="Wj3C7c">Andrew Ayre wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Karl Newman wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
On Tue, Dec 9, 2008 at 2:40 PM, Andrew Ayre <<a href="mailto:andy@britishideas.com" target="_blank">andy@britishideas.com</a> <mailto:<a href="mailto:andy@britishideas.com" target="_blank">andy@britishideas.com</a>>> wrote:<br>
<br>
Brett Henderson wrote:<br>
<br>
Karl Newman wrote:<br>
<br>
So, sounds like the most recent changes are not the issue.<br>
Do you know if SRTM2OSM creates entities with negative ids?<br>
If not, you may run into collisions with the other data file<br>
that you're trying to merge.<br>
<br>
Karl<br>
<br>
Andrew, it would be great if you could put your files online<br>
somewhere. I'll take a look this evening if Karl doesn't beat<br>
me to it.<br>
<br>
My biggest concern at this point is that you don't appear to be<br>
getting a decent error message. Perhaps I'm losing an exception<br>
stack trace somewhere. It makes it hard to diagnose the problem.<br>
<br>
Brett<br>
<br>
<br>
Karl - I'm not sure if SRTM2OSM produces entities with negative IDs.<br>
I'll try to check into that. But I think other people are doing the<br>
same thing I'm trying to do, making me wonder if I am doing<br>
something wrong here.<br>
<br>
Brett - I've uploaded the files to:<br>
<br>
<a href="http://files.britishideas.com/index.php?dir=osmosis/" target="_blank">http://files.britishideas.com/index.php?dir=osmosis/</a><br>
<br>
Note that I am running this on a set of 26 pairs of tiles. I get<br>
this errors on six of the pairs. The others seem to be OK, although<br>
I haven't rendered them yet to check.<br>
<br>
Thanks for the help! Andy<br>
<br>
<br>
Looking at your source data (generated by srtm2osm), there are a couple potential issues. The first is that the ids are not negative, although they start at 1E9, so you probably won't run into a collision (I don't think the new entity ids are anywhere near that yet). The bigger problem is that it intermingles nodes and ways, and generally Osmosis works better if all the nodes come first, then all the ways, then all the relations. For a merge, they have to be sorted. That's easy enough to do, though--just add a --sort task to your command line (for both input files). i.e., osmosis --rx osmfile.osm --sort --rx srtmfile.osm --sort --merge --wx outfile.osm<br>
<br>
In reality, maybe the problem is entirely with your osm data and not at all with the SRTM. It's possible some of your osm data files have items out of order, which is causing a problem with the merge.<br>
</blockquote>
<br>
Thanks for looking into this. I added --sort after each input file name and it still gives the same error. The OSM map file was generated by osmosis, which I used to split planet.osm.<br>
<br>
java -Xmx1560m -jar /home/nav/scripts/osmosis.jar --read-xml file="/home/nav/temp/tile.63255095.osm" --sort --read-xml file="/home/nav/temp/eletile.63255095.osm" --sort --merge --write-xml file="/home/nav/temp/mergedtile.63255095.osm"<br>
<br>
</blockquote></div></div>
I had a quick play and think I've found the problem.<br>
<br>
Firstly, when you receive an error in osmosis, look further down the stack trace. The real reason is almost always in there somewhere. Osmosis is multi-threaded so it is difficult to have the real reason show up at the start of the error message. In your case the first bit of error was from the input xml readers detecting that the merge task had aborted. The merge task output was further down the error messages.<br>
<br>
In this case there were two problems:<br>
1. The data was unsorted.<br>
2. Some data doesn't appear to have dates attached (I haven't found the offending entry yet).<br>
<br>
This command line works:<br>
osmosis --rx tile.63255095.osm.gz enableDateParsing=false --sort --rx eletile.63255095.osm.gz enableDateParsing=false --sort --merge --wx out.osm.gz<br>
<br>
It sorts all data before the merge, and ignores all dates in the input data (replacing them with current system time). This is a bit of a kludge, ideally the data itself should be fixed to include the correct dates but hopefully it points you in the right direction.<br>
<br>
Hope that helps.<br><font color="#888888">
Brett<br>
<br>
</font></blockquote></div><br>The SRTM data has no timestamps, which is causing the problem. So you only need to set enableDateParsing=false for that file. It might be good to show a warning if an entity has no timestamp attribute in the XML. I propose adding the following lines in BaseElementProcessor.java:<br>
<br>+ private static final Logger log = Logger.getLogger(BaseElementProcessor.class.getName());<br><br>...<br><br> protected TimestampContainer createTimestampContainer(String data) {<br> if (enableDateParsing) {<br>
+ if (data == null || data == "") {<br>+ log.warning("Element missing timestamp attribute.");<br>+ }<br> return new UnparsedTimestampContainer(timestampFormat, data);<br>
} else {<br> return dummyTimestampContainer;<br> }<br> }<br><br>That encapsulates it in the lowest possible level, but the downside is that it doesn't report anything about the problematic element--Ideally it would show the id and whether it's a node, way or relation. That could conceivably be solved by passing that information to the createTimestampContainer method, but that seems sloppy.<br>
<br>Karl<br>