<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 10pt;
font-family:Verdana
}
--></style>
</head>
<body class='hmmessage'>
Your idea seams nifty, although it's giving an error on the --rx command. I don't know Osmosis well enough to figure out where the error comes from. Is it just an error in the syntax or is it because of the osm-file?<br><br><br>C:\osmosis-0.35\bin>osmosis --rx amsterdam.osm --rx empty.osm --dc --sort-change<br>-0.6 --simc --wxc amsterdam_fixed.osc<br>4-mei-2010 13:29:22 org.openstreetmap.osmosis.core.Osmosis run<br>INFO: Osmosis Version 0.35<br>4-mei-2010 13:29:23 org.openstreetmap.osmosis.core.Osmosis run<br>INFO: Preparing pipeline.<br>4-mei-2010 13:29:23 org.openstreetmap.osmosis.core.Osmosis run<br>INFO: Launching pipeline execution.<br>4-mei-2010 13:29:23 org.openstreetmap.osmosis.core.Osmosis run<br>INFO: Pipeline executing, waiting for completion.<br>4-mei-2010 13:29:23 org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskMan<br>ager waitForCompletion<br>SEVERE: Thread for task 1-rx failed<br>org.openstreetmap.osmosis.core.OsmosisRuntimeException: An output error has occu<br>rred, aborting.<br> at org.openstreetmap.osmosis.core.store.DataPostbox.checkForOutputErrors<br>(DataPostbox.java:76)<br> at org.openstreetmap.osmosis.core.store.DataPostbox.populateCentralQueue<br>(DataPostbox.java:127)<br> at org.openstreetmap.osmosis.core.store.DataPostbox.put(DataPostbox.java<br>:182)<br> at org.openstreetmap.osmosis.core.merge.v0_6.impl.DataPostboxSink.proces<br>s(DataPostboxSink.java:34)<br> at org.openstreetmap.osmosis.core.xml.v0_6.impl.NodeElementProcessor.end<br>(NodeElementProcessor.java:117)<br> at org.openstreetmap.osmosis.core.xml.v0_6.impl.OsmHandler.endElement(Os<br>mHandler.java:107)<br> at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source<br>)<br> at org.apache.xerces.parsers.AbstractXMLDocumentParser.emptyElement(Unkn<br>own Source)<br> at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanStartElemen<br>t(Unknown Source)<br> at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContent<br>Dispatcher.dispatch(Unknown Source)<br> at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Un<br>known Source)<br> at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)<br> at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)<br> at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)<br> at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)<br> at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Sour<br>ce)<br> at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source)<br> at javax.xml.parsers.SAXParser.parse(SAXParser.java:198)<br> at org.openstreetmap.osmosis.core.xml.v0_6.XmlReader.run(XmlReader.java:<br>108)<br> at java.lang.Thread.run(Thread.java:619)<br>4-mei-2010 13:29:23 org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskMan<br>ager waitForCompletion<br>SEVERE: Thread for task 2-rx failed<br>org.openstreetmap.osmosis.core.OsmosisRuntimeException: Unable to parse xml file<br> empty.osm. publicId=(null), systemId=(null), lineNumber=1, columnNumber=1.<br> at org.openstreetmap.osmosis.core.xml.v0_6.XmlReader.run(XmlReader.java:<br>113)<br> at java.lang.Thread.run(Thread.java:619)<br>Caused by: org.xml.sax.SAXParseException: Content is not allowed in prolog.<br> at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Un<br>known Source)<br> at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)<br><br> at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)<br> at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)<br> at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)<br> at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source)<br> at org.apache.xerces.impl.XMLDocumentScannerImpl$PrologDispatcher.dispat<br>ch(Unknown Source)<br> at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Un<br>known Source)<br> at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)<br> at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)<br> at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)<br> at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)<br> at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Sour<br>ce)<br> at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source)<br> at javax.xml.parsers.SAXParser.parse(SAXParser.java:198)<br> at org.openstreetmap.osmosis.core.xml.v0_6.XmlReader.run(XmlReader.java:<br>108)<br> ... 1 more<br>4-mei-2010 13:29:23 org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskMan<br>ager waitForCompletion<br>SEVERE: Thread for task 3-dc failed<br>org.openstreetmap.osmosis.core.OsmosisRuntimeException: An input error has occur<br>red, aborting.<br> at org.openstreetmap.osmosis.core.store.DataPostbox.checkForInputErrors(<br>DataPostbox.java:88)<br> at org.openstreetmap.osmosis.core.store.DataPostbox.consumeCentralQueue(<br>DataPostbox.java:158)<br> at org.openstreetmap.osmosis.core.store.DataPostbox.hasNext(DataPostbox.<br>java:235)<br> at org.openstreetmap.osmosis.core.change.v0_6.ChangeDeriver.run(ChangeDe<br>river.java:100)<br> at java.lang.Thread.run(Thread.java:619)<br>4-mei-2010 13:29:23 org.openstreetmap.osmosis.core.Osmosis main<br>SEVERE: Execution aborted.<br>org.openstreetmap.osmosis.core.OsmosisRuntimeException: One or more tasks failed<br>.<br> at org.openstreetmap.osmosis.core.pipeline.common.Pipeline.waitForComple<br>tion(Pipeline.java:146)<br> at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:85)<br> at org.openstreetmap.osmosis.core.Osmosis.main(Osmosis.java:30)<br> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)<br> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.<br>java:39)<br> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces<br>sorImpl.java:25)<br> at java.lang.reflect.Method.invoke(Method.java:597)<br> at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Laun<br>cher.java:329)<br> at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.jav<br>a:239)<br> at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(La<br>uncher.java:409)<br> at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:<br>352)<br> at org.codehaus.classworlds.Launcher.main(Launcher.java:31)<br><br><br><hr id="stopSpelling">Date: Tue, 4 May 2010 22:56:18 +1000<br>Subject: Re: [osmosis-dev] Can't load osm file to PostGRESql<br>From: brett@bretth.com<br>To: ibrahim_bouchrika@hotmail.com<br>CC: osmosis-dev@openstreetmap.org<br><br><div class="ecxgmail_quote">On Tue, May 4, 2010 at 10:50 PM, Brett Henderson <span dir="ltr"><<a>brett@bretth.com</a>></span> wrote:<br><blockquote class="ecxgmail_quote" style="padding-left: 1ex;">
<div class="ecxgmail_quote"><div class="ecxim">On Tue, May 4, 2010 at 5:56 PM, Ibrahim Bouchrika <span dir="ltr"><<a>ibrahim_bouchrika@hotmail.com</a>></span> wrote:<br>
<blockquote class="ecxgmail_quote" style="padding-left: 1ex;">
<div>
The database i'm using is a fresh made one, so the error must be duplicate entries in the osm. I have the same problem with a duplicate relation in the new york city file i extracted from a cloudmade map.<br></div></blockquote>
<blockquote class="ecxgmail_quote" style="padding-left: 1ex;"><div><br>Could the duplicate values be a result of a bug while extracting a bbox with osmosis?<br>
</div></blockquote></div><div><br>I doubt it, extracting a bbox should never create multiple instances of the same id. It's more likely to be caused by incorrect use of the --apply-change task when applying minute or hourly diff files. It was possible with older versions of osmosis to apply a full history diff containing multiple changes for a single entity which would result in multiple versions of the same entity being written to the result file.<br>
</div><div class="ecxim"><blockquote class="ecxgmail_quote" style="padding-left: 1ex;"><div><br>Is there a way around those duplicate entries, like perhaps overwriting duplicate data? Going through all those entries manually seems almost impossible.<br>
</div></blockquote></div><div><br>Hmm, I'm not sure about the best way to tackle this ...<br></div></div><br>I seem to remember Frederik having a clever way of removing duplicates. I think he did something like the following:<br>
1. Convert the entire file to an osc file by wrapping the entire contents of the file in osmChange element with an action of create or modify.<br>2. Feed the osc file through the --simplify-change task.<br>3. Create an empty osm file, then use the --apply-change task to add the entire change file to it.<br>
</blockquote><div><br>Found it. His steps were:<br><br><quote><br>Just for laughs - and wonder if you can come up with something better?
- this is what I did to remove the duplicate objects from an existing
OSM file named "faulty.osm":<br>
<br>
echo "<osm></osm>" > empty.osm<br>
<br>
osmosis --rx faulty.osm --rx empty.osm --dc --sort-change-0.6 --<span class="ecxil">simc</span> --wxc good.osc<br>
<br>
osmosis --rxc good.osc --rx empty.osm --ac --wx good.osm<br>
<br>
(The two osmosis steps could be written as one but that would probably make it more confusing.)<br></quote><br><br>Can you try these steps on your cloudmade file and see if that fixes the problem? If it does, Cloudmade has a problem with duplicate data in their files. If it doesn't fix it, then we'll have to dig further :-)<br>
<br>Brett<br></div></div><br> <br /><hr />Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox. <a href='http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_1' target='_new'>Learn more.</a></body>
</html>