[OSM-dev] Osmosis Failing to Read Particular Replicate Files

[Aaron Schaefer]

So I'm trying to set up downloads of the hour-replicate diffs and
osmosis seems to be having issues reading certain files from the
server. I did a full planet download and dump into my database from
the 03-Apr-2010 03:38 planet-latest.osm.bz2.

I created a working directory for the diffs and ran 'osmosis --rri
workingDirectory=.' to initialize the necessary files and then
downloaded the Fri Apr 02 23:02:09 UTC 2010 state.txt file (a few
hours earlier than my previous full import). I updated
configuration.txt to have the proper baseUrl and set the maxInterval =
0. Now when I try to get the diffs, this is what happens:

--------
$  osmosis --rri workingDirectory=. --wxc diffs.osc.gz
Apr 6, 2010 12:21:07 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Osmosis Version 0.34
Apr 6, 2010 12:21:08 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Preparing pipeline.
Apr 6, 2010 12:21:08 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Launching pipeline execution.
Apr 6, 2010 12:21:08 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Pipeline executing, waiting for completion.
Apr 6, 2010 12:25:26 PM
org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager
waitForCompletion
SEVERE: Thread for task 1-rri failed
org.openstreetmap.osmosis.core.OsmosisRuntimeException: Unable to read
the changeset file 000/003/246.osc.gz from t
he server.
        at org.openstreetmap.osmosis.core.merge.v0_6.BaseReplicationDownloader.downloadReplicationFile(BaseReplicat
ionDownloader.java:128)
        at org.openstreetmap.osmosis.core.merge.v0_6.BaseReplicationDownloader.download(BaseReplicationDownloader.j
ava:236)
        at org.openstreetmap.osmosis.core.merge.v0_6.BaseReplicationDownloader.runImpl(BaseReplicationDownloader.ja
va:275)
        at org.openstreetmap.osmosis.core.merge.v0_6.BaseReplicationDownloader.run(BaseReplicationDownloader.java:3
45)
        at java.lang.Thread.run(Unknown Source)
Caused by: java.net.SocketException: Connection reset
        at java.net.SocketInputStream.read(Unknown Source)
        at java.io.BufferedInputStream.read1(Unknown Source)
        at java.io.BufferedInputStream.read(Unknown Source)
        at sun.net.www.MeteredStream.read(Unknown Source)
        at java.io.FilterInputStream.read(Unknown Source)
        at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(Unknown
Source)
        at java.io.BufferedInputStream.read1(Unknown Source)
        at java.io.BufferedInputStream.read(Unknown Source)
        at java.io.FilterInputStream.read(Unknown Source)
        at org.openstreetmap.osmosis.core.merge.v0_6.BaseReplicationDownloader.downloadReplicationFile(BaseReplicationDownloader.java:114)
        ... 4 more
Apr 6, 2010 12:25:26 PM org.openstreetmap.osmosis.core.Osmosis main
SEVERE: Execution aborted.
org.openstreetmap.osmosis.core.OsmosisRuntimeException: One or more
tasks failed.
        at org.openstreetmap.osmosis.core.pipeline.common.Pipeline.waitForCompletion(Pipeline.java:146)
        at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:85)
        at org.openstreetmap.osmosis.core.Osmosis.main(Osmosis.java:30)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
        at org.codehaus.classworlds.Launcher.main(Launcher.java:31)
--------

It always fails on the 000/003/246.osc.gz file, and I've verified that
it does exist on the server. I tried using both
planet.openstreetmap.org and the heanet mirror with the same results.
I had actually tried this process previously with an older full planet
dump, and it was giving me an error on the 000/003/014.osc.gz file
(which was also present on the server)...I figured maybe it was just a
fluke and that particular file had been corrupted, but now I'm seeing
it error out on the more recent diff as well.

Is anyone else seeing these errors when trying to generate a diff file?

--
Aaron Schaefer