[OSM-dev] Osmosis Failing to Read Particular Replicate Files

[Aaron Schaefer]

On Thu, Apr 8, 2010 at 7:45 AM, Brett Henderson <brett at bretth.com> wrote:
> Did you get any further with this?  The error is unlikely to be caused by
> Osmosis.  The following line in the stack trace appears to indicate a
> network problem:
> Caused by: java.net.SocketException: Connection reset
>
> ...
>
> There are a few things you can try.
>
> I never set the maxInterval to 0.  It is worth setting to a smaller time
> period so that if encounter problems you don't have to start from the
> start.  You've only downloaded about 30 files which is just over a day when
> encountering the error though so this isn't a huge number.  But the more
> files you download, the greater the chances of tripping over a network
> issue.  I'd suggest setting it to one day (ie. 86400) and running it several
> times until you catch up.  If you're importing into a database then there's
> no reason to try to get it all in one hit.  Patching a planet file is more
> time consuming, but if you're doing that you probably should be using daily
> diffs anyway.
> Have you tried using wget or curl to download the file?  It would be worth
> verifying that you can download it outside of Osmosis.


The file does download just fine using wget/curl, and I always thought
it was odd that it was failing on the same file even on different
mirrors. I had assumed that due to the sequence number that this was
not the first file it had tried to download, but that might not be the
case. Because you mentioned it looked like a network issue, I ran
things verbosely and then tried adding the trailing slash to the
baseURL...that fixed it!

When you initialize the working directory, the example
configuration.txt file lists
'baseUrl=http://planet.openstreetmap.org/minute-replicate' (note
there's no trailing slash), so I had just switched /minute/hour/ and
thought it would be fine. So, dumb mistake, but it works fine now.

Per your advice, I did switch maxInterval as well, but had another
question. Daily diffs would be great, but from my understanding after
talking with people on IRC, was that there wasn't an easy way to
download and import daily diffs using osmosis because there are no
managed state.txt files like there are with the minute/hour replicate
diffs. So, the conclusion I came to was to just run a daily cron job
that would grab a days worth of hourly diffs and apply those. Is there
a better way? Thanks again for your help...I really appreciate it.

--
Aaron Schaefer