[OSM-dev] Download errors for changesets-latest from Planet Replication Sites

[Pierre Béland]

I am sharing below the solution I found to correct the problem I was faced with  using the Windows 10 OS to  download the planet.osm Changesets.
This ssl_error_rx_record_too_long error message arise with https url's  while a client application negociate with a TLS server to download a file. The client is either your navigator or a script with a wget command.

While Trying to bypass https and use the http protocol, my file downloaded was corrupted.
 ssl_error_rx_record_too_long is an indication of bad hand exchange between the client and server at the beginning of the transaction. This is related to how the server or the client are configured and it is possbile to select the protocol for negociation.
The Mozilla team suggests if you your navigator have such TLS negociation problem to revise the key security.tls.version.max in about:config from 4 to 3
    -> about:config : security.tls.version.max = 3    https://support.mozilla.org/en-US/questions/1222787

In Windows Powershell, I did force SSL v1.2 with the following instruction.    [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12After Download via the wget - Invoke-WebRequest instruction, the MD5 checksum was ok. 
Also, Test with the pbzip2 -tvv file.osm.bz2 instruction did confirm that the file is ok.
Using the wsl / Ubuntu as an alternative solution, wget error did come yes this time from an agressive firewall.
 
Pierre 
 

    Le mardi 26 janvier 2021 03 h 58 min 30 s UTC−5, Stephan Knauss <osm at stephans-server.de> a écrit :  
 
 Hello Pierre,

On 26.01.2021 06:05, Pierre Béland via dev wrote:
> I succeed to download changesets-latest.osm.bz2 for 2021-01-01 and 01-22 
> from a Mirror site but the file is not recognized by pbzip2 or python 
> bz2file.

I tried a download and both download and file integrity are fine for me 
on this URL:
https://ftp5.gwdg.de/pub/misc/openstreetmap/planet.openstreetmap.org/planet/changesets-latest.osm.bz2

Verified with md5:
https://ftp5.gwdg.de/pub/misc/openstreetmap/planet.openstreetmap.org/planet/changesets-latest.osm.bz2.md5

ce7a57a2a46c6369f6d286cc385950a5  changesets-latest.osm.bz2

> Trying to download for  2021-01-22 either with wget or from 
> FirefoxxNavigator, the sites below return error code SSL_ERROR_BAD_MAC_READ
> - 
> https://ftp5.gwdg.de/pub/misc/openstreetmap/planet.openstreetmap.org/planet/
> - https://mirror.math.princeton.edu/pub/openstreetmap/planet/
> - https://ftp.osuosl.org/pub/openstreetmap/planet/

You indicate that using different software on your end and multiple 
servers on the other side it fails including some very unusual SSL 
connection problems.

I think this points towards a problem with your infrastructure.
Could it be that you have an aggressive virus scan intercepting your 
network traffic and messing up the file content?

I don't think this is related to OSM.

Stephan
    
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20210127/a4249c1d/attachment-0001.htm>