[Tile-serving] [osm2pgsql-dev/osm2pgsql] TLS issues with osm2pgsql-replication (Discussion #2273)

Mon Dec 9 12:53:01 UTC 2024

Hi,

First, context

```
Debian 12

osm2pgsql version 2.0.1
Build: None
Compiled using the following library versions:
Libosmium 2.19.0
Proj 9.1.1
Lua 5.3.6

replication_base_url: https://download.geofabrik.de/europe/united-kingdom-updates
```

I'm trying to run `osm2pgsql-replication update` in this machine and I'm getting SSL errors; not with the database, but with the replication source, it seems. Bellow are the logs with `-v` option.

```bash
osm2pgsql-replication update -v 
2024-12-09 11:41:17 [INFO]: Using replication service 'https://download.geofabrik.de/europe/united-kingdom-updates'.
2024-12-09 11:41:17 [DEBUG]: Starting new HTTPS connection (1): download.geofabrik.de:443
2024-12-09 11:41:17 [DEBUG]: https://download.geofabrik.de:443 "GET /europe/united-kingdom-updates/state.txt HTTP/1.1" 200 123
2024-12-09 11:41:17 [DEBUG]: Applying 3 sequence(s) (382 → 379), covering 2 day(s) 23 hour(s) 59 minute(s) 27 second(s) (259167 sec) of changes (2024-12-05T21:21:18Z → 2024-12-08T21:20:45Z)
2024-12-09 11:41:17 [DEBUG]: Calling osm2pgsql with: /usr/bin/osm2pgsql --append --slim --prefix planet_osm /tmp/tmpiveprhuw/osm2pgsql_diff.osc.gz
2024-12-09 11:41:17 [DEBUG]: Importing from sequence 379
2024-12-09 11:41:17 [DEBUG]: Starting new HTTPS connection (1): download.geofabrik.de:443
2024-12-09 11:41:17 [DEBUG]: https://download.geofabrik.de:443 "GET /europe/united-kingdom-updates/state.txt HTTP/1.1" 200 123
2024-12-09 11:41:17 [DEBUG]: Starting new HTTPS connection (1): download.geofabrik.de:443
2024-12-09 11:41:18 [DEBUG]: https://download.geofabrik.de:443 "GET /europe/united-kingdom-updates/000/000/380.osc.gz HTTP/1.1" 200 2547392
2024-12-09 11:41:18 [DEBUG]: Downloaded change 380. (501038 kB available in download buffer)
2024-12-09 11:41:18 [DEBUG]: Starting new HTTPS connection (1): download.geofabrik.de:443
2024-12-09 11:41:18 [DEBUG]: https://download.geofabrik.de:443 "GET /europe/united-kingdom-updates/000/000/381.osc.gz HTTP/1.1" 200 2412863
2024-12-09 11:41:18 [DEBUG]: Downloaded change 381. (490515 kB available in download buffer)
2024-12-09 11:41:18 [DEBUG]: Starting new HTTPS connection (1): download.geofabrik.de:443
2024-12-09 11:41:19 [DEBUG]: https://download.geofabrik.de:443 "GET /europe/united-kingdom-updates/000/000/382.osc.gz HTTP/1.1" 200 2320764
2024-12-09 11:41:19 [DEBUG]: Downloaded change 382. (480543 kB available in download buffer)
2024-12-09 11:41:20  osm2pgsql version 2.0.1
2024-12-09 11:41:20  Database version: 17.2
2024-12-09 11:41:20  PostGIS version: 3.5
2024-12-09 11:41:20  Loading properties from table '"public"."osm2pgsql_properties"'.
2024-12-09 11:41:21  Not using flat node file (same as on import).
2024-12-09 11:41:21  Using output 'flex' (same as on import).
2024-12-09 11:41:21  Using style file '/data/geocoder.lua' (same as on import).
2024-12-09 11:41:21  Storing properties to table '"public"."osm2pgsql_properties"'.
2024-12-09 11:41:22  Storing properties to table '"public"."osm2pgsql_properties"'.
2024-12-09 11:49:31  Reading input files done in 489s (8m 9s).                            
2024-12-09 11:49:31    Processed 234592 nodes in 40s - 6k/s
2024-12-09 11:49:31    Processed 50206 ways in 398s (6m 38s) - 126/s
2024-12-09 11:49:31    Processed 851 relations in 51s - 17/s
2024-12-09 11:49:35  Going over 10276 pending ways (using 4 threads)
Left to process: 0.......
2024-12-09 11:51:35  Processing 10276 pending ways took 119s (1m 59s) at a rate of 86.35/s
2024-12-09 11:51:35  Going over 2639 pending relations (using 4 threads)
Left to process: 0......
2024-12-09 11:53:17  Processing 2639 pending relations took 102s (1m 42s) at a rate of 25.87/s
2024-12-09 11:53:17  Skipping stage 1c for nodes (no marked nodes).
2024-12-09 11:53:17  Skipping stage 1c for ways (no marked ways).
2024-12-09 11:53:17  No marked nodes or ways (Skipping stage 2).
2024-12-09 11:53:17  Done postprocessing on table 'planet_osm_nodes' in 0s
2024-12-09 11:53:17  Done postprocessing on table 'planet_osm_ways' in 0s
2024-12-09 11:53:17  Done postprocessing on table 'planet_osm_rels' in 0s
2024-12-09 11:53:17  All postprocessing on table 'boundaries' done in 0s.
2024-12-09 11:53:17  All postprocessing on table 'roads' done in 0s.
2024-12-09 11:53:17  All postprocessing on table 'addrs' done in 0s.
2024-12-09 11:53:17  Storing properties to table '"public"."osm2pgsql_properties"'.
2024-12-09 11:53:17  osm2pgsql took 717s (11m 57s) overall.
2024-12-09 11:53:17 [DEBUG]: Starting new HTTPS connection (1): download.geofabrik.de:443
2024-12-09 11:53:17 [DEBUG]: https://download.geofabrik.de:443 "GET /europe/united-kingdom-updates/000/000/382.state.txt HTTP/1.1" 200 123
2024-12-09 11:53:17 [CRITICAL]: Exception during execution: SSL connection has been closed unexpectedly
```

At a certain point, I get the above error, which is awkward, since *curling* that address (or with *wget*) works fine on the same machine and it's always at this point.

I tried setting `SSL_VERSION=tlsv1.2` and upgrading `openssl` and `libssl-dev`  but that didn't work out though.

It always happens at this point and not before, giving me the impression that it's not the source, but maybe something with the client?

Does anyone have any hints on what could be happening?

-- 
Reply to this email directly or view it on GitHub:
https://github.com/osm2pgsql-dev/osm2pgsql/discussions/2273
You are receiving this because you are subscribed to this thread.

Message ID: <osm2pgsql-dev/osm2pgsql/repo-discussions/2273 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20241209/25fc6bc3/attachment.htm>