[Tile-serving] [openstreetmap/osm2pgsql] Update stuck waiting for postgresql "COPY ... FROM STDIN" to finish (#1273)

[mtmail]

* psql (PostgreSQL) 12.4 (Ubuntu 12.4-0ubuntu0.20.04.1)
* osm2pgsql version 1.3.0, Compiled using the following library versions: Libosmium 2.15.6. Lua support not included
* I verified in `pg_settings` that `jit=off` and `jit_above_cost=-1`

Running any daily Nominatim update gets stuck with `COPY "place" FROM STDIN` Postgresql query running and CPU core at 100% and no disk activity. The `osmosischange.osc` file is about 1.2GB but no change in 20 hours.

Nominatim's `./utils/update.php --import-osmosis --index-instances 30` executes `osm2pgsql --hstore --latlong --append --slim --number-processes 1 --cache 2000 --output gazetteer --style /srv/nominatim/Nominatim/settings/import-full.style --database nominatim --port 5432 --flat-nodes /ssd/flatnode.file`

Output to STDOUT stops after
```
[2020-09-16 22:37:20] Node stats: total(4185341), max(7899006801) in 105s
[2020-09-16 22:37:20] Way stats: total(557522), max(846592436) in 75s
[2020-09-16 22:37:20] Relation stats: total(11637), max(11610493) in 6785s
```

Attached the Postgresql process list and stack trace of the osm2pgsql process. Let me know if there's anything else I can provide.

[postgresql_process_list.txt](https://github.com/openstreetmap/osm2pgsql/files/5241007/postgresql_process_list.txt)
[stack_trace.txt](https://github.com/openstreetmap/osm2pgsql/files/5241011/stack_trace.txt)

Seeing the issue on two servers. Both fresh installed with last week's planet pdf dump imported.

I kind of expect that you advise me to run use `--index-instances 1` instead. But maybe it's a threading issue that can be addressed.


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/1273
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20200917/b438e64e/attachment.htm>