[Tile-serving] [openstreetmap/osm2pgsql] Segmentation Fault during Nominatim daily update (#1014)

[Arvind Kartik]

After the full planet import via Nominatim, I've started the daily updates and it gets killed due to osm2pgsql throwing "segmentation fault"

```
Processing: Node(2000k 125.0k/s) Way(62k 3.44k/s) Relation(0 0.00/s)Segmentation fault
Error executing osm2pgsql: 139
```
Daily update command & summary
```nominatim at myhost:~/Nominatim-3.4.0/build$ ./utils/update.php --verbose --import-osmosis-all
Currently at sequence 2628 (22/11/2019 23:58:47) - 1 indexed
/srv/nominatim/.local/bin/pyosmium-get-changes --server http://planet.openstreetmap.org/replication/day -o /srv/nominatim/Nominatim-3.4.0/build/osmosischange.osc -s 30 -I 2628
/srv/nominatim/Nominatim-3.4.0/build/osm2pgsql/osm2pgsql -klas --number-processes 1 -C 2000 -O gazetteer -S /srv/nominatim/Nominatim-3.4.0/settings/import-full.style -d nominatim -P 5432 /srv/nominatim/Nominatim-3.4.0/build/osmosischange.osc
osm2pgsql version 1.2.0 (64 bit id space)
Allocating memory for dense node cache
Allocating dense node cache in one big chunk
Allocating memory for sparse node cache
Sharing dense sparse
Node-cache: cache=2000MB, maxblocks=32000*65536, allocation method=11
Mid: pgsql, cache=2000
Parsing gazetteer style file '/srv/nominatim/Nominatim-3.4.0/settings/import-full.style'.
Using projection SRS 4326 (Latlong)
Reading in file: /srv/nominatim/Nominatim-3.4.0/build/osmosischange.osc
Using XML parser.
Processing: Node(2000k 125.0k/s) Way(62k 3.44k/s) Relation(0 0.00/s)Segmentation fault
Error executing osm2pgsql: 139
```

Hardware configuration:
1. RAM Size - 96GB 
2. CPUs - 16 cores, 32 threads
3. Disk size - 1TB

Operating system:
1. Ubuntu 16.04 LTS
2. Architecture - 64-bit

Nominatim dependencies packages details:
1. php 7.1
2. postgresql 10
3. postgis 2.4
4. osm2psql 1.2.0

Postgres Settings:
```
shared_buffers = 8GB
maintenance_work_mem = 10GB
work_mem = 50MB
effective_cache_size = 24GB
synchronous_commit = off
checkpoint_timeout = 10min
checkpoint_completion_target = 0.9
autovacuum = on
fsync = on
full_page_writes = on
autovacuum_vacuum_cost_delay = 10ms 
autovacuum_vacuum_cost_limit = 1000
```

Is there something I'm doing wrong?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/1014
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20191204/30adde07/attachment.html>