[Tile-serving] [openstreetmap/osm2pgsql] Segmentation fault (core dumped) (Issue #2079)
Goncalo Oliveira
notifications at github.com
Thu Sep 14 10:49:43 UTC 2023
## What version of osm2pgsql are you using?
Built from repository
```
osm2pgsql version 1.9.2 (1.9.2-8-g34d52652)
Build: RelWithDebInfo
Compiled using the following library versions:
Libosmium 2.19.0
Proj [API 6] 8.2.1
Lua 5.3.6
```
## What operating system and PostgreSQL/PostGIS version are you using?
Using Ubuntu 22.034x64
```
Linux 5.15.0-67-generic #74-Ubuntu SMP Wed Feb 22 14:14:39 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
```
Using PostgreSQL 15.4 on a different host (DigitalOcean managed cluster)
```
PostgreSQL 15.4 on x86_64-pc-linux-gnu, compiled by gcc, a 42a124907b p 428fabf8fc, 64-bit
POSTGIS="3.2.4 23.7.0-3914-gbebcaee610" [EXTENSION] PGSQL="150" GEOS="3.9.2-CAPI-1.14.3" PROJ="7.2.1" LIBXML="2.10.4" LIBJSON="0.17" LIBPROTOBUF="1.4.1" WAGYU="0.5.0 (Internal)" TOPOLOGY
```
## Tell us something about your system
4 GB RAM
2 vcpu (dedicated)
25 GB disk space
VM on DigitalOcean
## What did you do exactly?
```
wget -q -O data.osm.pbf https://download.geofabrik.de/europe/great-britain/england-latest.osm.pbf || exit 1
osm2pgsql --create --slim -v \
--hstore-all \
data.osm.pbf \
|| exit 1
```
## What did you expect to happen?
The import process to finish without errors.
## What did happen instead?
```
2023-09-13 13:55:53 osm2pgsql version 1.9.2 (1.9.2-8-g34d52652)
2023-09-13 13:55:53 [00] Database version: 15.4
2023-09-13 13:55:53 [00] PostGIS version: 3.2
2023-09-13 13:55:53 [00] Found properties table 'osm2pgsql_properties': false.
2023-09-13 13:55:53 [00] Storing properties to table '"public"."osm2pgsql_properties"'.
2023-09-13 13:55:53 [00] Storing attributes='false'
2023-09-13 13:55:53 [00] Storing db_format='1'
2023-09-13 13:55:53 [00] Storing flat_node_file=''
2023-09-13 13:55:53 [00] Storing output='pgsql'
2023-09-13 13:55:53 [00] Storing prefix='planet_osm'
2023-09-13 13:55:53 [00] Storing style='/usr/local/share/osm2pgsql/default.style'
2023-09-13 13:55:53 [00] Storing updatable='true'
2023-09-13 13:55:53 [00] Storing version='1.9.2'
2023-09-13 13:55:53 [00] Reading file: data.osm.pbf
2023-09-13 13:55:53 [00] Started pool with 2 threads.
2023-09-13 13:55:53 [00] Mid: pgsql, cache=800, db_format=1
2023-09-13 13:55:53 [00] Setting up table 'planet_osm_nodes'
NOTICE: table "planet_osm_nodes" does not exist, skipping
2023-09-13 13:55:53 [00] Setting up table 'planet_osm_ways'
NOTICE: table "planet_osm_ways" does not exist, skipping
2023-09-13 13:55:53 [00] Setting up table 'planet_osm_rels'
NOTICE: table "planet_osm_rels" does not exist, skipping
2023-09-13 13:55:53 [00] Using projection SRS 3857 (Spherical Mercator)
2023-09-13 13:55:53 [00] Using built-in tag transformations
2023-09-13 13:55:53 [00] Middle 'pgsql' options:
2023-09-13 13:55:53 [00] nodes: true
2023-09-13 13:55:53 [00] untagged_nodes: true
2023-09-13 13:55:53 [00] db_format: 1
2023-09-13 13:55:53 [00] use_flat_node_file: false
2023-09-13 13:55:53 [00] way_node_index_id_shift: 5
2023-09-13 13:55:53 [00] with_attributes: false
2023-09-13 13:55:53 [00] Setting up table 'planet_osm_point'
NOTICE: table "planet_osm_point" does not exist, skipping
NOTICE: table "planet_osm_point_tmp" does not exist, skipping
2023-09-13 13:55:53 [00] Setting up table 'planet_osm_line'
NOTICE: table "planet_osm_line" does not exist, skipping
NOTICE: table "planet_osm_line_tmp" does not exist, skipping
2023-09-13 13:55:53 [00] Setting up table 'planet_osm_polygon'
NOTICE: table "planet_osm_polygon" does not exist, skipping
NOTICE: table "planet_osm_polygon_tmp" does not exist, skipping
2023-09-13 13:55:53 [00] Setting up table 'planet_osm_roads'
NOTICE: table "planet_osm_roads" does not exist, skipping
NOTICE: table "planet_osm_roads_tmp" does not exist, skipping
2023-09-13 14:45:54 [00] Reading input files done in 3001s (50m 1s).
2023-09-13 14:45:54 [00] Processed 137746877 nodes in 494s (8m 14s) - 279k/s
2023-09-13 14:45:54 [00] Processed 21153498 ways in 847s (14m 7s) - 25k/s
2023-09-13 14:45:54 [00] Processed 218907 relations in 1660s (27m 40s) - 132/s
2023-09-13 14:45:54 [00] Overall memory usage: peak=2033MByte current=1591MByte
2023-09-13 14:45:56 [01] Starting task...
2023-09-13 14:45:56 [02] Starting task...
2023-09-13 14:45:56 [02] Clustering table 'planet_osm_line' by geometry...
2023-09-13 14:45:56 [02] Using native order for clustering table 'planet_osm_line'
2023-09-13 14:45:56 [01] Clustering table 'planet_osm_point' by geometry...
2023-09-13 14:45:56 [01] Using native order for clustering table 'planet_osm_point'
2023-09-13 14:47:56 [01] Creating geometry index on table 'planet_osm_point'...
2023-09-13 14:50:58 [01] Creating osm_id index on table 'planet_osm_point'...
2023-09-13 14:51:25 [01] Analyzing table 'planet_osm_point'...
2023-09-13 14:51:35 [01] Done task in 339181ms.
2023-09-13 14:51:35 [01] Starting task...
2023-09-13 14:51:35 [01] Clustering table 'planet_osm_polygon' by geometry...
2023-09-13 14:51:35 [01] Using native order for clustering table 'planet_osm_polygon'
2023-09-13 14:53:15 [02] Creating geometry index on table 'planet_osm_line'...
2023-09-13 15:02:10 [02] Creating osm_id index on table 'planet_osm_line'...
2023-09-13 15:02:49 [02] Analyzing table 'planet_osm_line'...
2023-09-13 15:03:02 [02] Done task in 1026100ms.
2023-09-13 15:03:02 [02] Starting task...
2023-09-13 15:03:02 [02] Clustering table 'planet_osm_roads' by geometry...
2023-09-13 15:03:02 [02] Using native order for clustering table 'planet_osm_roads'
2023-09-13 15:03:28 [02] Creating geometry index on table 'planet_osm_roads'...
2023-09-13 15:03:42 [02] Creating osm_id index on table 'planet_osm_roads'...
2023-09-13 15:03:43 [02] Analyzing table 'planet_osm_roads'...
2023-09-13 15:03:43 [02] Done task in 41632ms.
2023-09-13 15:03:43 [02] Starting task...
2023-09-13 15:03:43 [02] Done task in 0ms.
2023-09-13 15:03:43 [02] Starting task...
2023-09-13 15:03:43 [00] Done postprocessing on table 'planet_osm_nodes' in 0s
2023-09-13 15:03:44 [02] Building index on table 'planet_osm_ways'
2023-09-13 15:05:08 [01] Creating geometry index on table 'planet_osm_polygon'...
2023-09-13 15:17:27 [02] Starting task...
2023-09-13 15:17:28 [02] Building index on table 'planet_osm_rels'
Segmentation fault (core dumped)
```
## What did you do to try analyzing the problem?
I executed again using the `--verbose` argument to try to get more details. I've double checked the machine's memory, cpu and disk, to ensure I wasn't out of any of the required resources. Memory doesn't go higher than 50%, CPU 60% and disk utilization is also quire comfortable although I don't remember the exact values.
I've set up a different machine with 8 GB and 4 vcpus. Got the same results. The memory never surpassed 20%.
```
2023-09-14 09:27:34 osm2pgsql version 1.9.2 (1.9.2-8-g34d52652)
2023-09-14 09:27:34 [00] Database version: 15.4
2023-09-14 09:27:34 [00] PostGIS version: 3.2
2023-09-14 09:27:34 [00] Found properties table 'osm2pgsql_properties': false.
2023-09-14 09:27:34 [00] Storing properties to table '"public"."osm2pgsql_properties"'.
2023-09-14 09:27:34 [00] Storing attributes='false'
2023-09-14 09:27:34 [00] Storing db_format='1'
2023-09-14 09:27:34 [00] Storing flat_node_file=''
2023-09-14 09:27:34 [00] Storing output='pgsql'
2023-09-14 09:27:34 [00] Storing prefix='planet_osm'
2023-09-14 09:27:34 [00] Storing style='/usr/local/share/osm2pgsql/default.style'
2023-09-14 09:27:34 [00] Storing updatable='true'
2023-09-14 09:27:34 [00] Storing version='1.9.2'
2023-09-14 09:27:34 [00] Reading file: data.osm.pbf
2023-09-14 09:27:34 [00] Started pool with 4 threads.
2023-09-14 09:27:34 [00] Mid: pgsql, cache=800, db_format=1
2023-09-14 09:27:34 [00] Setting up table 'planet_osm_nodes'
NOTICE: table "planet_osm_nodes" does not exist, skipping
2023-09-14 09:27:34 [00] Setting up table 'planet_osm_ways'
NOTICE: table "planet_osm_ways" does not exist, skipping
2023-09-14 09:27:34 [00] Setting up table 'planet_osm_rels'
NOTICE: table "planet_osm_rels" does not exist, skipping
2023-09-14 09:27:34 [00] Using projection SRS 3857 (Spherical Mercator)
2023-09-14 09:27:34 [00] Using built-in tag transformations
2023-09-14 09:27:34 [00] Middle 'pgsql' options:
2023-09-14 09:27:34 [00] nodes: true
2023-09-14 09:27:34 [00] untagged_nodes: true
2023-09-14 09:27:34 [00] db_format: 1
2023-09-14 09:27:34 [00] use_flat_node_file: false
2023-09-14 09:27:34 [00] way_node_index_id_shift: 5
2023-09-14 09:27:34 [00] with_attributes: false
2023-09-14 09:27:34 [00] Setting up table 'planet_osm_point'
NOTICE: table "planet_osm_point" does not exist, skipping
NOTICE: table "planet_osm_point_tmp" does not exist, skipping
2023-09-14 09:27:34 [00] Setting up table 'planet_osm_line'
NOTICE: table "planet_osm_line" does not exist, skipping
NOTICE: table "planet_osm_line_tmp" does not exist, skipping
2023-09-14 09:27:34 [00] Setting up table 'planet_osm_polygon'
NOTICE: table "planet_osm_polygon" does not exist, skipping
NOTICE: table "planet_osm_polygon_tmp" does not exist, skipping
2023-09-14 09:27:34 [00] Setting up table 'planet_osm_roads'
NOTICE: table "planet_osm_roads" does not exist, skipping
NOTICE: table "planet_osm_roads_tmp" does not exist, skipping
2023-09-14 10:10:19 [00] Reading input files done in 2565s (42m 45s).
2023-09-14 10:10:19 [00] Processed 137779558 nodes in 504s (8m 24s) - 273k/s
2023-09-14 10:10:19 [00] Processed 21159803 ways in 858s (14m 18s) - 25k/s
2023-09-14 10:10:19 [00] Processed 218933 relations in 1203s (20m 3s) - 182/s
2023-09-14 10:10:19 [00] Overall memory usage: peak=2121MByte current=1689MByte
2023-09-14 10:10:21 [01] Starting task...
2023-09-14 10:10:21 [03] Starting task...
2023-09-14 10:10:21 [02] Starting task...
2023-09-14 10:10:21 [04] Starting task...
2023-09-14 10:10:21 [01] Clustering table 'planet_osm_point' by geometry...
2023-09-14 10:10:21 [01] Using native order for clustering table 'planet_osm_point'
2023-09-14 10:10:21 [03] Clustering table 'planet_osm_line' by geometry...
2023-09-14 10:10:21 [02] Clustering table 'planet_osm_polygon' by geometry...
2023-09-14 10:10:21 [04] Clustering table 'planet_osm_roads' by geometry...
2023-09-14 10:10:21 [04] Using native order for clustering table 'planet_osm_roads'
2023-09-14 10:10:21 [03] Using native order for clustering table 'planet_osm_line'
2023-09-14 10:10:21 [02] Using native order for clustering table 'planet_osm_polygon'
2023-09-14 10:11:48 [04] Creating geometry index on table 'planet_osm_roads'...
2023-09-14 10:12:55 [04] Creating osm_id index on table 'planet_osm_roads'...
2023-09-14 10:13:00 [04] Analyzing table 'planet_osm_roads'...
2023-09-14 10:13:14 [04] Done task in 173079ms.
2023-09-14 10:13:14 [04] Starting task...
2023-09-14 10:13:14 [04] Done task in 0ms.
2023-09-14 10:13:14 [04] Starting task...
2023-09-14 10:13:14 [00] Done postprocessing on table 'planet_osm_nodes' in 0s
2023-09-14 10:13:14 [04] Building index on table 'planet_osm_ways'
2023-09-14 10:16:07 [01] Creating geometry index on table 'planet_osm_point'...
2023-09-14 10:20:14 [03] Creating geometry index on table 'planet_osm_line'...
2023-09-14 10:21:28 [01] Creating osm_id index on table 'planet_osm_point'...
2023-09-14 10:21:49 [01] Analyzing table 'planet_osm_point'...
2023-09-14 10:21:59 [01] Done task in 697966ms.
2023-09-14 10:21:59 [01] Starting task...
2023-09-14 10:21:59 [01] Building index on table 'planet_osm_rels'
2023-09-14 10:22:38 [01] Done task in 38902ms.
2023-09-14 10:23:28 [02] Creating geometry index on table 'planet_osm_polygon'...
2023-09-14 10:31:38 [03] Creating osm_id index on table 'planet_osm_line'...
2023-09-14 10:32:05 [03] Analyzing table 'planet_osm_line'...
2023-09-14 10:32:10 [03] Done task in 1308952ms.
Segmentation fault (core dumped)
```
--
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/2079
You are receiving this because you are subscribed to this thread.
Message ID: <openstreetmap/osm2pgsql/issues/2079 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20230914/6c090fa2/attachment-0001.htm>
More information about the Tile-serving
mailing list