[Tile-serving] [openstreetmap/osm2pgsql] Why ways importing speed depends on imported nodes? Is it memory allocation bug? (#1432)

[Valerij Ivashchenko]

@joto But according to osm2pgsql output there is clustering by geometry and indexing runs **after** reading input file. Am I wrong?

```
2021-03-19 05:40:00  osm2pgsql version 1.4.1 (1.4.1-16-gf7993c95)
2021-03-19 05:40:00  Database version: 12.6 (Ubuntu 12.6-0ubuntu0.20.04.1)
2021-03-19 05:40:00  PostGIS version: 3.0
2021-03-19 05:40:00  Node-cache: cache=16000MB, maxblocks=256000*65536, allocation method=11
2021-03-19 05:40:00  Setting up table 'planet_osm_hstore_point'
2021-03-19 05:40:00  Setting up table 'planet_osm_hstore_line'
2021-03-19 05:40:00  Setting up table 'planet_osm_hstore_polygon'
2021-03-19 05:40:00  Setting up table 'planet_osm_hstore_roads'
2021-03-19 22:23:02  Reading input files done in 60182s (16h 43m 2s).
2021-03-19 22:23:02    Processed 0 nodes in 0s - 0/s
2021-03-19 22:23:02    Processed 748079580 ways in 60182s (16h 43m 2s) - 12k/s
2021-03-19 22:23:02    Processed 0 relations in 0s - 0/s
2021-03-19 22:23:02  Clustering table 'planet_osm_hstore_point' by geometry...
2021-03-19 22:23:02  Clustering table 'planet_osm_hstore_line' by geometry...
2021-03-19 22:23:02  Clustering table 'planet_osm_hstore_polygon' by geometry...
2021-03-19 22:23:02  Clustering table 'planet_osm_hstore_roads' by geometry...
2021-03-19 22:23:02  Building index on table 'planet_osm_hstore_ways'
2021-03-19 22:23:02  Building index on table 'planet_osm_hstore_rels'
2021-03-19 22:23:02  Done postprocessing on table 'planet_osm_hstore_rels' in 0s
2021-03-19 22:23:02  Creating geometry index on table 'planet_osm_hstore_line'...
2021-03-19 22:23:02  Creating geometry index on table 'planet_osm_hstore_point'...
2021-03-19 22:23:02  Creating geometry index on table 'planet_osm_hstore_roads'...
2021-03-19 22:23:02  Creating geometry index on table 'planet_osm_hstore_polygon'...
2021-03-19 22:23:02  Creating osm_id index on table 'planet_osm_hstore_line'...
2021-03-19 22:23:02  Creating osm_id index on table 'planet_osm_hstore_polygon'...
2021-03-19 22:23:02  Creating osm_id index on table 'planet_osm_hstore_point'...
2021-03-19 22:23:02  Creating osm_id index on table 'planet_osm_hstore_roads'...
2021-03-19 22:23:02  Analyzing table 'planet_osm_hstore_roads'...
2021-03-19 22:23:02  Analyzing table 'planet_osm_hstore_polygon'...
2021-03-19 22:23:02  Analyzing table 'planet_osm_hstore_line'...
2021-03-19 22:23:02  Analyzing table 'planet_osm_hstore_point'...
2021-03-19 22:23:02  All postprocessing on table 'planet_osm_hstore_point' done in 0s.
2021-03-19 22:23:02  All postprocessing on table 'planet_osm_hstore_polygon' done in 0s.
2021-03-19 22:23:02  All postprocessing on table 'planet_osm_hstore_roads' done in 0s.
2021-03-19 22:23:02  All postprocessing on table 'planet_osm_hstore_line' done in 0s.

2021-03-20 13:03:58  Done postprocessing on table 'planet_osm_hstore_ways' in 52856s (14h 40m 56s)
2021-03-20 13:03:58  node cache: stored: 0(0.00%), storage efficiency: 0.00% (dense blocks: 0, sparse nodes: 0), hit rate: 0.00%
2021-03-20 13:03:58  osm2pgsql took 113038s (31h 23m 58s) overall.
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/discussions/1432#discussioncomment-506087
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20210320/c8a93ccb/attachment-0001.htm>