[Tile-serving] [openstreetmap/osm2pgsql] non slim mode planet import skips some data (#1555)

[zimirrr]

## What version of osm2pgsql are you using?

2021-08-17 10:06:18  osm2pgsql version 1.5.0
Build: Release
Compiled using the following library versions:
Libosmium 2.17.0
Proj [API 4] Rel. 4.9.3, 15 August 2016
Lua 5.3.4


## What operating system and PostgreSQL/PostGIS version are you using?

Windows server
Database version: 13.3
PostGIS version: 3.1


## Tell us something about your system

1Tb RAM, 24 CPUs


## What did you do exactly?

Import `planet-latest.osm.pbf` into Postgresql with this command

```
osm2pgsql-latest\osm2pgsql.exe --create ^
 --number-processes 8 ^
 --style nocolumns.style ^
 --hstore-all ^
 --hstore-match-only ^
 --hstore-add-index ^
 --latlong ^
 --multi-geometry ^
 --prefix=%table_prefix% ^
 --database=%PGDATABASE% ^
 %osm_file%
```
*It takes ~9 hours*

## What did you expect to happen?

I've expected to see every building in DB that I see on the openstreetmap.org


## What did happen instead?

instead there were a lot of missing objects.


## What did you do to try analyzing the problem?

First step I did was 
I clipped from planet-latest only small part where there were missing buildings and imported it with same command.
Everything was fine with this import.


Later I saw on https://gis.stackexchange.com/ this post
>What is occurring appears to be a somewhat subtle bug with osm2pgsql where osm2pgsql requires id-sorted nodes, but the API does not guarantee that.

and tried `slim` mode for whole planet and all builds were imported correctly.

slim mode is slower than non slim mode, so is there any suggestions for osm2pgslq to be faster in slim?
my settings for slim was
```
 --number-processes 8 ^
 --slim ^
 --cache=60000 ^
 --flat-nodes=%flat_nodes_file% ^
```
I've got this results
```
2021-08-16 22:59:52  Reading input files done in 20996s (5h 49m 56s).
2021-08-16 22:59:52    Processed 6977700217 nodes in 2582s (43m 2s) - 2702k/s
2021-08-16 22:59:52    Processed 774861920 ways in 11685s (3h 14m 45s) - 66k/s
2021-08-16 22:59:52    Processed 8974165 relations in 6729s (1h 52m 9s) - 1k/s
```

for normal I've got this
```
2021-08-10 20:00:42  Reading input files done in 8580s (2h 23m 0s).
2021-08-10 20:00:42    Processed 6977700217 nodes in 851s (14m 11s) - 8199k/s
2021-08-10 20:00:42    Processed 774861920 ways in 6199s (1h 43m 19s) - 125k/s
2021-08-10 20:00:42    Processed 8974165 relations in 1530s (25m 30s) - 6k/s
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/1555
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20210817/be0b8540/attachment.htm>