[Tile-serving] [osm2pgsql-dev/osm2pgsql] Bug: Data corruption (?) on COPY (Issue #2297)

[Seth Miller]

## What version of osm2pgsql are you using?

Dockerized, based on Debian:

```
osm2pgsql version 2.0.1
Build: RelWithDebInfo
Compiled using the following library versions:
Libosmium 2.20.0
Proj 9.1.1
Lua 5.3.6
```

## What operating system and PostgreSQL/PostGIS version are you using?

Debian 12.8 container, running on Debian 12 host, Kernel: 6.9.10


## Tell us something about your system

Processor: AMD Ryzen 9 7950X3D 16-Core Processor
Memory: 96GB 
Disk Usage: <100%, limit not hit


## What did you do exactly?

Importing a global PBF downloaded from AWS, using Themepark

docker-compose snippet:
```
    command:
      - --create
      - --output=flex
      - --hstore
      - --slim
      - --cache
      - "4096"
      - -S
      - /config/style-power-only.lua
      - -d
      - 'postgresql://${POSTGRES_MAPS_USER}:${POSTGRES_MAPS_PASSWORD}@db/${POSTGRES_MAPS_DB}'
      - /osm.pbf
```

## What did you expect to happen?

Data file should import without error.

## What did happen instead?

I received this error:

```
Storing properties to table '"public"."osm2pgsql_properties"'.
Processing: Node(9561040k 3336.0k/s) Way(0k 0.00k/s) Relation(0 0.0/s)
ERROR: DB copy thread failed: Ending COPY mode for 'planet_osm_nodes' failed: ERROR:  invalid input syntax for type integer: "-1224516p87"
CONTEXT:  COPY planet_osm_nodes, line 1961329060, column lon: "-1224516p87"
.
```

The kicker though, I can run that again and receive a different error each time. I don't know what's going on, looks like some concurrency or corruption issue on copy? Data seems to get mangled together. A different one was upset about an invalid hash map/json with a random `]` character inside. 


## What did you do to try analyzing the problem?

I'm a bit at a loss because the logs aren't super detailed here. This is after a bit of runninng.

1. Run smaller files - this seems to work fine. I can import a US state, the full US, and those work fine. It's just when scaling up to global it fails.
2. Running with THEMEPARK_DEBUG off - [IN PROGRESS]


Is this an `osm2pgsql` error message or postgres? This is an odd one...



-- 
Reply to this email directly or view it on GitHub:
https://github.com/osm2pgsql-dev/osm2pgsql/issues/2297
You are receiving this because you are subscribed to this thread.

Message ID: <osm2pgsql-dev/osm2pgsql/issues/2297 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20250114/cf8e2875/attachment.htm>