[Tile-serving] [openstreetmap/osm2pgsql] Ability to have pre-processing script for osm2pgsql-replication (Issue #1729)

[Branko Kokanovic]

## What version of osm2pgsql are you using?

```
osm2pgsql version 1.6.0
Build: None
Compiled using the following library versions:
Libosmium 2.18.0
Proj [API 6] 9.0.1
Lua 5.3.6
```

## What operating system and PostgreSQL/PostGIS version are you using?

Debian testing

## What did you do exactly?

Scenario: I want to be able to trim .osc file inside osm2pgsql-replication **after** it is collected from replication server, but **before** it is send to `osm2pgsql` for processing. One use case is calling [trim_osc](https://github.com/Zverik/regional/blob/master/trim_osc.py), another could be doing diff of .osc and postgis (as this is only time to do proper diff with current data, before .osc data enters database).

As far as I know, there is no way to stop processing between these two events (getting .osc from repl server and sending to osm2pgsql).

I was thinking it could follow same pattern as post-processing logic. Another `--pre-processing` switch and even same arguments (seq and timestamp). As discussed at #1719 , if one needs to get path to .osc, one can use provide `--diff-file` argument to `osm2pgsql-replication` script.

Only question (besides do you want to support this) is do we want to .osc file to be edited in-place, or we want some more sophisticated algorithm (for example - pre-processing script do not edit file in-place, but save it somewhere differently and return output path in stdout, or just stream new .osc to stdout, and we capture it...). IMHO, easiest would be in-place editing, but I am open for suggestion.

I am also volunteering to implement this logic (if we agree it can be useful).


-- 
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/1729
You are receiving this because you are subscribed to this thread.

Message ID: <openstreetmap/osm2pgsql/issues/1729 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20220801/e8e0f071/attachment.htm>