[Tile-serving] [openstreetmap/osm2pgsql] Supporting --append in non-slim mode (#1076)

[Sarah Hoffmann]

It should be fairly easy to get the C behaviour back:

* remove the sanity check in options.cpp that forbids append an non-slim
* return an empty list in relations_using_way()
* change osmdata.cpp to not call `slim->*_delete()` and `slim->*_changed()` when `!slim` in the  `*_modify()` functions
* make sure all your data tables have indexes on `osm_id` before appending a file

I haven't really tested this, so no guarantees that I've found all the places.

I'm undecided if I would accept such a patch in the master version for two reasons. 1) there is a good chance that people try to apply change files in non-slim mode which will go horribly wrong. 2) While I'm reasonably convinced that the changes above do the right thing for classic osm2pgsql (well, provided
the extracts happen to have the data in just the right way), I'm not sure about the second stage of the flex output. It does draw on the ability to get data out of middle and there might be crucial data missing when adding an additional extract.

That said, the patch with the changes above should be so small that you can easily maintain a patched version that follows master and works around the pesky maintainers.


I would also suggest that you look into alternatives to your current import mode that might completely avoid appending extracts. Two suggestions in particular:

* *merge extracts and import with slim + --drop (+ --disable-parallel-indexing)*  There have been quite a few changes to osm2pgsql that reduce the disk footprint in drop mode. You might find that it works for you with the most recent versions. A side effect is that you end up with geometrically sorted tables, improving rendering speed.
* *import each extract into separate tables* Then combine tables either by copying  or using Postgresql partitioning. This might do odd things to relations that  go over extract boundaries (although I'm not even sure they work correctly  with the append method either).


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/1076#issuecomment-587869083
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20200218/153606ed/attachment.htm>