[Tile-serving] [openstreetmap/osm2pgsql] Fix multistage processing (#1208)

[Jochen Topf]

The multi-stage processing in the flex output has some problems when updating data, for instance when relations are deleted after their member ways use their tags.

This commit reorganizes how the multi-stage processing is done to address these problems.

1. The way "marking" of objects is done is now different. Instead of doing this in the process_* Lua functions a new Lua function "check_relation()" is introduced. This will be called not only for all new relations but also for deleted relations, or when a relation changed, for the old relation. This is needed so that we also mark and then re-create way entries in the database that used to depend on a parent relations tags, but don't do that any more.
2. We remove the Lua mark() function and replace it by mark_way(). It was never possible to mark nodes anyway. And there is no use case I am aware of currently that needs marking relations. Both can be reintroduced later when we have a better idea how to handle them. For the time being we concentrate on the important use case where member ways of relations are handled specially.
3. This introduces a new processing stage 1c:
    * stage 1a: Read input file and process all objects in it.
    * stage 1b: Process dependent objects of objects from 1a (ie
      changed nodes trigger changes in ways with those nodes, changes
      in all objects potentially trigger changes in parent relations).
    * stage 1c: Process dependent objects (currently only relations)
      of objects marked during stage 1a/1b (this one is new).
    * stage 2: Process objects marked in stage 1a or 1b.
4. Add more checks for which C++ functions can be called from which Lua functions.
5. New Lua helper function osm2pgsql.mark_member_ways() that iterates over way members of a relation and marks them all. This is more convenient than doing it "mannually". If you need more complex processing, for instance use the member roles to decide which ways need stage 2 processing, you can still write your own loop.
You can view, comment on, or merge this pull request online at:

  https://github.com/openstreetmap/osm2pgsql/pull/1208

-- Commit Summary --

  * Keep track of which Lua function was called
  * Reorganize multi-stage processing fixing some update problems

-- File Changes --

    M docs/flex.md (11)
    M docs/lua-lib.md (17)
    M flex-config/route-relations.lua (58)
    M src/init.lua (12)
    M src/middle-pgsql.cpp (2)
    M src/osmdata.cpp (43)
    M src/osmdata.hpp (4)
    M src/output-flex.cpp (166)
    M src/output-flex.hpp (40)
    M src/output.hpp (6)
    M tests/data/test_output_flex_extra.lua (40)
    M tests/test-output-flex-extra.cpp (131)

-- Patch Links --

https://github.com/openstreetmap/osm2pgsql/pull/1208.patch
https://github.com/openstreetmap/osm2pgsql/pull/1208.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/pull/1208
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20200610/303c07df/attachment.htm>