[Tile-serving] [openstreetmap/osm2pgsql] Split pgsql middle into query and insertion class (#899)

[Sarah Hoffmann]

This PR changes middle-pgsql to have separate implementations for middle_query_t and middle_slim_t. The main advantage of this setup is that the query class is much more lightweight and opens only one connection to the database. This means that we save 2*num_thread connections in the processing phase (improves on #885).

Given that output now also receive a middle_pgsql_query instance, some structural changes were necessary:
* Tables need to be created before the query instance is created because of prepared queries. They are set up during construction of the query instance and that requires that the table the prepared query references has already been created. Therefore middle::start() has been moved to osm2pgsql.cpp and is now done before output are set up.
* As the data is queried on a different connection, the inserted data needs to be committed in time. There is now an additional commit when switching OSM types during the import phase using the flush() function introduced a couple of PRs ago.

Only small adaptions of the tests this time. In particular adding missing flush() where appropriate.
You can view, comment on, or merge this pull request online at:

  https://github.com/openstreetmap/osm2pgsql/pull/899

-- Commit Summary --

  * remove unused defines and includes
  * split middle-pgsql into insert and query class

-- File Changes --

    M middle-pgsql.cpp (187)
    M middle-pgsql.hpp (52)
    M middle-ram.cpp (12)
    M middle-ram.hpp (4)
    M middle.hpp (4)
    M osm2pgsql.cpp (15)
    M osmdata.cpp (1)
    M tests/common.hpp (7)
    M tests/middle-tests.hpp (28)
    M tests/mockups.hpp (8)
    M tests/test-middle-flat.cpp (10)
    M tests/test-middle-pgsql.cpp (12)
    M tests/test-options-parse.cpp (3)
    M tests/test-output-multi-line.cpp (10)
    M tests/test-output-multi-point-multi-table.cpp (11)
    M tests/test-output-multi-point.cpp (11)
    M tests/test-output-multi-polygon.cpp (11)
    M tests/test-output-pgsql.cpp (10)

-- Patch Links --

https://github.com/openstreetmap/osm2pgsql/pull/899.patch
https://github.com/openstreetmap/osm2pgsql/pull/899.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/pull/899
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20190115/dd3a4e51/attachment-0001.html>