[Tile-serving] [openstreetmap/osm2pgsql] Use osmium types for all inclomming data (#650)

[Sarah Hoffmann]

Forward the data coming from libosmium directly into the processing pipeline instead of copying them into custom types. To make this work, middle also has to create libosmium types when fetching objects from its cache. osm2pgsql types are still used whenever data is manipulated, most notably after tagtransform and for the results of geometry projection.

This work is mostly a preparation for using more libosmium functions in osm2pgsql. Performance is about the same as before.

For details about the change, see the individual commits. Up to 'move ways_get* functions to libosmium types' is the step-by-step conversion of the code, after that follow bug fixes.
You can view, comment on, or merge this pull request online at:

  https://github.com/openstreetmap/osm2pgsql/pull/650

-- Commit Summary --

  * replace OsmType with osmium::item_type
  * move conversion from libosmium types into type constructors
  * make osmdata work directly with libosmium data types
  * simplify writing of relation middle table
  * use libosmium types in middle layer
  * forward libosmium types into output modules
  * fix warnings about unused parameters
  * move gazetter output to libosmium types
  * convert tag filter functions to libosmium types
  * use node lists from libosmium directly in outputs
  * move middle::relations_get() to libosmium types
  * use libosmium version of output-pgsql pgsql_process_relation
  * use libosmium version of output-multi process_relation
  * remove now unused tagtransform::filter_rel_tags
  * move ways_get* functions to libosmium types
  * adapt to latest changes in libosmium
  * take into account duplicate ways when getting relation ways
  * avoid size() for tags and members
  * do not handle extra attributes via parameter passthough
  * add global buffer for relations
  * fix reverted emtpy check

-- File Changes --

    M geometry-processor.cpp (84)
    M geometry-processor.hpp (18)
    M middle-pgsql.cpp (344)
    M middle-pgsql.hpp (24)
    M middle-ram.cpp (65)
    M middle-ram.hpp (31)
    M middle.hpp (36)
    M node-persistent-cache.cpp (8)
    M node-persistent-cache.hpp (2)
    M node-ram-cache.cpp (2)
    M node-ram-cache.hpp (2)
    M osm2pgsql.cpp (6)
    M osmdata.cpp (74)
    M osmdata.hpp (22)
    M osmtypes.hpp (104)
    M output-gazetteer.cpp (550)
    M output-gazetteer.hpp (91)
    M output-multi.cpp (149)
    M output-multi.hpp (23)
    M output-null.cpp (26)
    M output-null.hpp (12)
    M output-pgsql.cpp (206)
    M output-pgsql.hpp (19)
    M output.cpp (4)
    M output.hpp (12)
    M parse-osmium.cpp (68)
    M parse-osmium.hpp (18)
    M taginfo.cpp (29)
    M taginfo_impl.hpp (13)
    M tagtransform.cpp (183)
    M tagtransform.hpp (19)
    M tests/common-pg.cpp (1)
    M tests/common.hpp (3)
    M tests/middle-tests.cpp (408)
    M tests/mockups.hpp (70)
    M tests/test-hstore-match-only.cpp (2)
    M tests/test-options-projection.cpp (2)
    M tests/test-output-multi-line-storage.cpp (2)
    M tests/test-output-multi-line.cpp (4)
    M tests/test-output-multi-point-multi-table.cpp (4)
    M tests/test-output-multi-point.cpp (4)
    M tests/test-output-multi-poly-trivial.cpp (2)
    M tests/test-output-multi-polygon.cpp (4)
    M tests/test-output-multi-tags.cpp (2)
    M tests/test-output-pgsql-area.cpp (2)
    M tests/test-output-pgsql-schema.cpp (2)
    M tests/test-output-pgsql-tablespace.cpp (2)
    M tests/test-output-pgsql-z_order.cpp (2)
    M tests/test-output-pgsql.cpp (10)
    M tests/test-parse-diff.cpp (22)
    M tests/test-parse-xml2.cpp (60)

-- Patch Links --

https://github.com/openstreetmap/osm2pgsql/pull/650.patch
https://github.com/openstreetmap/osm2pgsql/pull/650.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/pull/650
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20161119/7bd31419/attachment-0001.html>