[Tile-serving] [openstreetmap/osm2pgsql] WIP: remove support for old-style MPs (#862)

Sarah Hoffmann notifications at github.com
Fri Aug 10 21:08:01 UTC 2018


Also removes support for superseding ways from polygons. The lua bindings still expect that the supersede table is present but will simply drop it.

Still todo:
 - [ ] recheck that really all old-style MP code is gone
 - [ ] adapt example Lua style
 - [ ] fix regression tests 
  
I've done some performance tests as this PR moves all way processing to the mono-threaded first computation stage. Test were done on a Hetzner EX41-SSD (i7-7700, 32GB RAM, SSD in Raid0) with command:

    ./osm2pgsql -d gis --drop -C 10 -s -S ../default.style --flat-nodes gis.flatnodes planet-180625.osm.pbf

The minimal cache is a bit of a worst case scenario as all node lookups go to the slightly slower flatnode file.

master:
```
Node stats: total(4581403065), max(5716309124) in 425s
Way stats: total(504094523), max(601431841) in 49995s
Relation stats: total(5899892), max(8406934) in 26409s

Going over pending ways...
        357968593 ways are pending

Using 8 helper-processes
Finished processing 357968593 ways in 28948 s

Osm2pgsql took 125689s overall
```

This PR:
```
Node stats: total(4581403065), max(5716309124) in 471s
Way stats: total(504094523), max(601431841) in 71760s
Relation stats: total(5899892), max(8406934) in 25839s

Going over pending ways...
        0 ways are pending

Using 8 helper-processes
Finished processing 0 ways in 0 s

Osm2pgsql took 116951s overall
```

So this looks fairly promising and it would be ok to go ahead with dropping old-style MPs without improving the first-stage processing first.
You can view, comment on, or merge this pull request online at:

  https://github.com/openstreetmap/osm2pgsql/pull/862

-- Commit Summary --

  * remove support for old-style MPs

-- File Changes --

    M geometry-processor.cpp (3)
    M geometry-processor.hpp (1)
    M output-multi.cpp (25)
    M output-pgsql.cpp (31)
    M tagtransform-c.cpp (35)
    M tagtransform-c.hpp (6)
    M tagtransform-lua.cpp (17)
    M tagtransform-lua.hpp (6)
    M tagtransform.hpp (1)
    M tests/test-output-pgsql-tablespace.cpp (2)
    M tests/test-output-pgsql-validgeom.cpp (2)
    M tests/test-output-pgsql.cpp (6)

-- Patch Links --

https://github.com/openstreetmap/osm2pgsql/pull/862.patch
https://github.com/openstreetmap/osm2pgsql/pull/862.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/pull/862
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20180810/6a5547c1/attachment.html>


More information about the Tile-serving mailing list