[Tile-serving] [openstreetmap/osm2pgsql] Dropped ways (#735)

[Paul Norman]

This has a similar effect to #722 but completely different causes

==Background==
When updating a planet database with diffs I was observing some ways disappearing from the output table, and isolated a case of it with way 127018391 in diff 2391120.

My update script does ``osmosis --read-replication-interval --write-xml-change file="-" | osmium merge-changes --simplify -F osc --fsync -o "${file}"`` and I pulled the file, changes-2391119.osc.gz. This diff reproduced the problem, but manually reproducing the changes to the way from that file didn't.

The following XML reproduces the problem when processed with ``osm2pgsql -E 3857 --flat-nodes /srv/planet/planet/nodes.bin --cache 200 --number-processes 1 --append --slim -d ct --output multi --style cleartables.json -G -r osc changes-2391119-reduced.osc ; psql -d ct -c 'select osm_id,name from buildings where osm_id in (127018391);'``

```xml
<?xml version='1.0' encoding='UTF-8'?>
<osmChange version="0.6" generator="osmium/1.5.1">
  <delete>
    <node id="1406709647" version="2" timestamp="2017-04-07T01:00:21Z" uid="355617" user="pnorman" changeset="47524627" lat="49.2003542" lon="-122.9898656"/>
  </delete>
  <modify>
    <node id="1406709654" version="2" timestamp="2017-04-07T01:00:20Z" uid="355617" user="pnorman" changeset="47524627" lat="49.2003208" lon="-122.9905305"/>
    <node id="1406709657" version="2" timestamp="2017-04-07T01:00:20Z" uid="355617" user="pnorman" changeset="47524627" lat="49.2004211" lon="-122.9906811"/>
  </modify>
  <delete>
    <node id="1406709667" version="2" timestamp="2017-04-07T01:00:21Z" uid="355617" user="pnorman" changeset="47524627" lat="49.2007643" lon="-122.9905898"/>
  </delete>
  <modify>
    <node id="1406709669" version="2" timestamp="2017-04-07T01:00:21Z" uid="355617" user="pnorman" changeset="47524627" lat="49.2008595" lon="-122.9899974"/>
    <node id="1406709672" version="2" timestamp="2017-04-07T01:00:21Z" uid="355617" user="pnorman" changeset="47524627" lat="49.2007592" lon="-122.9898468"/>
  </modify>
  <modify>
    <node id="3183336278" version="2" timestamp="2017-04-07T01:00:51Z" uid="4023025" user="alexandreos" changeset="47524632" lat="-29.3581502" lon="-50.811"/>
    <node id="3183336282" version="2" timestamp="2017-04-07T01:00:51Z" uid="4023025" user="alexandreos" changeset="47524632" lat="-29.3582158" lon="-50.8110269"/>
  </modify>
  <create>
    <node id="4780166721" version="1" timestamp="2017-04-07T01:00:20Z" uid="355617" user="pnorman" changeset="47524627" lat="49.2005388" lon="-122.9900862"/>
    <node id="4780166722" version="1" timestamp="2017-04-07T01:00:20Z" uid="355617" user="pnorman" changeset="47524627" lat="49.200363" lon="-122.9900892"/>
    <node id="4780166723" version="1" timestamp="2017-04-07T01:00:20Z" uid="355617" user="pnorman" changeset="47524627" lat="49.2004502" lon="-122.9899531"/>
    <node id="4780166724" version="1" timestamp="2017-04-07T01:00:20Z" uid="355617" user="pnorman" changeset="47524627" lat="49.2004516" lon="-122.9902223"/>
    <node id="4780166725" version="1" timestamp="2017-04-07T01:00:20Z" uid="355617" user="pnorman" changeset="47524627" lat="49.2003401" lon="-122.9903962"/>
    <node id="4780166726" version="1" timestamp="2017-04-07T01:00:20Z" uid="355617" user="pnorman" changeset="47524627" lat="49.2005285" lon="-122.9901023"/>
    <node id="4780166727" version="1" timestamp="2017-04-07T01:00:20Z" uid="355617" user="pnorman" changeset="47524627" lat="49.2003742" lon="-122.9904473"/>
    <node id="4780166728" version="1" timestamp="2017-04-07T01:00:20Z" uid="355617" user="pnorman" changeset="47524627" lat="49.2005626" lon="-122.9901535"/>
    <node id="4780166729" version="1" timestamp="2017-04-07T01:00:20Z" uid="355617" user="pnorman" changeset="47524627" lat="49.2008287" lon="-122.9902478"/>
    <node id="4780166730" version="1" timestamp="2017-04-07T01:00:20Z" uid="355617" user="pnorman" changeset="47524627" lat="49.2006186" lon="-122.9905754"/>
    <node id="4780166731" version="1" timestamp="2017-04-07T01:00:20Z" uid="355617" user="pnorman" changeset="47524627" lat="49.2005525" lon="-122.9904762"/>
    <node id="4780166732" version="1" timestamp="2017-04-07T01:00:20Z" uid="355617" user="pnorman" changeset="47524627" lat="49.2007626" lon="-122.9901486"/>
  </create>
  <modify>
    <way id="127018391" version="2" timestamp="2017-04-07T01:00:21Z" uid="355617" user="pnorman" changeset="47524627">
      <nd ref="1406709669"/>
      <nd ref="1406709672"/>
      <nd ref="4780166728"/>
      <nd ref="4780166726"/>
      <nd ref="4780166721"/>
      <nd ref="4780166723"/>
      <nd ref="4780166722"/>
      <nd ref="4780166724"/>
      <nd ref="4780166725"/>
      <nd ref="4780166727"/>
      <nd ref="1406709654"/>
      <nd ref="1406709657"/>
      <nd ref="4780166731"/>
      <nd ref="4780166730"/>
      <nd ref="4780166729"/>
      <nd ref="4780166732"/>
      <nd ref="1406709669"/>
      <tag k="amenity" v="restaurant"/>
      <tag k="building" v="yes"/>
      <tag k="name" v="Riverway Clubhouse"/>
    </way>
  </modify>
</osmChange>
```

The interesting bit is the node changes from alexandreos. With that XML as-is, I get zero rows in the query. If I remove the entire <modify> block or the first node I get the building in the rendering tables.

This is quite frankly, befuddling. Nothing is obviously different between the two alexandreos node changes except for id, latitude, and longitude.

c.f. https://phabricator.wikimedia.org/T160781

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/735
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20170406/1af55f69/attachment-0001.html>