[Tile-serving] [openstreetmap/osm2pgsql] Duplicate rows with --expire-tiles option (#766)

[Michael Reichert]

I cannot confirm this issue with the following setup (only a small extract):

* Download extract of Lower Saxony from Geofabrik of 2017-07-17 ([link](http://download.geofabrik.de/europe/germany/niedersachsen-170717.osm.pbf))
* Import it using `./osm2pgsql --database expirytest --slim --style ../default.style --number-processes 8 --hstore-all --cache 2000 --merc niedersachsen-170717.osm.pbf`
* Check if following objects exist: node 48644902, node 115966540, way 4473628, way 23741521, relation 84869
* Apply the diff of the extract [000001578](http://download.geofabrik.de/europe/germany/niedersachsen-updates/000/001/578.osc.gz) using `osm2pgsql --database expirytest --slim --append --expire-tiles 9-19 --expire-output extest.txt --style ../default.style --number-processes 8 --hstore-all --cache 2000 --merc 578.osc.gz`
* Look for the objects again. Node 48644902 and way 23741521 should be gone, way 4473628 should now have more than two nodes. Relation 84869 should now have three tags.

osm2pgsql works as expected.

ravhed wrote:
> I've seen this behavior as well and having the expire-tiles parameter set actually affect if rows in the database are deleted or not. See this method which deletes ways:
>
> ```cpp
> int output_pgsql_t::pgsql_delete_way_from_output(osmid_t osm_id)
> {
>     /* Optimisation: we only need this is slim mode */
>     if( !m_options.slim )
>         return 0;
>     /* in droptemp mode we don't have indices and this takes ages. */
>     if (m_options.droptemp)
>         return 0;
> 
>     m_tables[t_roads]->delete_row(osm_id);
>     if ( expire.from_db(m_tables[t_line].get(), osm_id) != 0)
>         m_tables[t_line]->delete_row(osm_id);
>     if ( expire.from_db(m_tables[t_poly].get(), osm_id) != 0)
>         m_tables[t_poly]->delete_row(osm_id);
>     return 0;
> }
> ```
> 
> As you can see it will only delete the way if it is expired. When expire-tiles is not set it will always be deleted as from_db will return -1 in that case.

I do not understand what's wrong here.

`expire_tiles::from_db` returns `table_t::wkb_reader::get_count()` which returns  `table_t::wkb_reader::m_count`. `table_t::wkb_reader::m_count` is set in the constructor of `table_t::wkb_reader`. It is the number of rows returned by the database after the execution of the prepared statement `get_wkb`. If the object to be deleted does not exist in the table, `table_t::wkb_reader::get_count()` will return 0. This prevents the unnecessary execution of `DELETE FROM %1% WHERE osm_id = %2%`.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/766#issuecomment-316442540
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20170719/08ccf832/attachment.html>