<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
</head>
<body text="#000000" bgcolor="#FFFFFF">
On 2018-11-08 6:34 AM, Nick Whitelegg wrote:<br>
<blockquote type="cite"
cite="mid:AM4PR07MB3425F84CD3E570715D580624A6C50@AM4PR07MB3425.eurprd07.prod.outlook.com">
<p style="margin-top:0;margin-bottom:0">At the moment I download
full planet extracts about every 6 months. However, due to the
limitations of my server, I filter out (with osmosis) a lot of
stuff I don't need so that I am basically left with roads,
footpaths, natural features, water features and selected POIs.</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0">I'd like to move towards a
system which applies diffs from geofabrik instead, and applies
them regularly (daily or weekly) with osm2pgsql.</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0">My question is this; given
that not everything in the diff will be in my database (as I
filter out what I don't need during the import process), will
osm2pgsql apply the diff successfully or will it complain that
not all features in the diff are in my database?</p>
</blockquote>
<br>
I can think of four ways to do this, all which have a different
balance of correctness, performance, and ease of use.<br>
<br>
There are two "right" ways to do this. The first one is to re-import
every week. Because imports without slim tables (either --slim
--drop or no --slim) are faster, this is a good option and needs
less space than a database able to consume diffs.<br>
<br>
The second right way involves keeping two files, one with the
current full data, and one with the filtered data. Call these
"planet.pbf" and "planet-filtered.pbf". Then when updating create
"planet-new.pbf", filter it to get "planet-filtered-new.pbf", create
a diff for the differences between "planet-filtered-new.pbf" and
"planet-filtered.pbf", and apply that diff to the database. Then
replace the old files with the new ones. This will keep the database
correct.<br>
<br>
A "wrong" way to do it is to import the filtered data, apply updates
directly, and periodically delete data from the DB. The problem with
this is that if someone adds one of the selected POI tags to a
building that you have filtered out, osm2pgsql won't have the node
data to create a geometry. This might be acceptable, depending on
use case.<br>
<br>
A less wrong way would be to modify your filtering so no nodes are
filtered out. There are still potential errors with relations, but
these are less common. If you're doing the planet or a large extract
and using flat nodes there's no storage penalty for having all the
nodes.<br>
</body>
</html>