[Tile-serving] [osm2pgsql-dev/osm2pgsql] Allow a limit to be set on expiry of line segments (PR #2185)

[Jochen Topf]

The PR is clean and technically looks okay to me. But I am concerned with a few things:

1. We are moving away from configuration by command line options. I don't think it is a good idea to create more of them. In this case we would need it as long as we are using the pgsql output.
2. The change seems to be rather specific to a very specific problem. There are many other cases possible which would ceate the same kind of damage and which are not caught by this code. Are we going to solve this by adding ever more tuning options for, say, the overall length of a line (as opposed to the segment length which we have here), or similar settings for polygons and so on?
3. The value this option is set to looks totally "magic" to me. There is no clear way how to decide what this number should be. If we can't explain to a user why they should set an option to what value, that's problematic from a usability standpoint.
4. If we had had this change in place for the recent vandalism it might have prevented some tiles showing the vandalism from rendering, but some tiles would have been re-rendered for other reasons so the vandalism would still show up. But than this very change prevents the tile from being re-rendered after a fix is applied. (Because expire is not only calculated based on the new geometry of an object but also the old geometry.)

I think we should clearly think about the two issues raised here, one is about protecting "the map", the other is about protecting "the system". osm2pgsql (potentially) running out of memory is bad and needs addressing. And if an excessive number of tiles in the expire list creates some situation where other parts of the system are likely going to fail, than that needs addressing, too. I'd rather see those issues solved in a different way though that's more directly tied to the problem, i.e. the excessive number of tiles, not to just one case which this PR addresses. And I am not sure whether osm2pgsql is the right place to solve system stability issues (or if there is even an issue), but if that is the case we should talk about that. To solve this problem we probably need some limits in the number of tiles that can show up in the expire list or so. But the goal of that would not be to protect "the map", but to make the system more robust. Protecting "the map" from changes in the data that we don't want to see rendered is a different issue we should not conflate with the first.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/osm2pgsql-dev/osm2pgsql/pull/2185#issuecomment-2127043996
You are receiving this because you are subscribed to this thread.

Message ID: <osm2pgsql-dev/osm2pgsql/pull/2185/c2127043996 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20240523/6b90a470/attachment.htm>