[Tile-serving] [osm2pgsql-dev/osm2pgsql] Add grouped-linemerge generalization (PR #2482)

[leijurv]

leijurv left a comment (osm2pgsql-dev/osm2pgsql#2482)

Okay, I will try to figure out how to get a table of ways that changed 👍 

For the case of a new way that connects existing ways, yes that's the rub. Your case can already happen currently, when a way is itself larger than the tile. It's actually even worse than that, because ST_LineMerge will **stop** merging when the degree of a node goes from 2 to 3, so adding a way can actually **split** a road geometry. There were broadly two ways to do this: 1. write extremely complicated recursive CTE queries to try to fix up exactly the "edit" and no further (in your case, it would just connect those two geometries without recursively exploring them on the input table side) or 2. explore the entire geometry, delete the entire geometry, re-ST_LineMerge it. I picked the latter in this PR.

I actually think multithreading would help specifically because of how the recursive CTE requires many serial lookups, that is I/O bound but there is a difference between being bound by I/O latency for dependent/sequential random lookup versus being bound by I/O throughput (whether the lookups are sequential or random). So if you were to have several tiles that are all expired, you would probably get a speedup with each thread expiring one of them, versus one thread expiring all sequentially (note: this PR doesn't quite do that, it uses joins to get postgres to do many tiles "at once" internally). But my point on multithreading was not actually about two threads that are both doing expiring, rather about render threads not being blocked by expiring threads, because the expiring threads (and the render threads) are doing dependent lookups through the GiST structure with inherent latency in the processor-side logic from one read to the next.

In any event I will think about this more https://github.com/leijurv/openstreetmap-carto/commit/b79c0fa3033ed5339643eb1b9590edc7e684dd28 This is how I had it, but I think I'll need to use a totally different method (not tile merging at all). My initial hope is that whenever a way is added, updated, or deleted, I can simply add the exact (x,y) of both endpoints to a table of geometries to be updated, then the generalization can re-explore the geometry from there? That would be the happy / easy path, but I am probably missing something.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/osm2pgsql-dev/osm2pgsql/pull/2482#issuecomment-4615223664
You are receiving this because you are subscribed to this thread.

Message ID: <osm2pgsql-dev/osm2pgsql/pull/2482/c4615223664 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20260603/3447d0f9/attachment.htm>