[Tile-serving] [openstreetmap/osm2pgsql] Allow propagating tags to ways during relations processing (#230)

[Kevin B Kenny]

@lonvia Thanks for noticing this and taking the time to respond.  I hope that you will tolerate my asking more questions, because there _is_ a problem to be solved here, and we've not yet converged on a complete solution.

As you note, the proposal can indeed be generalised to arbitrary relations. I simply did not do so out of a desire to keep the database footprint as small as possible. Moreover, relations are not simply dumb containers; I suspect that different types of relation may need somewhat different structures, with part of the structure determined by the 'role' and the type of object (point, open way, closed way, other relation) contained. At the very least, I'd most likely exclude multipolygons, since they really are quite well handled by the existing mechanisms.

I agree with you in general that we want to push the processing as early as possible in the rendering sequence. With that said, though, I observe that the performance cost of the render-time join in this particular use case is almost negligible. The join winds up being executed only for numbered ways that are known to be inside the current super-tile and of enough importance to be rendered with markers at the current zoom level. The cost of doing a JOIN (which turns out to be all against cached data in many cases) is small compared with the cost of retrieving and rendering PNG or SVG files containing the graphic elements for the markers. At least it turned out to be so in a multi-hour job that prerendered all tiles down to zoom level 15 in the area that I cover on my private server (most of the US east of the MIssissippi) - adding the marker rendering added at most a few minutes to the total time.

The join at rendering time is still, I concede, not ideal, but I've not et been able to come up with a better approach.  It appears to me that the entries with negative osm_id in the 'roads' table have had their ways coalesced (using ST_LineMerge or something similar) prematurely. The result is that in a route concurrence, the renderer will simply see two unrelated lines. In a place like [this](https://kbk.is-a-geek.net/catskills/test4.html?la=42.5956&lo=-74.0045&z=14), for instance, it will have one line for the SR 443 relation, one for the SR 85 relation, and one for each individual way segment (including the changes of speed limit and the addition of climbing lanes as the road goes up the hill).

Moreover, some of the numbered ways are minor roads that do not appear in 'roads' and do not get marked as 'pending ways' in the initial pass, so don't wind up in the second pass of assembling the routes at all. Inspecting the 'line' table, I did not see entries corresponding to the relations on these minor roads.

In any case, as I said, the LineMerge is simply premature. What's actually needed for usable marker placement is that there be one multilinestring for each distinct _subset_ of the routes, so that there is a set of linestrings that's 'just SR 85', another that's 'just SR 443' and another that's 'SR 85/443 running concurrently.' For good marker placement, these strings need to be ST_LineMerge'd, of course, because otherwise, it's insanely difficult to produce markers at more-or-less regular intervals (and avoid excess ones at spots where the way is split for a reason such as a change to the speed limit), but that _must_ be deferred until a point where the process is aware of concurrencies.

Keeping concurrency sets up to date during, say, a minutely update was something where the processing was not at all obvious to me - and it appears to be beyond the capabilities of the Lua API. At least my understanding is that the the Lua script takes geometries as they are given, and that the only decisions it can make are whether or not to include the object in the table under construction and what tags to use to decorate it. If you have an example of a Lua script for osm2pgsql that actually modifies the structure of what it is given in some nontrivial way, I'd really appreciate a pointer to it! I had considered the approach, but put it on hold when my initial searches on Google appeared to turn up nothing useful. Failing that, though, building the sets in SQL at render time works quite well.

Your suggestion of joining to the 'rels' table is indeed more or less workable. The resulting PL/SQL code is rather more complicated, and this stems from the fact that the 'role' is important (including links, in particular, makes a mess of the rendering). Determining the role once you know that a given way is a member of the relation means rescanning the arrays to search for the way a second time and parsing out the 'role' element from a heterogeneous bucket of strings. It works, but it's really slow when dealing with, say, a transcontinental route and trying to pick out the roles for just those members in the tile being rendered. It also strikes me as being brittle code. When @pnorman pointed out that he considered any use of the slim tables to be unacceptable, I abandoned it.

> As for using the slim tables here, officially speaking that is indeed frowned upon. However, fact is that they are so widely used, that we can't really change the format at a whim. It might happen at some point (there is a discussion about using JSON) but is more something of a once in ten years occurrence.

Have the maintainers of **osm2pgsql** considered that the widespread use - in spite of the vehement disparagement of such use - indicates that they address a need that cannot be satisfied in any other way?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/230#issuecomment-426689789
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20181003/e8310670/attachment.html>