[Tile-serving] [osm2pgsql] Adding new tags to stylesheets (#88)

[Paul Norman]

*In some ways this is an issue with the stylesheets, but osm2pgsql is the common component and it'd be nice to solve this once.*

Now that the stylesheets are being actively developed again, we're going to reach a point where we need to render something based on a new tag. There have been three [openstreetmap-carto](https://github.com/gravitystorm/openstreetmap-carto) requests for new features with tags not in [default.style](../blob/master/default.style) (gravitystorm/openstreetmap-carto#111 gravitystorm/openstreetmap-carto#180 gravitystorm/openstreetmap-carto#81) and although none of those are certain to get in, it's a matter of time before we want to add a feature which relies on tags not in default.style. [HDM-CartoCSS](https://github.com/hotosm/HDM-CartoCSS) is already using default.style + hstore.

I believe it is desirable to use a common .style file among multiple stylesheets if at all possible.

I see four options for adding new tags. All of them would involve a reload of orm/yevaud because the required information simply isn't in their databases.

1. Add new columns to default.style, requiring a database reload to get all the newly desired data in each time a tag is added. This has been the traditional way to add tags and is *probably* still possible for tile.osm.org with both orm and yevaud running although it would take care to avoid overloading them. For most servers it would involve downtime.

  It would also mean you need to be very careful about osm2pgsql/.style versions to make sure that you've got all the tag columns.
  
  With the database reloads on every tag addition and the chance of version hell, this is my least preferred option.
  
2. Use the existing default.style with `--hstore`. This avoids the database reloads, but leaves us with an incoherent .style file. For example, default.style has `generator:source` which by any logic should be in hstore.

  This would work, but we'd be missing an opportunity to rationalize some of the legacy cruft in default.style.
  
3. Use [empty.style](../blob/master/empty.style) and a tags hstore column exclusively for OSM tags. This would be combined with suitable indexes, either on the entire tags column (`GIN (tags)`), partial indexes like `GIST (way) WHERE tags ? 'amenity' OR tags ? 'shop'`, or a composite index like `GIST (way, tags)`.

  This would require a reload and a rewrite of existing queries for efficiency. It would be possible to create a view that allows an existing stylesheet to be used without changes but this would end up running queries like `WHERE (tags->'building') IS NOT NULL` instead of `WHERE NOT tags ? 'building'`.

4. Create a new style file with significant keys as columns and the remainder in hstore. This would put tags like `natural`, `waterway`, `amenity`, `name`, etc as columns and have the rest in hstore. As with right now, partial indexes would be necessary for performance. I expect all medium to medium-high queries would use have one of the tag columns but there are certain to be some that would not. Because of the differences between stylesheets having every tag used in a WHERE condition is a moving target.

I considered options involving `--hstore-match-only` but this would fail in the case of `area:highway` (gravitystorm/openstreetmap-carto#180) and `abandoned:building` (hotosm/HDM-CartoCSS#126).

At this point I think what is necessary is benchmarking of options 3 and 4 to see what kind of performance considerations there are.

cc @gravitystorm, @yohanboniface, @jburgess777

---
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/88
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20130927/fa4ce2bb/attachment.html>