[OSM-dev] [Mapnik-users] How does Mapnik know which nodes are "important"?

[Jon Burgess]

On 27/12/2007, Kelly Jones <kelly.terry.jones at gmail.com> wrote:
> Martijn,
>
> That's great for "zoom level 17" tiles where you always want to
> display everything in the bounding box.

Not necessarily. We might want to display some things such as Country
names at low zooms only.

> For "zoom level 11" (for example), there are probably millions of
> nodes in the tile, and you can't map them all (most are too
> unimportant to be mapped).

I'll restrict my comments to Mapnik and how it uses the
PostgreSQL/Postgis DB queries. This is a problem but we have a partial
solution already. We do not store the plain nodes in the Posgres DB
but instead geometries created by combining the nodes, way and tag
data in the Postgres DB.

In order to reduce the amount of data we query from the DB for the
lower zoom tiles the data is split so that we can query the smaller
planet_osm_roads table instead of the larger planet_osm_line table.

> By using QuadTiles, you only have to look at 120,000 nodes (10,000
> nodes each for the QuadTile and all of its parents) to determine what to
> map. Of course, you could use this in conjunction with spatial
> indexing to make things even faster.

We do not directly deal with this level of detail in Mapnik. The
PostgreSQL Postgis extensions deal with efficiently performing spatial
queries on the data in the tables. It uses an radix-index scheme which
I believe is a more generalised form of quad-tiling.

> This scales well: the number of nodes you look at to render a "zoom
> level 11" tile remains constant at 120,000 (or less if the tile and
> its parents are fairly blank).
>
> Note: at Andy Allan's suggestion, I've moving this thread from
> 'newbies' to 'dev' (dev folks, please see the newbies archive for the
> first part of this thread). This message is cc'd to both lists, but
> future messages will go just to the 'dev' list. Sorry about the
> confusion.

I had a quick look at the other threads. Things are really fairly
complex. The data is filtered through multiple levels:-
1) osm2pgsql converts the planet dump into 4 Postgres tables (points,
lines, areas, roads). We never do node lookups as such while
rendering, all the node co-ordintes are recorded in the geometries in
the features.
2) Mapnik uses the zoom level information to select a subset of data
to query based on the osm.xml style. We keep the motorway and primary
roads in a small table to avoid needing to query all the minor roads
when rendering lower zooms.
3) Mapnik performs spatial queries on the data sources to get the
possible data to render. This allows us to efficient locate geometries
covering a given area. Postgis is intelligent enough to know if we are
completely inside an area even if all the points are outside of the
current tile.
4) Mapnik applies the data filter rules to determine exactly which
rendering style to apply to the data.

As the data set grows we may need to split the data further (perhaps
by introducing more table to cover a finer range of zooms or feature
types). Also we could do with reducing the detail of features at lower
zooms (we can simplify the data so we don't need to render thousands
of nodes for a motorway when it is only a few pixels long on the final
image). I've had some initial discussions with Artem about this since
it may need some design changes in Mapnik and the style definitions.
Nothing concrete has come from these discussions at the moment.

If you've got some ideas and the necessary programming experience then
I'd suggest trying out some things and let us know how you get on.

-- 
    Jon