[OSM-talk] live rendering

[Robert (Jamie) Munro]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Frederik Ramm wrote:
> Hi,
> 
>> The
>> problem is that it can't run on the current database, mainly because the
>> current database isn't spatially indexed. Schuler is working on what I
>> think will be the answer to all our problems - migrating the current
>> database structure to PostGIS without changing it, and then adding
>> spatial columns with triggers.
> 
> Nothing against all that. (As far as I know, the current database *is*
> spatially indexed, only not in a way suitable for Mapnik.)
> 
>> These spatial columns are not part of the logical model of the data,
>> which remains unchanged, they are just added as a sort of index or
>> cache. Mapnik should be able to work directly from these columns
> 
> I remember reading Artem complaining about lots of things that our
> current model has and which upset rendering, e.g. self-intersecting ways
> and so on. While I can easily believe that these can be fixed in a
> conversion step, I am not sure whether they can be fixed by adding
> indexes and not changing the real data.
> 
> Furthermore, I believe that spatial indexing isn't all; you will
> probably also need indexes on various metadata aspects. Rendering a
> level-7 tile "on the fly" would require transferring more than a
> gigabyte of data from the database if you only have "bounding box"
> requests. You will need to have the ability to query the database for
> exactly what you want to render - and even then, say if you only use
> motorways and cosatlines, the amount of data retrieved for processing
> will be very large.

We could use triggers to assign an importance field to features. The
importance would roughly correspond to the zoom level the feature is
visible on. We could then use partial indexes to speed pulling data from
them. Alternatively, we could use triggers to maintain low-zoom summary
features in their own database tables.

>> you won't have to wait a week for Planet.osm, or even an hour for T at H to
>> see your changes, they will be rendered instantly once you have uploaded
>> them.
> 
> Provided that a proper mechanism for invalidating tiles exists, and
> provided that both database and rendering engine really deliver the
> power to re-render a level-7 tile every time something in its area has
> changed.

It's possible that we may not be able to render level 7 tiles on the fly
- - but so what. We could render high zoom levels on the fly (say 12 and
above), and that is what users editing will want to see.

>> As a side effect, the API will run a lot quicker because it will
>> be able to use the spatial columns to speed queries.
> 
> Unless, of course, the database is bogged down by delivering data to the
> rendering viewer.

It's currently bogged down delivering data to a distributed network of
T at H clients :-)

> I would be extremely happy to see a "live renderer", don't get me wrong
> - but I am not exactly over-optimistic that we'll have it within the
> next half year.

It is a solved problem. See http://labs.metacarta.com/osm. We just have
to get updates into it as they happen, rather than once a week. PostGres
triggers (or rules) are almost certainly capable of doing this.

I'm sure improvements can be made. I'm mainly posting this stuff to
avoid duplication of effort. Schuler is working on moving the DB to
PostGIS and writing the triggers. I'm hoping to pick up his DB and port
the rails port to it at the Oxford Dev meeting on Saturday (unless he
does that first). The next stage is to integrate Schuler's DB layout
with Mapnik.

Robert (Jamie) Munro
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGOghVz+aYVHdncI0RAr9aAKDlU9/tcfyBy/78iItunv5izUrgGwCg31WC
6FOvhLD7S9hj331sw/+cc+M=
=0Ow7
-----END PGP SIGNATURE-----