[OSM-talk] live rendering

[Robert (Jamie) Munro]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Frederik Ramm wrote:
> Hi,
> 
>> We could use triggers to assign an importance field to features. 
> 
> The more you use triggers, the more you slow down a simple update with
> tasks that are in fact the first step of a rendering process. I still
> maintain it would be a much more solid alternative to invest work in a
> way of keeping a live copy of the data, where you can run triggers to
> your heart's content without delaying changes or insertions of new data
> to the central repository.
> 
> But this is an opinion built from theory, and from having seen
> considerable harm done by triggers in the wrong hands ;-) - I cannot say
> how complicated the required operations will be, and maybe Postgres even
> has a way to execute triggers asynchronously without delaying the
> operation that fired the trigger.

Another thought of mine was to have the triggers just delete (set to
null) the summary data, then have a daemon come and refill it in another
thread, perhaps every minute or two. This would be equivalent to your
asynchronous trigger execution idea. I posted a python script the other
day (osm-mysql2geom.py) that could form the basis of such a daemon.

>> Alternatively, we could use triggers to maintain low-zoom summary
>> features in their own database tables.
> 
> I was thinking of replication one level above the database - let the API
> write a kind of "spool file" where it stuffs things in addition to
> writing them to the database. Others can then use that spool file to
> feed their own databases. Again, in my eyes triggers are unsuitable for
> handling the intensity of write access we are seeing, but I haven't
> *tried* and I'd be happy for anyone to prove the contrary.

I agree with the spool file idea. This is the only sensible way to do
the global RSS feed, for example. Of course, one good way to implement
the spool file is as a table in the DB that is updated with triggers :-)
(see http://fibre.sf.net/ for my generic DB replication system that does
just that)

OTOH, I'm not sure that the intensity of write access is the problem we
have. I think (but this is only a hunch) the writing problems are mostly
due to things being locked by the read operations. PostGres would remove
these issues with it's MGA architecture.

>>> I would be extremely happy to see a "live renderer", don't get me wrong
>>> - but I am not exactly over-optimistic that we'll have it within the
>>> next half year.
>>
>> It is a solved problem. See http://labs.metacarta.com/osm. We just have
>> to get updates into it as they happen, rather than once a week. PostGres
>> triggers (or rules) are almost certainly capable of doing this.
> 
> When saying "live renderer", I was referring to something that uses live
> data. The solved part is creating PNGs from specially prepared data on
> the fly; the unsolved part is special-preparing our data in real time. I
> see all sorts of problems looming, but maybe I am just overly skeptical.

Converting a way to geometric is really just a case of selecting the
segments of the way in order, and for each segment checking that it
starts where the previous segment ends. As long as it does, you have a
line. If at any point it doesn't, you have one line of a multi-line
geometry (like a forked road or an area with holes). Once you've done
that, you just check for loops to get polygons and holes.

The only problem I foresee is that we might not want roundabouts etc. to
be polygons in the PostGIS DB, which might need the triggers to
understand the tags. That's a fairly horrible concept, because not only
do we not have separate polygon and linear way object types, we can't
even tell the difference by looking at one key. We have to check a whole
multitude of keys that might signal something is an area. I think that
we should say that any area way should have a tag area=1 or something,
and migrating to separate polygonal and linear way objects in the
medium/long term would be a good idea. They should both still be
collections of nodes (with or without segments in between - I'm
undecided on that), however. I'm really keen that something like a car
park area can be topologically connected to the road network at it's
entrances and exits. If you look at something like a TomTom navigation
system, what happens is a tiny subset of the roads in a car park are
modelled, or none at all, and either way the device gives nonsense
instructions. If it knew that the area was a car park, it could just
draw the right shape, and instruct you to head in roughly the right
direction to reach the exit that it wants to direct you to for the next
part of your journey.

Robert (Jamie) Munro
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGOnLAz+aYVHdncI0RAqygAJ0ZTE/KF4MqHIQ635z/zMNl6fWXsACfUwIG
UkJ7Z6dNDzUmM7sJdyXWoJM=
=tTlJ
-----END PGP SIGNATURE-----