[OSM-talk] Mapnik tileset coherency issues

[Steve Chilton]

Patrick raises an issue that many in the project often forget about, and
that is the image that is presented to allcomers, whether familiar with
the ins-and-outs of the project or not.
A commercial project would background render a new tileset and release
it in one go on a planned basis. This is not a commercial project and we
have arranged (decided on?) a response mechanism that allows work to
show as soon as possible - people just want to see their mapped stuff
ASAP. This is what I mean by it being an inwardly focussed rather than
outwardly focussed project in many ways.
I do have qualms about seeing adjacent mapnik layer tiles rendered with
seemingly different data. It can of course occur both in adjacent tiles
on the same layer, and between the same area/tile on different layers.
Could I suggest that Jon (and any others) working on revising the
process consider trying to apply some "adjacency" intelligence? So, for
instance when rendering a dirty tile is it possible to check adjacent
tiles (immediately there will be 4 abutting tiles at the 4 points of the
compass, and 4 more which meet at the corners) for "dirtiness" and
render them next? Equally, when rendering a particular dirty tile is it
possible to immediately queue all iterations of that tile through the
zoom levels?

Cheers
STEVE

Steve Chilton, Learning Support Fellow
Learning and Technical Support Unit Manager 
School of Health and Social Sciences
Middlesex University
phone/fax: 020 8411 5355
email: steve8 at mdx.ac.uk

Chair of the Society of Cartographers: http://www.soc.org.uk/

SoC conference 2007:
http://www.port.ac.uk/special/soc/

Mind the (Map) Gap:
http://news.bbc.co.uk/1/hi/magazine/5413010.stm



-----Original Message-----
From: talk-bounces at openstreetmap.org
[mailto:talk-bounces at openstreetmap.org] On Behalf Of Tom Hughes
Sent: 18 October 2007 09:23
To: talk at openstreetmap.org
Subject: Re: [OSM-talk] Mapnik tileset coherency issues

In message <471711BB.5080302 at ucl.ac.uk>
        Patrick Weber <p.weber at ucl.ac.uk> wrote:

> Mapnik's tileset gets updated once a week on Wednesday's, is that
> right? I wondered how long the update of the whole tileset takes, as I
> can see now on Thursday morning, that depending on the zoom level,
> changes are included, and when zooming in, they disappear again. Also,
> between tile boundaries, changes might appear and thus new roads get
> cut off.

There is no global update, nor do we ever have a "complete" set of
tiles on disk so it's impossible to give a time for something that
doesn't actually happen.

Broadly speaking, the algorithm is that, when you look at a tile it
is marked as dirty if it is more than three days old. If the machine
is fairly idle then it is rendered immediately and returned otherwise
the old tile is returned and the dirty tile is rendered in the
background.

The consequence of that is that if somebody looks at an area of the
map on Tuesday evening then the earliest it will be rerendered is
Friday evening even though a new planet was loaded on Wednesday.

Tiles which haven't been looked at for a file (several weeks I think)
are
removed from disk.

The new solution Jon is working on has knowledge of when planet was
loaded, so will mark tiles as dirty when they are viewed after the
planet load even if three days have passed, and won't bother if they
have been rendered since the last planet even if three days have
passed - currently they are needlessly rerendered at that point.

> As I see it, Mapnik is the "production" tileset, the official face of
> OSM, versus OsmaRender T at H is for map editors who want to check their
> updates and people wanting the latest data. My point is then, that
> Mapnik should be an attempt to present a coherent view of the OSM
> data. Thats where I think the updating artefacts become an issue.

I think a few people are going to disagree with you on that one ;-)

> Now, as I understand it, only tiles marked "dirty" get rerendered and
> immediately replace old versions in the live tileset.

More or less correct...

> Could there be a mecanism to render these tiles without committing
> them immediately to the live tileset, saving them in a cache, and only
> when all dirty tiles have been rendered, to update the tileset in one
> go by replacing tiles in the live tileset? Would that solve coherency
> issues I just explained? Would there be enough space on the server to
> keep while rendering thus dual copies of "dirty" tiles ?

I expect it is very rare that the dirty tile queue ever falls to
zero, and even then it is no guarantee that the map is consistent.

I suspect what you really mean is that you want all existing tiles
to be marked dirty when the planet is loaded and for them all the
be rerendered and then deployed? I don't think that is feasible
unfortunately - we would probably still be rerendering when the 
next planet was released ;-)

Tom

-- 
Tom Hughes (tom at compton.nu)
http://www.compton.nu/

_______________________________________________
talk mailing list
talk at openstreetmap.org
http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk