[OSM-talk] Mapnik tileset coherency issues

Tom Hughes tom at compton.nu
Thu Oct 18 09:23:27 BST 2007


In message <471711BB.5080302 at ucl.ac.uk>
        Patrick Weber <p.weber at ucl.ac.uk> wrote:

> Mapnik's tileset gets updated once a week on Wednesday's, is that
> right? I wondered how long the update of the whole tileset takes, as I
> can see now on Thursday morning, that depending on the zoom level,
> changes are included, and when zooming in, they disappear again. Also,
> between tile boundaries, changes might appear and thus new roads get
> cut off.

There is no global update, nor do we ever have a "complete" set of
tiles on disk so it's impossible to give a time for something that
doesn't actually happen.

Broadly speaking, the algorithm is that, when you look at a tile it
is marked as dirty if it is more than three days old. If the machine
is fairly idle then it is rendered immediately and returned otherwise
the old tile is returned and the dirty tile is rendered in the background.

The consequence of that is that if somebody looks at an area of the
map on Tuesday evening then the earliest it will be rerendered is
Friday evening even though a new planet was loaded on Wednesday.

Tiles which haven't been looked at for a file (several weeks I think) are
removed from disk.

The new solution Jon is working on has knowledge of when planet was
loaded, so will mark tiles as dirty when they are viewed after the
planet load even if three days have passed, and won't bother if they
have been rendered since the last planet even if three days have
passed - currently they are needlessly rerendered at that point.

> As I see it, Mapnik is the "production" tileset, the official face of
> OSM, versus OsmaRender T at H is for map editors who want to check their
> updates and people wanting the latest data. My point is then, that
> Mapnik should be an attempt to present a coherent view of the OSM
> data. Thats where I think the updating artefacts become an issue.

I think a few people are going to disagree with you on that one ;-)

> Now, as I understand it, only tiles marked "dirty" get rerendered and
> immediately replace old versions in the live tileset.

More or less correct...

> Could there be a mecanism to render these tiles without committing
> them immediately to the live tileset, saving them in a cache, and only
> when all dirty tiles have been rendered, to update the tileset in one
> go by replacing tiles in the live tileset? Would that solve coherency
> issues I just explained? Would there be enough space on the server to
> keep while rendering thus dual copies of "dirty" tiles ?

I expect it is very rare that the dirty tile queue ever falls to
zero, and even then it is no guarantee that the map is consistent.

I suspect what you really mean is that you want all existing tiles
to be marked dirty when the planet is loaded and for them all the
be rerendered and then deployed? I don't think that is feasible
unfortunately - we would probably still be rerendering when the 
next planet was released ;-)

Tom

-- 
Tom Hughes (tom at compton.nu)
http://www.compton.nu/




More information about the talk mailing list