[OSM-dev] osm2pgsql tile expiry freaks me out

[sly (sylvain letuffe)]

On mercredi 4 novembre 2009, Frederik Ramm wrote:
> Hi,
> 
>     I'm trying to work with the tile expiry list produced by osm2pgsql 
> but something seems to be very wrong there. Last night's daily diff, for 
> example, had 459 MB; about 2.3 million objects were changed. The 
> corresponding osm2pgsql tile expiry list for zoom level 18 (done with -e 
> 18-18) has 5.6 GB and claims to mark dirty a whopping 350 million 
> objects. That would mean that one changed object, on average, touches 
> more than 100 tiles on zoom level 18.

I'm sorry I don't have any clue about what your problem is, but I confirm 
those figures to be very high and strange. 

My version is rather old :
osm2pgsql SVN version 0.66-15819M

Maybe that could help spot the problem.

Figures I can provide are like this :
- I'm only working on an europe BBOX (--bbox -27,31,50,72 -e 18)
- Using hourly diffs
- Every 5 hours I compute full tile list to expire

My "expire_list" after 5 hours is around 50Mb on average
Contains around 2M tiles to set dirty at zoom 18

haven't you duplicates in your tiles_list ?

<blabla my life>
This may occasionnaly vary a lot when some bulk modification are down to large 
relations (such as france admin_level 8 boundaries ;-) ) but it stays 
manageable at cruise speed.

On the overall, I'm not unhappy with the result, and just faced the "large 
area" problem.
Last big france import of corine land cover data (where you would need to 
expire tiles inside big forest) let my tiles into a strange funny state where 
I only had the perimeter of large forest/farms/... 
But that is the price to pay to avoid expiring every tiles when one changes 
the name:XY of the boundary of a big country.

More perfect but harder approax would probably be (as said in this thread) 
have a clue about what is an area, what is a border.

-- 
sly 
Sylvain Letuffe liste at letuffe.org
qui suis-je : http://slyserv.dyndns.org