[Tilesathome] Proposal: Keeping tileset as one file

[Florian Lohoff]

On Mon, Jun 09, 2008 at 12:18:01AM +0200, Dirk-Lüder Kreie wrote:
> Instead of proclaiming a bottleneck, why not show us?
> 
> Things I can see in favor of these aggregated files would be that you 
> would probably need to look at most at 4 of these on any higher 
> zoomlevel (~z14 and up) to get a whole area out of them.
> 
> Sadly I don't have any statistics on the workload and/or bottlenecks we 
> encounter *serving* the tiles, I only know of the bottleneck of 
> uploading the tiles, which is getting the data onto the target disks.
> I remain unconvinced that this method you propose would do much.
> So let me reiterate:
> 
> Show us.

flo at touch:~/projects/tah$ time ( ./createworkset -t dir; sync )
Pages: 191390 Pagesize: 4096
Need to create a working set of minimum 2242Mbyte

real    12m26.486s
user    0m0.812s
sys     0m17.013s
flo at touch:~/projects/tah$ time ( ./createworkset -t flat; sync )
Pages: 191390 Pagesize: 4096
Need to create a working set of minimum 2242Mbyte

real    1m36.243s
user    0m0.772s
sys     0m14.749s

This is a working set creation with 0-32Kbyte files e.g. average of
16Kbyte. The dirset is a 2 level directory e.g. workset/z/x/y.png
with z ranging from 0-7 x and y ranging from 0-512

The flat file model e.g. container is a single container with an
additional index file.

The hardware is a single P4 2.4Ghz, 768MByte memory, a single IDE disk
~80% full, ext3 filesystem with atime set.

The difference in time is caused by the disk performance of unzipping or
creation of files is bound by seeks not by streaming performance. One
can see from the times that in the end the one file per tile model does
not need significantly more CPU time but rather waits more on the disk.

I'll write a simple benchmark for access e.g. serve performance in a
minute. I'll put it online for others to play afterwards ...

Flo
-- 
Florian Lohoff                  flo at rfc822.org             +49-171-2280134
	Those who would give up a little freedom to get a little 
          security shall soon have neither - Benjamin Franklin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.openstreetmap.org/pipermail/tilesathome/attachments/20080609/84580240/attachment.pgp>