[Tilesathome] Observations after having played around with t at h server

[Robert (Jamie) Munro]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

spaetz wrote:
>> Thirdly, on my laptop its not like the _entire_ DB access is a low 
>> hanging fruit at all. My disk is arguably very slow imo. Removing all DB 
>> access alltogether takes the time down to 10 seconds from 19s. Thats 
>> still "only" one tileset per second. Initially I stored all tiles on the 
>> disk, and the times for 10 zips was ~65 seconds, moving to a RAM-disk 
>> brought this down to 30 seconds.
> 
> Nice, unzipping files to RAM disk would be good. However on this machine with 1GB RAM and two data abses running and apache running, I think, we can't afford big RAM disks. There are plans to upgrade dev's memory though. Tiles are just moved on our disk within one partition, so the moves should be really quick.


I'm not sure how a RAM disc could ever help at all. The zip file will be
uploaded to real disc. The unzipped files will end up on the real disc.
mv's are cheap. Copying the zip file to a RAM disc, then unzipping, then
copying the files back seems like 2 pointless copies. And then the RAM
disc takes away some of the RAM. I'm sure I've heard that using RAM
discs can often be slower because the real disc is cached, so if you
create a file, edit it, then delete it soon after, it may never touch
the disc - it only ever exists in the cache. But if you create a file,
leave it alone for a while, it quickly gets saved to the disc and the
memory is freed for other things. If you create it on a RAM disc, it is
more likely to stay in RAM (but it may be possible for it to be paged
into swap - I'm not sure).

Another thought just occurred to me. Why are we using zip files?
Compressing already-compressed PNGs isn't going to save any bandwidth.
We may be better using uncompressed .tar files. I'm sure performance
will be better, both for packing and unpacking, and the cost in extra
data would probably be small. Alternatively, is it actually that
inefficient to upload tiles one by one? If you can use a persistent ftp
or http connection of some kind, it may not be.

Robert (Jamie) Munro
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGqn83z+aYVHdncI0RAuZ/AJ9OhfbyC1cTvAgiOXAkO2Ptm6LH7wCePRwJ
9verdZ4LN4UUBzVsopJzkpY=
=K3RW
-----END PGP SIGNATURE-----