[Tilesathome] [osm at sspaeth.de: Re: Tileset-at-once - we can now upload 8MB ZIP files]

[spaetz]

----- Forwarded message from spaetz <osm at sspaeth.de> -----

From: spaetz <osm at sspaeth.de>
Date: Mon, 23 Jul 2007 12:58:24 +0200
To: tilesathome at openstreetmap.org
User-Agent: Mutt/1.5.16 (2007-06-09)

> >> (3) There is also a suggestion that the limit should be more like 20MB (so 
> >> that even the largest tilesets can be uploaded in one go) - any comments on 
> >> that idea?

> If you turn this into a sort of optimization problem and postulate that
> the average tileset contains a 1000 Tiles and that a split tileset thus
> costs the server 1000x as much metadata updates

The disk where the mysql database is stored is more busy (or at least equal) than the disk where the tiles are stored, so *anything* that reduces MySQL load helps us a lot.

So if we can reduce mySQL "replaces" by a factor of 1000, I'd say: go for it.

I wonder whether our current custom PHP upload script is needed at all. Why don't we just have a "chroot'ed" vsftp installation and uploaders upload their zips into that directory. Then the queue processing script comes along and processes files as it gets to them. If the queue gets too long, people won't be able to upload anymore and get "disk full" messages (as the allocated queue space ran out).
This let's us take advantage of a couple of mature and existing stock tools without the need of much home-grown php glue.

spaetz



----- End forwarded message -----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 186 bytes
Desc: not available
URL: <http://lists.openstreetmap.org/pipermail/tilesathome/attachments/20070723/bca8eb9a/attachment.pgp>