[Tilesathome] [PATCH] Don't rename uploaded files to ${file}_uploaded if using a common upload directory

[Ævar Arnfjörð Bjarmason]

On Mon, Oct 27, 2008 at 6:28 PM, Matthias Julius <lists at julius-net.net> wrote:
>> The uploadable directory in each client's working directory is then
>> symlinked to a central uploadable directory everyone uses:
>>
>> $ file work/tah-{3,upload}/uploadable
>> work/tah-3/uploadable:      symbolic link to `/home/avar/src/tah/uploadable'
>> work/tah-upload/uploadable: symbolic link to `/home/avar/src/tah/uploadable'
>
> I would call that a broken setup.  All rendered zip files should now be
> in /home/avar/src/tah/uploadable and collect dust.
>
> Each client's uploadable directory is supposed to be private to the
> client.  If UploadToDirectory is set the client copies all zip files
> from there to UploadTargetDirectory.  So, instead of symlinking you
> should set UploadTargetDirectory to
> /home/avar/src/tah/work/tah-upload/uploadable and everything should
> work as expected.

Deleting the symlinks in each of the non-upload client's working
directories does work, they now use their working directory's
uploadable directory as a scratch dir and move things to the global
UploadTargetDirectory when they're finished.

The only reason I was using the symlinks however was that at one time
(I'm almost positive) it was required, I tried setting up multiple
slave clients and one upload client before but even with
UploadTargetDirectory set on all of the non-uploading clients files
would only move as far as their working directory's uploadable, so I
symlinked them. It's good to see that this now works whether by
something being fixed in SVN or through some previous convoluted error
of my own.

The slave client is now configured as:

WorkingDirectory = /home/avar/src/tah/work/tah-3
UploadToDirectory = 1
UploadTargetDirectory = /home/avar/src/tah/uploadable

And /home/avar/src/tah/work/tah-3/uploadable is not a symlink but its
private work area, it drops things in the UploadTargetDirectory when
done, as expected:

 ls -l /home/avar/src/tah/uploadable/
total 560
-rw-r--r-- 1 avar avar   6925 2008-10-27 18:58
captionless_12_1854_1097_62744.zip
-rw-r--r-- 1 avar avar  75853 2008-10-27 18:58 maplint_12_1854_1097_62744.zip
-rw-r--r-- 1 avar avar 477823 2008-10-27 18:58 tile_12_1854_1097_62744.zip

However the upload client configured as:

WorkingDirectory = /home/avar/src/tah/work/tah-upload
UploadToDirectory = 0
ForkForUpload = 0
DeleteZipFilesAfterUpload = 0

Will die when I run `perl tilesGen.pl upload_loop':

    could not read /home/avar/src/tah/work/tah-upload/uploadable at
lib/Upload.pm line 90.

This is because the upload code expects the directory it uploads from
to be inside the clients working directory:

    $self->{TileDir} = $self->{Config}->get("WorkingDirectory"),
    $self->{ZipDir}  =
File::Spec->catdir($self->{Config}->get("WorkingDirectory"),
"/uploadable"),

So it'll work if I symlink
/home/avar/src/tah/work/tah-upload/uploadable to
/home/avar/src/tah/uploadable but it's a somewhat hacky solution.

At the risk of submitting more frivolous patches the following patch
also makes it work in a slightly less hacky way, it allows the user to
set a UploadZipDirectory variable allowing him to point the upload
client to an upload directory outside its working tree, if it's not
set the old behavior will be unchanged.

[Regardless of the sanity of my patch the comma-as-statment delimiter
around line 49 in Upload.pm should be changed, it only works because
bless is a listop]
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Upload.pm.patch
Type: text/x-diff
Size: 750 bytes
Desc: not available
URL: <http://lists.openstreetmap.org/pipermail/tilesathome/attachments/20081027/62bd95c2/attachment.patch>