[Tilesathome] [PATCH] Don't rename uploaded files to ${file}_uploaded if using a common upload directory

[Dirk-Lüder Kreie]

Ævar Arnfjörð Bjarmason schrieb:
> On Mon, Oct 27, 2008 at 6:28 PM, Matthias Julius <lists at julius-net.net> wrote:
>>> The uploadable directory in each client's working directory is then
>>> symlinked to a central uploadable directory everyone uses:
>>>
>>> $ file work/tah-{3,upload}/uploadable
>>> work/tah-3/uploadable:      symbolic link to `/home/avar/src/tah/uploadable'
>>> work/tah-upload/uploadable: symbolic link to `/home/avar/src/tah/uploadable'
>> I would call that a broken setup.  All rendered zip files should now be
>> in /home/avar/src/tah/uploadable and collect dust.
>>
>> Each client's uploadable directory is supposed to be private to the
>> client.  If UploadToDirectory is set the client copies all zip files
>> from there to UploadTargetDirectory.  So, instead of symlinking you
>> should set UploadTargetDirectory to
>> /home/avar/src/tah/work/tah-upload/uploadable and everything should
>> work as expected.
> 
> Deleting the symlinks in each of the non-upload client's working
> directories does work, they now use their working directory's
> uploadable directory as a scratch dir and move things to the global
> UploadTargetDirectory when they're finished.

That's the intended behaviour.

> The only reason I was using the symlinks however was that at one time
> (I'm almost positive) it was required, I tried setting up multiple
> slave clients and one upload client before but even with
> UploadTargetDirectory set on all of the non-uploading clients files
> would only move as far as their working directory's uploadable, so I
> symlinked them. It's good to see that this now works whether by
> something being fixed in SVN or through some previous convoluted error
> of my own.

This is highly unlikely because I use this exact method for my own 
renderfarm.

> The slave client is now configured as:
> 
> WorkingDirectory = /home/avar/src/tah/work/tah-3
> UploadToDirectory = 1
> UploadTargetDirectory = /home/avar/src/tah/uploadable
> 
> And /home/avar/src/tah/work/tah-3/uploadable is not a symlink but its
> private work area, it drops things in the UploadTargetDirectory when
> done, as expected:
> 
>  ls -l /home/avar/src/tah/uploadable/
> total 560
> -rw-r--r-- 1 avar avar   6925 2008-10-27 18:58
> captionless_12_1854_1097_62744.zip
> -rw-r--r-- 1 avar avar  75853 2008-10-27 18:58 maplint_12_1854_1097_62744.zip
> -rw-r--r-- 1 avar avar 477823 2008-10-27 18:58 tile_12_1854_1097_62744.zip
> 
> However the upload client configured as:
> 
> WorkingDirectory = /home/avar/src/tah/work/tah-upload
> UploadToDirectory = 0
> ForkForUpload = 0
> DeleteZipFilesAfterUpload = 0
> 
> Will die when I run `perl tilesGen.pl upload_loop':
> 
>     could not read /home/avar/src/tah/work/tah-upload/uploadable at
> lib/Upload.pm line 90.
> 
> This is because the upload code expects the directory it uploads from
> to be inside the clients working directory:

This is the desired behaviour. Each client caters for it's own files, 
and not for some strange files. Some platforms don't support locking as 
woukld be required otherwise so this is sort-of mandatory.

>     $self->{TileDir} = $self->{Config}->get("WorkingDirectory"),
>     $self->{ZipDir}  =
> File::Spec->catdir($self->{Config}->get("WorkingDirectory"),
> "/uploadable"),
> 
> So it'll work if I symlink
> /home/avar/src/tah/work/tah-upload/uploadable to
> /home/avar/src/tah/uploadable but it's a somewhat hacky solution.

you could also do what was intended, and use the workdir/uploadable of 
the uploader client as target directory for your actual renderers.

-- 

Dirk-Lüder "Deelkar" Kreie
Bremen - 53.0952°N 8.8652°E

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 260 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstreetmap.org/pipermail/tilesathome/attachments/20081027/4e88a541/attachment.pgp>