[OSM-dev] TIGER upload automation

[Tom Hughes]

In message <1188499685.28903.204.camel at localhost>
          Dave Hansen <dave at sr71.net> wrote:

> On Thu, 2007-08-30 at 18:45 +0100, Tom Hughes wrote:
> > In message <1188491703.28903.188.camel at localhost>
> >           Dave Hansen <dave at sr71.net> wrote:
> > 
> > > On Thu, 2007-08-30 at 09:37 +0200, spaetz wrote:
> > >
> > > > Dave has a dev account now, so he can try if it goes any faster. If
> > > > his Uploads hurt t at h I'll do likewise to him :-)
> > > 
> > > So, the upload from dev is proceeding a wee bit faster: about 20 times!
> > 
> > Excellent - the next thing to try is a reserved API daemon. Do you
> > know what user agent the bulk uploader sends?
> 
> It is just using the perl API client.  So:
> 
>   $obj->{client} = new LWP::UserAgent(agent => 'Geo::OSM::APIClientV4');

There are now an extra three API daemons and all requests from that
agent are being funnelled to those daemons.

Also Kleptog and I have just tweaked the APIClientV4 module to halve
the number of server round trips it does so you might want to update
it from svn and restart your uploads.

If you wonder how we managed that, the answer is that I noticed that
every request was happening twice - the first one was done with no
authentication and resulted in a 401 challenge and only then did it
try again with the authentication details.

Tom

-- 
Tom Hughes (tom at compton.nu)
http://www.compton.nu/