[Tilesathome] API Bottleneck

[Robert (Jamie) Munro]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

80n wrote:
> Sebastian
> The current bottleneck for t at h appears to be the throttled API. 
> Hopefully this will ease a bit after the NL upload is complete, although
> there will still be plenty of load from the TIGER upload.
> 
> I'm wondering if we should consider a mechanism whereby priority 2
> requests are serviced by the Zappy API.  Currently a lot of the priority
> 2 requests are taking over a week to get processed so the Zappy API
> would be just as good even though it's data is up to a week old.
> 
> I would need to make a small change to extend the Zappy API to recognize
> the /api/0.4/map?bbox= style of request, but that is pretty trivial. 
> There would also need to be a change to  tilesGen.pl to tell it where to
> get the data from.
> 
> Perhaps then all pending priority 2 tiles that are unprocessed each
> Wednesday evening could automatically be flagged as being renderable by
> the Zappy API.

I think that the T at H system should move wholesale to the zappy api. The
main API is far too overloaded to support rendering tiles that will
probably never be looked at.

In exchange, the planet dump should be run more often - perhaps every 48
hours or so. In the medium term, moving to osmosis would be even better,
as this would allow planet diffs to run every hour, or maybe even less,
and they could feed the zappy api. Can we create a read-only database
user for osmosis on the real database, and start trying it out? It
sounds like it can already produce a planet dump faster than the normal
planet dumping script.

Robert (Jamie) Munro
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFG8lKmz+aYVHdncI0RAs+uAKCB1CeqmUgyasU0gMRxzgGBfXLLXgCfb4xb
D7utJ4NwpAoN7+7bh451xy4=
=Ljzl
-----END PGP SIGNATURE-----