[Openstreetmap] speed, at last

[SteveC]

* @ 11/01/06 06:33:29 PM lars at aronsson.se wrote:
> I'm drawing maps, using the applet, and it's slow as usual.  
> Suddenly around 18.30 GMT (a few minutes ago) something breaks 
> down and the background satellite images disappear.  The 
> background tiles are now only white with yellow dots.  And they 
> come fast.  Please let it stay this fast.  Don't fix it.

I'm sorry it's slow.

How does the tile server work?

An incoming http request hits squid and if the URL has been seen in the
past 48 hours it's returned from squids 2Gb cache (I know, it's too
small). If not then it's passed on to Apache which boots mapserver.
Mapserver wakes up and makes a request to the landsat WMS server in the
US which may or may not return within mapservers timeout range. After
that happens, it makes another http request to itself. This hits squid
and then apache which boots up a ruby script which either generates a
transparent png with yellow dots or street segments.

This is returned to mapserver (which does the requests sequentially, not
in parallel) which composits them with the landsat tile and then returns
a completed tile to apache, which passes it to squid, which passes it to
you.

Now when the applet wakes up it can only contact www.open... and not
tile.open... because of the default Java security thingumys. So there is
an added layer in that it makes http requests to www which passes them
on to tile, which passes them through squid to apache...

There are a lot of dependencies there where I would prefer to KISS. The
other problem is bandwidth and the servers lack of memory.

There are multiple things we're doing to try and make this all faster:

The applet could download the gps data seperately. It could download the
actual data or pictures and do the compositing at the client. Open
question of which is faster and uses less memory. The memory thing is
important because otherwise you get java.lang.OutOfMemeoryErrors at the
client and a crashed applet. From the sound of it, people don't even
care about the landsat for editing anyway. Is that the case? I know
making landsat an option has been a ticket for a while.

The tile server needs re-architecting. Some scheme where it stores tiles
indefinitely if nothing's changed. And I'd like to replace mapserver
with something threaded to generate the tiles more quickly.

As mikel mentioned, having our own landsat WMS would be really useful
and at 300Gb is good because drives that size are no-brainer cheap. But
it's unprojected AFAIK and so might need some work. In the meantime
we'll get a better WMS cache together.

As for the pass-through the applet uses to get to the tile server via
www.open... one way to solve that is to sign the applet. My limited
experience is that signed applets scare people but it has the massive
advantage that the applet could contact the landsat WMS service itself,
completely removing the need for the tile server to serve the applet and
giving us lots of bandwidth back. Does anyone mind clicking 'yes' to an
applet security box?

One thing you might not know is that you can use the applet from your
computer as an application. It's not packaged up as nicely as JOSM just
yet. Running as an application also side-steps all the securitty issues
and so will make it faster. Is that something anyone would want to do?

Another issue is the tiling. Unfortunately the javascript slippy map and
the applet use different tilings. And they use different tilings (and
projection) from things like worldkit. They're all seperate
implimentations... and I know mikel has talked to the landsat guys about
standardising on a tiling. Making things use the same tiling will reduce
our cache usage and make everything faster.

On the other hand we could write a flash editor. Petter will scream at
me because the runtime is non-free :-) but it solves all of the issues
above and is more widely installed than Java. And it's faster.

Finally I want to make a request to the people writing editors to re-use
the tiling code from the editor (you don't want to try reading the
javascript code) so that we all use the same cache, whether that turns
out to be OSM or the landsat wms in the US.

have fun,

SteveC steve at asklater.com http://www.asklater.com/steve/