[Tilesathome] WANTED: Tile viewing fastcgi written in C/C++

[Kai Krueger]

spaetz wrote:
> Hi all, thanks for bearing with the recent outages on the t at h server, it was overloaded for several reasons (e.g. having OSMXAPI and t at h on the same server doesn't help as one is very busy when the other is). One of the big problems proved that the upload slots were taking up all apache slots quite quickly and many of the uploads took between 5-8 mintes. I don't know whether that is because people are running t at h from slow home ADSL connections with crappy upload times. Thanks go to Firefishy (Grant Slater) for helping debug the apache server.

I could imagine that quite a few people upload through ADSL, which would 
easily take 5 - 10 minutes on the more complex tilesets. I know that I 
do. I could stop if it is seen as a problem though. Hopefully the move 
to the colour quantized tiles will have reduced the average tileset size 
by quite a bit and thus the time it takes to upload.

Some of the load on XAPI is probably caused by t at h clients? Does this 
fallback make sense at the moment in this case? I feel that nearly 
always when the main API has no data (i.e. times out), the same happens 
to the XAPI. So it might only add extra load onto the XAPI without 
gaining much? I don't have any numbers to back this up, so I don't 
really know if this would help or harm.

> 
> I have now installed a lighthttp instance with 7 slots and all uploads will be redirected to using that. So from now on, uploads should not prevent us from doing tile serving. Let's see if this helps already.

Although this is a great idea in general and I think it makes complete 
sense to split those two as well as possible, since then I can't upload 
any tiles any more. I suspect it is caused by a stupid firewall that 
blocks port 81 on our university network. Do more people have this problem?


> 
> Another issue - and that is where you can help - is that currently all tiles are served via mod_python. While this works fine and reasonably fast, it consumes quite a bit of memory. I'd appreciate if someone could sit down and would write a C-based (or whatever high performance, low overhead lingo you prefer) fastcgi that does tileserving. All the python code is in those 2 files:

I had started writing an apache module in C for the tileset stuff before 
I heard that Spaetz rewrote the entire server code. If it is helpful, I 
would be happy to continue on it again and adapt it to the new server 
layout. It would probably take a couple of days though till I find the 
time to finish of the testing. Please let me know if other people are 
working on it or if the memory consumption is no longer an issue to 
prevent duplicate effort.

> 
> http://svn.openstreetmap.org/sites/other/tilesAtHome_tahngo/tah_intern/serve_tiles.py
> http://svn.openstreetmap.org/sites/other/tilesAtHome_tahngo/tah_intern/Tile.py
> 
> And I would be happy to give assistance to those needing it.
> 
> Thanks,
> spaetz

Kai

> 
> _______________________________________________
> Tilesathome mailing list
> Tilesathome at openstreetmap.org
> http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/tilesathome