[OSM-talk] Contours server (was: Re: ski pistes)

[Dave Stubbs]

On Mon, Mar 17, 2008 at 5:43 PM, Steve Hill <steve at nexusuk.org> wrote:
> On Mon, 17 Mar 2008, Dave Stubbs wrote:
>
>  > There's also the amount of time it would take to render the tiles. It
>  > takes over 15 hours to render the contours used on the cycle map, and
>  > all things considered that covers a very small % of the planet surface
>  > at any kind of decent zoom level. It's a slow enough process that
>  > render on demand doesn't work so well either. So any such server would
>  > need a few TB of disk space or be a "come back in 24 hours" after
>  > request job, or both.
>
>  I've been quite successful with doing render on demand - tiles which
>  have never been rendered are pushed into a render queue and the HTTP
>  connection is left idling.  If the tile is rendered within 30 seconds the
>  user gets it sent over the HTTP connection as soon as possible, otherwise
>  they get a 404 and the tile remains in the queue until it is rendered.
>  The render-server process is currently set to be able to render up to 4
>  tiles in parallel and also copes well with requests for tiles that are
>  already queued or being rendered.
>
>  That said, it is running on a pretty meaty server (an 8 core 1.86GHz Xeon
>  E5320), and it isn't very busy (serving 15k - 36k tiles per day over the
>  past 5 days).  I suspect that storing the contours in PostGIS helps quite
>  a bit too.

Yeah, on the cycle map it manages 12 tiles/sec with 4 cores. I was
mainly comparing with standard OSM tiles which I can get nearly 200
tiles/sec. I'm assuming you can get about 20-30 tiles/sec, probably
higher with your wider spaced contours. So it'll probably work as long
as no one starts trying to scrape the tiles...

>
>  My main performance issues revolve around importing the planet OSM file
>  (osm2pgsql uses up crazy amounts of RAM (or rather, swap, in my case) -
>  there is the "-s" option, but that is currently broken...).  What I'm
>  really after is a way to import the daily diffs into the existing PostGIS
>  DB.

That would hopefully speed up the weekly import I do quite a lot. Not
a very trivial change though. I wouldn't recommend the -m option even
if it did work... it's a LOT slower -- it's faster to just let it use
swap, but it might be worth it if you could then apply updates.
Personally I now have 8GB RAM so don't care so much as long as the
memory requirements stay below 4GB (it's only on a 32bit OS).