[OSM-talk] Server slowness

[Nick Hill]

Let's hold back on dropping the coastline at least for today; given the database 
is so slow, perhaps people wouldn't mind me taking it offline to try database 
partitioning, at least for nodes and GPS points. I may take it offline at around 
  15:00 GMT.



Ray Booysen wrote:
> On that note, as the coastline import is pretty much automatic, should 
> we delete the coastlines for africa until we can move to the tile based 
> index.  Its not very important to have the coastline import currently 
> and that should aleviate the issues for the time being.  I've spoken to 
> the other Africa coastline importer (Firefishy) and he agrees that we 
> should do this.  Between the two of us, large portions of the east and 
> west of Africa have been done, but both of these can be re-imported at a 
> later date.
> 
> Also, our imports finished right at the beginnning of January and we 
> have stopped imports until these other issues can be sorted out first.
> 
> Comments? :)
> 
> Thanks
> Ray
> 
> On 1/14/07, *Nick Hill* <nick at nickhill.co.uk 
> <mailto:nick at nickhill.co.uk>> wrote:
> 
>     Looking at the disc usage for the db, 10% of the hard disc was
>     filled with data
>     between 29th Dec and 5th Jan. This represents 4Gb of data, including
>     indexes,
>     which represents a 25% increase in the total OSM data set.
> 
>     I have taken a look at the Africa coastline. It is very detailed,
>     and looks like
>     it even details mangrove swamps etc. So it is not beyond possibility
>     this
>     increase in data set is caused by African coastline.
> 
>     This could cause a substantial hit for queries hitting Europe. Our
>     database
>     currently narrows the index down on either lat or lon, not both.
>     Therefore,
>     look-ups for Europe may require brute force searches to remove the
>     African
>     coastline from the indexed dataset. That would impact performance
>     until we move
>     to a tile based index.
> 
> 
> 
> 
> 
> 
> 
>     Nick Hill wrote:
>      > It is still worrying me how the database load has shot up on 30th
>     December. This
>      > inevitably causing slowness.
>      >
>      > When did the Africa coastline import run from/to?
>      > How many points were involved?
>      > Was there any publicity of OSM on that day which could have led
>     to a significant
>      > increase in users? Would this result in this profile of usage ?
>     (I think not but
>      > prepared to debate it).
>      >
>      > Steve, did you make any changes to the API or DB set-up around
>     that time?
>      >
>      >
>      >
>      >
>      > Andy Robinson wrote:
>      >
>      >> I thought it was probably the Africa Coastline stuff too. We had
>     nearly half
>      >> a million node writes a day going in earlier in the week, but as
>     you point
>      >> out that situation has declined now. The only other thing that
>     springs to
>      >> mind is Steve's mapnick db rendering. I'm assuming that's coming
>     off the db
>      >> direct rather than using the last planet. If this is the case I
>     presume then
>      >> it will take a while for the first run of tiles to be completed,
>     after that
>      >> the load should drop of course.
>      >
>      > _______________________________________________
>      > talk mailing list
>      > talk at openstreetmap.org <mailto:talk at openstreetmap.org>
>      > http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
>      >
> 
>     _______________________________________________
>     talk mailing list
>     talk at openstreetmap.org <mailto:talk at openstreetmap.org>
>     http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
> 
> 
> 
> 
> -- 
> Ray Booysen
> raybooysen at rjb.za.net <mailto:raybooysen at rjb.za.net>
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> talk mailing list
> talk at openstreetmap.org
> http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk