[OSM-talk] Server slowness
nick at nickhill.co.uk
Sun Jan 14 12:57:03 GMT 2007
Let's hold back on dropping the coastline at least for today; given the database
is so slow, perhaps people wouldn't mind me taking it offline to try database
partitioning, at least for nodes and GPS points. I may take it offline at around
Ray Booysen wrote:
> On that note, as the coastline import is pretty much automatic, should
> we delete the coastlines for africa until we can move to the tile based
> index. Its not very important to have the coastline import currently
> and that should aleviate the issues for the time being. I've spoken to
> the other Africa coastline importer (Firefishy) and he agrees that we
> should do this. Between the two of us, large portions of the east and
> west of Africa have been done, but both of these can be re-imported at a
> later date.
> Also, our imports finished right at the beginnning of January and we
> have stopped imports until these other issues can be sorted out first.
> Comments? :)
> On 1/14/07, *Nick Hill* <nick at nickhill.co.uk
> <mailto:nick at nickhill.co.uk>> wrote:
> Looking at the disc usage for the db, 10% of the hard disc was
> filled with data
> between 29th Dec and 5th Jan. This represents 4Gb of data, including
> which represents a 25% increase in the total OSM data set.
> I have taken a look at the Africa coastline. It is very detailed,
> and looks like
> it even details mangrove swamps etc. So it is not beyond possibility
> increase in data set is caused by African coastline.
> This could cause a substantial hit for queries hitting Europe. Our
> currently narrows the index down on either lat or lon, not both.
> look-ups for Europe may require brute force searches to remove the
> coastline from the indexed dataset. That would impact performance
> until we move
> to a tile based index.
> Nick Hill wrote:
> > It is still worrying me how the database load has shot up on 30th
> December. This
> > inevitably causing slowness.
> > When did the Africa coastline import run from/to?
> > How many points were involved?
> > Was there any publicity of OSM on that day which could have led
> to a significant
> > increase in users? Would this result in this profile of usage ?
> (I think not but
> > prepared to debate it).
> > Steve, did you make any changes to the API or DB set-up around
> that time?
> > Andy Robinson wrote:
> >> I thought it was probably the Africa Coastline stuff too. We had
> nearly half
> >> a million node writes a day going in earlier in the week, but as
> you point
> >> out that situation has declined now. The only other thing that
> springs to
> >> mind is Steve's mapnick db rendering. I'm assuming that's coming
> off the db
> >> direct rather than using the last planet. If this is the case I
> presume then
> >> it will take a while for the first run of tiles to be completed,
> after that
> >> the load should drop of course.
> > _______________________________________________
> > talk mailing list
> > talk at openstreetmap.org <mailto:talk at openstreetmap.org>
> > http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
> talk mailing list
> talk at openstreetmap.org <mailto:talk at openstreetmap.org>
> Ray Booysen
> raybooysen at rjb.za.net <mailto:raybooysen at rjb.za.net>
> talk mailing list
> talk at openstreetmap.org
More information about the talk