[Tile-serving] Problems benchmarking osm.xml
penorman at mac.com
Sat Apr 27 23:52:24 UTC 2013
I've been trying to benchmark the new carto style, so I decided to start
with a baseline from osm.xml.
I took a list of 2000 meta-tile requests from a log from yevaud, and put the
first half into one list and the second half into another.
I then rendered the first list, the second, disregarding the times. This was
to get the cache into the a consistent state.
I then alternated between the first list and the second five times,
recording the time to render the second.
The first time I did this I noticed that the 5 benchmark times were
monotonically decreasing. Worried that I was having caching issues, I reran
the entire test sequence twice with the same tile lists.
I got a result of an average of 4.9k seconds (0.35 meta-tiles per second)
with a standard deviation of 4k seconds.
If I throw out the two most extreme results, I still get a standard
deviation over 50% of the average.
Including both render sets, I did about 24 hours of rendering. I'm doing
this on errol which has other processes running, but I would expect them to
Does anyone have any suggestions on how to get times which I can use to
compare various options, I would welcome them. Right now it seems like I'd
have to do too much benchmarking to get a meaningful result.
More information about the Tile-serving