[openstreetmap/openstreetmap-website] Why are some diff uploads slow? (#1710)

mmd notifications at github.com
Fri Dec 22 16:31:04 UTC 2017


I think It was a good idea to take a look at the minutely diff contents rather than just checking `txnActiveList`. I don't seem to find those extreme cases anymore with 2++ hours runtime anymore. These really seem to originate from unrelated processes like backup or vacuum.

However, we still have minutely diffs, where the first and last timestamp differs by as much as 32 minutes (one example: http://www.openstreetmap.org/changeset/43755443). Those should be good candidates for further performance analysis.

I'm processing all minutely diffs in directory `002/*/*` now, which will still take a while to complete. The list below is just a snapshot after an hour or so. Will be updated again once finished.

Filename | Earliest TS | Latest TS | Delta
-- | -- | -- | --
002/190/051.osc.gz  | 2016-11-18T06:47:49Z | 2016-11-18T07:20:00Z | 00:32:11
002/468/973.osc.gz  | 2017-05-31T02:18:07Z | 2017-05-31T02:48:59Z | 00:30:52
002/086/408.osc.gz  | 2016-09-06T23:33:52Z | 2016-09-07T00:03:06Z | 00:29:14
002/468/764.osc.gz  | 2017-05-30T22:51:51Z | 2017-05-30T23:20:08Z | 00:28:17
002/273/775.osc.gz  | 2017-01-15T11:20:48Z | 2017-01-15T11:47:02Z | 00:26:14
002/468/490.osc.gz  | 2017-05-30T18:20:03Z | 2017-05-30T18:46:14Z | 00:26:11
002/468/560.osc.gz  | 2017-05-30T19:32:07Z | 2017-05-30T19:56:12Z | 00:24:05
002/748/051.osc.gz  | 2017-12-11T00:08:20Z | 2017-12-11T00:31:53Z | 00:23:33
002/468/367.osc.gz  | 2017-05-30T16:21:53Z | 2017-05-30T16:43:15Z | 00:21:22
002/397/331.osc.gz  | 2017-04-11T08:11:43Z | 2017-04-11T08:32:02Z | 00:20:19
002/468/609.osc.gz  | 2017-05-30T20:26:06Z | 2017-05-30T20:45:10Z | 00:19:04
002/048/419.osc.gz  | 2016-08-11T13:40:17Z | 2016-08-11T13:59:00Z | 00:18:43




-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/openstreetmap-website/issues/1710#issuecomment-353631926
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/rails-dev/attachments/20171222/4b470813/attachment.html>


More information about the rails-dev mailing list