[openstreetmap/openstreetmap-website] Partition large tables (#2076)

Tom Hughes notifications at github.com
Wed Nov 28 17:47:14 UTC 2018


I was thinking more like 10 to 100 thousand rows in a chunk if we want to make them small enough that a lock doesn't cause an issue!

Sure billion node partitions might reduce the time from a day to a few hours say, if there's enough I/O concurrency to get a more or less linear speedup, but it's still not something you'd want to do in production.

Also, if the indexes are separate doesn't that mean that all lookups become more expensive because (apart from any index that includes the id) you now have to check every partition for matches on every query...

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/openstreetmap-website/issues/2076#issuecomment-442540007
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/rails-dev/attachments/20181128/12f1b9f6/attachment.html>


More information about the rails-dev mailing list