[openstreetmap/openstreetmap-website] Partition large tables (#2076)

Paul Norman notifications at github.com
Thu Nov 29 14:02:00 UTC 2018

> Sure billion node partitions might reduce the time from a day to a few hours say, if there's enough I/O concurrency to get a more or less linear speedup, but it's still not something you'd want to do in production.

Unless you're saying that the tables can never be changed, we have to be able to have locks and read-only periods for maintenance. This lessens them.

> Also, if the indexes are separate doesn't that mean that all lookups become more expensive because (apart from any index that includes the id) you now have to check every partition for matches on every query...

Not really. My experience is that index size and performance remains about the same, unless there is a strong correlation between the partition criteria and the indexed value, in which case, it is faster. It's at most like a fraction of an extra node on the btree.

You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/rails-dev/attachments/20181129/d4a609db/attachment.html>

More information about the rails-dev mailing list