[openstreetmap/openstreetmap-website] Partition large tables (#2076)

[Paul Norman]

> Partition them on what?

ID for the OSM data tables. I'm not sure about GPX.

> I don't see how it helps with maintenance anyway

It avoids doing it one big chunk that requires a long lock. Even for operations that don't require a lock, it's easier to deal with a reasonable number of shorter operations than one very long operation.

The bloat reasons are because most of the `INSERT` and `UPDATE` activity will be in one partition.

It speeds up pg_dump and pg_restore because these are bottlenecked on the largest table.

>if a column needs to be rewritten then it needs to be rewritten and whether the table is in one chunk or a million chunks there is still the same amount of data to rewrite.

Schema changes requiring a rewrite aren't the primary reason everyone is suggesting this, but they will be faster because we can use the parallelism of the server, rather than be stuck single-threaded.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/openstreetmap-website/issues/2076#issuecomment-442530738
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/rails-dev/attachments/20181128/8c7fd216/attachment.html>