[openstreetmap/openstreetmap-website] Future of db/functions? (#3110)

Andy Allan notifications at github.com
Wed Mar 3 12:17:18 UTC 2021


> Before creating a pull request for this topic, I'd like to get some early feedback, if this really makes sense.

I'm always in favour of cleanup, and removing complexity in particular :smile:

>     * People might still want to try out osmosis replication in their dev environment. Is this in scope for this repo?

We should push the concept of setting up replication out of this repo. I think the best approach is for osmosis to ship the functions necessary for osmosis-based replication, and osmdbt to ship whatever it needs too. We should update CONFIGURE.md to indicate there are two options to consider (osmosis and osmdbt) depending on your PostgreSQL verison and point people to the right place, but otherwise remove ourselves from that.

>     * do we still need those old db functions to run old migrations?

This is the core of the discussion, I think. We need to keep the db functions for as long as they are used in the old migrations.

We could refactor the old migrations, to remove the functions. But this raises the question of what the old migrations are there for? Two possibilities:

* They a there to help people who have a (really old) database with data in it. So we need to keep the old migrations and the functions too.
* They are there to help people who have an empty database. In which case we don't need the old migrations at all, since it's much easier (and faster) to load the structure.sql than to run through 109 migrations.

I believe we should just ditch the old migrations, say every migration over 2 years old. If someone needs them (if they have a really old database dump, or they only update their code every 3 years) then they are in source control anyway, just like our old unused code is. We can keep the reasonably new migrations to help other developers and deployments who don't update every month.

The other alternative is to keep the old migrations, but continue to refactor them over the years, either for things like this, or rubocop, or to work with new database systems, or whatever. But I think this is just makework, like keeping old code in comments or like maintaining amf_controller even after we remove it from the API. The code in migrations is old, it has done its job, it's their in the git history if anyone needs it.

I think removing the old migrations will mean we need some tweaks to CI and documentation, e.g. to load empty databases from the structure.sql (and use commands like db:structure:load) instead of migrations. There is also the 'double check' that we do in CI where we test that the sum of the migrations is equal to the structure.sql, so we need an alternative (e.g. a check that old_structure.sql + migrations == structure.sql).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/openstreetmap-website/issues/3110#issuecomment-789673542
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/rails-dev/attachments/20210303/e8665dca/attachment.htm>


More information about the rails-dev mailing list