[openstreetmap/openstreetmap-website] How can we reduce the size of this repository? (Issue #3849)

Andy Allan notifications at github.com
Fri Dec 16 17:22:25 UTC 2022


### Problem

Or at least, reduce the speed at which it grows? On my development machine, the git repo now takes up over 400MB, but less than 10% of that is code or assets that we manage here. From time to time this causes a few problems - slow clone speeds for most developers (e.g. those who want `git blame` to work, and can't work with just a shallow clone), and I've run into the issue when using this repository for things like Heroku where you might need to upload the entire repo to a remote host.

### Description

The most noticeable source of repo size, to me, is a combination of historical versions of potlatch2 and iD. I'm not sure what the reasons are or were, but release versions of both editors are/were copied directly into this repository. This means we have lots of versions of the potlatch2.swf binaries (around 1 - 1.5MB each), compiled versions of iD (largest of these is 12MB, but there are dozens and dozens in the 2+MB range) and even an old Potlatch2 mp4 help video (9.5MB). All this adds up, and the repo keeps growing e.g. every time iD is released.

Note that this isn't about the size of the latest checkout, which is mostly OK, it's the accumulation of all the historical versions that is slowly adding up.

There's probably two things worth considering:
* How do we handle vendored assets from this point on? Can we use our dependency managers (e.g. yarn) to handle the distribution of things like iD, without impact on the development experience? I don't want to require additional steps (e.g. "download a tgz from here and untar it into some directory") for getting up and running.
* What do we do about the stuff already in the git history? For some things, like rebuilding the repository without the P2 video, that would leave old checkouts in a partially not-functioning state. But do we need to worry about that? Or is there an easy way to handle it, perhaps some git magic to move large files out of old commits and into a different but linked repo?

### Screenshots

_No response_

-- 
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/openstreetmap-website/issues/3849
You are receiving this because you are subscribed to this thread.

Message ID: <openstreetmap/openstreetmap-website/issues/3849 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/rails-dev/attachments/20221216/b8cc29ac/attachment-0001.htm>


More information about the rails-dev mailing list