Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider cleaning up the .git folder to reduce the large repo size #160

Open
NTBBloodbath opened this issue May 21, 2021 · 6 comments
Open

Comments

@NTBBloodbath
Copy link
Contributor

NTBBloodbath commented May 21, 2021

Hey, I've had to clone the repository again, and have found that it's a bit heavy (~12MB) even to download and can cause it to fail on slow connections with the default packer settings (60s timeout).

After doing a little search I have found that it's a nvim.appimage.1 file that doesn't exist and that it's stored in the .pack file.

Here are the steps I took and how to reduce the repository size locally by up to 90% (1.2MB), all steps must be done from the repository root:

  • To see the 10 biggest files:
git verify-pack -v .git/objects/pack/pack-26ca0b79e92d096b8ce4eb32e16238ca74461801.idx \
| sort -k 3 -n \
| tail -10
  • To see what each file is (the important and bigger one is 943f15):
git rev-list --objects --all | grep 'first few chars of the sha1 from previous output'
  • Clean up your git by removing all of those unnecessary files:
git filter-branch --index-filter 'git rm --cached --ignore-unmatch nvim.appimage.1' -- --all
rm -Rf .git/refs/original
rm -Rf .git/logs/
git gc --aggressive --prune=now
  • Verify your size-pack, should be a lot smaller now:
git count-objects -v
@tjdevries
Copy link
Member

Is there a way I can do this for other people? I just ran these locally but I don't think it did anything for pushing... haha I'm not an expert in this area of git

@NTBBloodbath
Copy link
Contributor Author

NTBBloodbath commented Jun 17, 2021

Yes, after cleaning it should be enough to do git push origin --force -all.

Edit: it will rewrite all the commits because of the git filter-branch command, but I think it's a better approach than creating a new branch to start again and lose the git commits logs.

@clason
Copy link
Contributor

clason commented Aug 13, 2021

Just a small data point: the slow download is in fact an issue with Telescope's minimal.lua, since it needs to bootstrap plenary (which it does async, so there's a good chance the user will run into problems if they miss the "ready" prompt...)

@clason
Copy link
Contributor

clason commented Aug 13, 2021

Maybe https://github.com/newren/git-filter-repo is a better approach?

  1. pip3 install git-filter-repo
  2. git filter-repo --strip-blobs-bigger-than 1M
  3. git push origin --force

This will still rewrite history, of course, breaking current clones (and PRs), though.

@clason
Copy link
Contributor

clason commented Aug 13, 2021

OK, I dug a little deeper. The issue is that packer clones with --no-single-branch, which pulls in metadata for all current branches -- some of which apparently still have the appimage in them. Pruning the offending branches should be enough to reduce the footprint of the clones.

@clason
Copy link
Contributor

clason commented Aug 13, 2021

And it's (only) the path branch, to be precise. (Still a good idea to delete the other closed/merged branches as well.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants