Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't do shallow clones if upstream does not support this #1678

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

cmuellner
Copy link
Collaborator

Since we changes our cloning strategy to do shallow clones only, all CI builds broke because the upstream servers don't allow us to clone specific commits. The error reads like this:

error: Server does not allow request for unadvertised object beb2cdbcda911764b2bed5e57921fe90493260bd
fatal: Fetched in submodule path 'binutils', but it did not contain beb2cdbcda911764b2bed5e57921fe90493260bd. Direct fetching of that commit failed.

See also: https://github.com/riscv-collab/riscv-gnu-toolchain/actions/runs/13434377368/job/37533209677?pr=1677

What we could do is to search for our target commit in remote branches, fetch those and then checkout our target commit. See also the following comment for an idea how this could be realized:
#1654 (comment)

However, it feels wrong to do add such scripts downstream. We need to find a better solution. But unless we have one, let's just disable shallow clones to unbreak the CI builds.

Since we changes our cloning strategy to do shallow clones only,
all CI builds broke because the upstream servers don't allow us
to clone specific commits. The error reads like this:

  error: Server does not allow request for unadvertised object
  beb2cdbcda911764b2bed5e57921fe90493260bd
  fatal: Fetched in submodule path 'binutils', but it did not
  contain beb2cdbcda911764b2bed5e57921fe90493260bd. Direct fetching
  of that commit failed.

What we could do is to search for our target commit in remote branches,
fetch those and then checkout our target commit. See also the following
comment for an idea how this could be realized:
  https://github.com/riscv-collab/riscv-gnu-toolchain/
  issues/1654#issuecomment-2591195261

However, it feels wrong to do add such scripts downstream.
We need to find a better solution. But unless we have one,
let's just disable shallow clones to unbreak the CI builds.

Signed-off-by: Christoph Müllner <[email protected]>
@TommyMurphyTM1234
Copy link
Collaborator

@cmuellner
Copy link
Collaborator Author

What I still don't know is:

  • Is this a server configuration issue or a fundamental issue with shallow clones?
  • How do other people deal with this issue (we are certainly not the first to run into this)?

@TommyMurphyTM1234
Copy link
Collaborator

What I still don't know is:

  • Is this a server configuration issue or a fundamental issue with shallow clones?
  • How do other people deal with this issue (we are certainly not the first to run into this)?

Doesn't @mickflemm explain the issue clearly here?

@kito-cheng
Copy link
Collaborator

Most GNU toolchain are hosted in https://sourceware.org/git/ but why only binutils need set that to false?

@pz9115
Copy link
Contributor

pz9115 commented Feb 21, 2025

I checked the beginning of this topic, seems we are missing a script to help if shallow clone failed

Since #1616 not merge and efc7cc9 not deal this case.

@cmuellner
Copy link
Collaborator Author

cmuellner commented Feb 21, 2025

Most GNU toolchain are hosted in https://sourceware.org/git/ but why only binutils need set that to false?

Sorry, I should have tagged the PR with RFC.
I only set Binutils because I have evidence that this repo causes issues.
However, since GCC is also affected, not using shallow clones will immediately bring back the CI storage space issue.

So, the only path forward seems indeed to be a downstream clone script (with inspiration from #1616 and #1654 (comment)). Volunteers are welcome!

@cmuellner cmuellner marked this pull request as draft February 21, 2025 09:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants