Skip to content

Fix bump.yml workflow #491

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
qmonnet opened this issue May 16, 2025 · 7 comments · Fixed by #495
Closed

Fix bump.yml workflow #491

qmonnet opened this issue May 16, 2025 · 7 comments · Fixed by #495
Assignees
Labels
ci Continuous Integration

Comments

@qmonnet
Copy link
Member

qmonnet commented May 16, 2025

bump.yml has been failing for a couple days already.

The workflow fails to update the ahash dependency, currently pointing to our own fork.

Apparently, there's an --offline option that is passed somewhere when we run cargo upgrade, and that is the cause for this failure; but I haven't found where it comes from yet.

@qmonnet qmonnet added the ci Continuous Integration label May 16, 2025
@qmonnet
Copy link
Member Author

qmonnet commented May 19, 2025

We also need to fix the workflow with regards to the addition of the CLI as a separate binary, with some dependencies not handled in the toplevel Cargo.toml file:

$ cargo upgrade --incompatible=allow --dry-run 2>/dev/null
name     old req compatible latest new req
====     ======= ========== ====== =======
netdev   0.34.0  0.34.1     0.34.1 0.34.1 
smallvec 1.14.0  1.15.0     1.15.0 1.15.0 
name old req compatible latest new req
==== ======= ========== ====== =======
log  ^0.4.25 0.4.27     0.4.27 ^0.4.27

We get two files processed. I couldn't find a combination of options to prevent that. We should either move the dep declarations for the CLI to the toplevel Cargo.toml, or add some more post-processing to the output from the dry run.

@qmonnet
Copy link
Member Author

qmonnet commented May 19, 2025

#494 for fixing the above comment, but I don't expect it to fix the main issue.

@qmonnet
Copy link
Member Author

qmonnet commented May 19, 2025

I think cargo upgrade calls cargo update --offline to pull the Git dependencies: https://github.com/killercup/cargo-edit/blob/42e15faea5e99b630df5d77fc4f4b4ce70ed0fec/src/bin/upgrade/upgrade.rs#L549 but I'm not sure why, or why it fails, or why I can't reproduce locally.

@qmonnet
Copy link
Member Author

qmonnet commented May 19, 2025

I still don't understand what happens or what it expects, but maybe one solution could be to move the cargo update step, that we do at the end after all the cargo upgrades, to the first position might help solve it. It feels like the tool expect some local index to be updated before running in --online mode, but I suppose it's not the case.

runner@lab-z6r4v-runner-vrsvj:/tmp/dataplane$ cargo update --manifest-path Cargo.toml --offline ahash --verbose
error: Unable to update https://github.com/githedgehog/aHash?branch=pr%2Fdaniel-noland%2Fbump-zero-copy

Caused by:
  failed to lookup reference in preexisting repository, and can't check for updates in offline mode (--offline)

Caused by:
  failed to find branch `pr/daniel-noland/bump-zero-copy`

Caused by:
  cannot locate remote-tracking branch 'origin/pr/daniel-noland/bump-zero-copy'; class=Reference (4); code=NotFound (-3)
runner@lab-z6r4v-runner-vrsvj:/tmp/dataplane$ cargo update --manifest-path Cargo.toml ahash --verbose
    Updating git repository `https://github.com/githedgehog/aHash`
    Updating crates.io index
     Locking 0 packages to latest Rust 1.87.0 compatible versions
   Unchanged bitflags v2.9.0 (available: v2.9.1)
   Unchanged cc v1.2.22 (available: v1.2.23)
   Unchanged matchit v0.8.4 (available: v0.8.6)
   Unchanged netdev v0.34.0 (available: v0.34.1)
   Unchanged proc-macro-crate v2.0.0 (available: v2.0.2)
   Unchanged windows-core v0.61.0 (available: v0.61.2)
   Unchanged windows-result v0.3.2 (available: v0.3.4)
   Unchanged windows-strings v0.4.0 (available: v0.4.2)
note: to see how you depend on a package, run `cargo tree --invert --package <dep>@<ver>`
runner@lab-z6r4v-runner-vrsvj:/tmp/dataplane$ cargo update --manifest-path Cargo.toml --offline ahash --verbose
     Locking 0 packages to latest Rust 1.87.0 compatible versions
   Unchanged bitflags v2.9.0 (available: v2.9.1)
   Unchanged cc v1.2.22 (available: v1.2.23)
   Unchanged netdev v0.34.0 (available: v0.34.1)
note: to see how you depend on a package, run `cargo tree --invert --package <dep>@<ver>`

@qmonnet
Copy link
Member Author

qmonnet commented May 19, 2025

Local reproducer:

$ docker run --rm -it ubuntu:24.04
root@5d16f982a0ae:/# apt update
root@5d16f982a0ae:/# apt install --yes git rustup
root@5d16f982a0ae:/# rustup default stable
root@5d16f982a0ae:/# git clone https://github.com/githedgehog/dataplane.git
root@5d16f982a0ae:/# cd dataplane/
root@5d16f982a0ae:/dataplane# cargo update --offline ahash
error: Unable to update https://github.com/githedgehog/aHash?branch=pr%2Fdaniel-noland%2Fbump-zero-copy

Caused by:
  can't checkout from 'https://github.com/githedgehog/aHash': you are in the offline mode (--offline)

To run the wrapping cargo upgrade, add:

root@5d16f982a0ae:/# apt install --yes gcc
root@5d16f982a0ae:/# cargo install cargo-edit
root@5d16f982a0ae:/dataplane# cargo upgrade -p log
    Checking virtual workspace's dependencies
    Checking dataplane's dependencies
    Checking dataplane-cli's dependencies
name old req compatible latest new req
==== ======= ========== ====== =======
log  ^0.4.25 0.4.27     0.4.27 ^0.4.27
    Checking dataplane-dpdk's dependencies
    Checking dataplane-dpdk-sys's dependencies
    Checking dataplane-dpdk-sysroot-helper's dependencies
    Checking dataplane-errno's dependencies
    Checking dataplane-id's dependencies
    Checking dataplane-interface-manager's dependencies
    Checking dataplane-mgmt's dependencies
    Checking dataplane-net's dependencies
    Checking dataplane-pipeline's dependencies
    Checking dataplane-rekon's dependencies
    Checking dataplane-routing's dependencies
    Checking dataplane-vpc-manager's dependencies
   Upgrading git dependencies
error: Unable to update https://github.com/githedgehog/aHash?branch=pr%2Fdaniel-noland%2Fbump-zero-copy

Caused by:
  failed to lookup reference in preexisting repository, and can't check for updates in offline mode (--offline)

Caused by:
  failed to find branch `pr/daniel-noland/bump-zero-copy`

Caused by:
  cannot locate remote-tracking branch 'origin/pr/daniel-noland/bump-zero-copy'; class=Reference (4); code=NotFound (-3)
Error: recursive dependency update failed

Workaround: populate the local index

root@5d16f982a0ae:/dataplane# cargo update
root@5d16f982a0ae:/dataplane# git restore .
root@5d16f982a0ae:/dataplane# cargo update --offline ahash
     Locking 0 packages to latest Rust 1.87.0 compatible versions
note: pass `--verbose` to see 2 unchanged dependencies behind latest

@qmonnet
Copy link
Member Author

qmonnet commented May 19, 2025

killercup/cargo-edit#939

qmonnet added a commit that referenced this issue May 20, 2025
We have an issue in the bump.yml workflow. It fails to update recursive
dependencies for the project when these dependencies are passed as links
to Git repositories (for example, a branch name and an URL to a GitHub
repository).

I'm not sure what happens exactly, but there seems to be an issue with
the "cargo upgrade" command: it expects some local index to be populated
when running the upgrade for the Git-based dependency, and calls "cargo
update" under the hood with the "--offline" option, but this command
fails.

We've opened a GitHub issue for the upstream "cargo-edit" project
(providing "cargo upgrade"), but in the meantime, let's re-order the
commands we run: if we do the "cargo update" first, this should populate
whatever index "cargo upgrade" seems to rely on when we try to bump the
dependencies in Cargo.toml.

See #491 for more
context.

Fixes: #491
Signed-off-by: Quentin Monnet <[email protected]>
qmonnet added a commit that referenced this issue May 20, 2025
We have an issue in the bump.yml workflow. It fails to update recursive
dependencies for the project when these dependencies are passed as links
to Git repositories (for example, a branch name and an URL to a GitHub
repository).

I'm not sure what happens exactly, but there seems to be an issue with
the "cargo upgrade" command: it expects some local index to be populated
when running the upgrade for the Git-based dependency, and calls "cargo
update" under the hood with the "--offline" option, but this command
fails.

We've opened a GitHub issue for the upstream "cargo-edit" project
(providing "cargo upgrade"), but in the meantime, let's re-order the
commands we run: if we do the "cargo update" first, this should populate
whatever index "cargo upgrade" seems to rely on when we try to bump the
dependencies in Cargo.toml.

See #491 for more
context.

Fixes: #491
Signed-off-by: Quentin Monnet <[email protected]>
@qmonnet
Copy link
Member Author

qmonnet commented May 20, 2025

Fixed at last with #500, as confirmed with #501 generated from a manual workflow dispatch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci Continuous Integration
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants