Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

workflows/build: init #368690

Closed
wants to merge 1 commit into from
Closed

Conversation

getchoo
Copy link
Member

@getchoo getchoo commented Dec 28, 2024

Part of #355847

With the end of OfBorg nigh, it's important to keep at least some automated builds going. So, this is meant to replace OfBorg's automatic building of attributes based on the prefix of commit messages

You can find a PR with this in action at getchoo-contrib#15

Some things to still do now and in the future:

  • Run passthru.tests
  • Set a neutral check for failed `nix-instantiate' invocations (I can't find a nice way to do this outside of the Node API :/)
    • Currently they are marked as a success, which implies a build 👎
  • Support comment invocations? (i.e., @ofborg build/test)
    • Might be better for a different workflow, as that would have to look for the invocation, parse, escape, handle multiple lines of it, etc etc.
  • Re-use the evaluation workflow artifacts?
    • Running this after the evaluation workflow would be ideal, as there's not much point in building when eval is broken (at least IMO)
    • We currently re-evaluate (what we assume are) the top-level packages changed, which is a bit wasteful
    • Main problem here is that we have no way of differentiating the packages directly changed by the PR from their dependents...and I don't think we want to be basically running nixpkgs-review for every PR 😆

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added 6.topic: policy discussion 6.topic: continuous integration Affects continuous integration (CI) in Nixpkgs, including Ofborg and GitHub Actions labels Dec 28, 2024
@github-actions github-actions bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Dec 28, 2024
@GaetanLepage
Copy link
Contributor

Thanks @getchoo for working on this !

I implemented the "comparison" logic directly in nixpkgs (see ci/eval/compare/).
This is actually run by a GitHub action for each PR. Also, nixpkgs-review now tries to fetch those results to know the "rebuilds" (i.e. affected packages).
Why would you need to re-compute anything ?

@infinisil
Copy link
Member

I don't think we should do this, because builds of attributes can be very resource-intensive, which is why we said that only evaluation should be done by GHA in https://discourse.nixos.org/t/equinix-metal-cta-infra-short-update/56538

@Mic92
Copy link
Member

Mic92 commented Dec 31, 2024

I did some experiments to run the CI in the github fork instead of the NixOS org.
Problem is reporting though: https://github.com/Mic92/nixpkgs/blob/main/.github/workflows/build.yml

Otherwise I would also agree that we shouldn't run this in the nixpkgs org since those rebuilds could easily block evaluation runs.

@getchoo
Copy link
Member Author

getchoo commented Jan 1, 2025

Why would you need to re-compute anything ?

The implementation in ci/eval/compare doesn't differentiate between dependencies being rebuilt and paths being directly changed in a PR. As I said in the initial comment

I don't think we want to be basically running nixpkgs-review for every PR

as OfBorg never did that either. I get why we still can't do this though, since

builds of attributes can be very resource-intensive

is totally true, especially when it's not on our own infra anymore. I hadn't considered it blocking evaluations either :(

I did some experiments to run the CI in the github fork instead of the NixOS org.
Problem is reporting though: https://github.com/Mic92/nixpkgs/blob/main/.github/workflows/build.yml

This seems like a nice idea. I'd love to follow up on Matrix about it sometime soon!

In any case, thanks to everyone for their time. Seems this isn't the solution

@getchoo getchoo closed this Jan 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: continuous integration Affects continuous integration (CI) in Nixpkgs, including Ofborg and GitHub Actions 6.topic: policy discussion 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants