PR Benchmark Diffing #144
Merged
PR Benchmark Diffing #144
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Please retarget to staging... |
MatthewCroughan
force-pushed
the
mc/benchmark-diff
branch
from
2b6da3d to
5250e1f
Compare
|
Is there any issue on GH for the bug? |
|
@L-as No, I'm in the process of making one. |
MatthewCroughan
force-pushed
the
mc/benchmark-diff
branch
from
5250e1f to
4f3ab1f
Compare
|
Turns out there was no bug, it was just a missing |
Hercules CI was not configured optimally for this repo. I've renamed nixCi to ciNix and configured it to be consistent with the Hercules CI template, documentation and our other repos. It is no longer an attribute that can be evaluated, instead Hercules CI is supposed to simply build all of our existing flake attribute outputs, instead of being a set that contains all of them recursively.
MatthewCroughan
force-pushed
the
mc/benchmark-diff
branch
2 times, most recently
from
9a7a2d2 to
6f059de
Compare
srid
approved these changes
Jan 19, 2022
MatthewCroughan
force-pushed
the
mc/benchmark-diff
branch
from
6f059de to
9eb0add
Compare
This adds a new Hercules CI effect which takes a pull request and performs a diff on the benchmark by using `nix run .#benchmark` on the HEAD of the PR, and on the merge-base of that PR, which effectively gives us a performance measurement before the PR and after the PR.
MatthewCroughan
force-pushed
the
mc/benchmark-diff
branch
from
9eb0add to
5735d70
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds a new
effectvia https://github.com/hercules-ci/hercules-ci-effects which takes a pull request and performs a diff on the benchmark by usingnix run .#benchmarkon the PRHEADand on the merge-base of that PR.