Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(config): add max-fail option to configuration file #2063

Merged
merged 4 commits into from
Jan 26, 2025

Conversation

Jayllyz
Copy link
Contributor

@Jayllyz Jayllyz commented Jan 11, 2025

Closes #1944

Copy link

codecov bot commented Jan 11, 2025

Codecov Report

Attention: Patch coverage is 97.63780% with 3 lines in your changes missing coverage. Please review.

Project coverage is 80.05%. Comparing base (c36c654) to head (229ab71).
Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
nextest-runner/src/config/max_fail.rs 97.54% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2063      +/-   ##
==========================================
+ Coverage   79.97%   80.05%   +0.08%     
==========================================
  Files         102      102              
  Lines       23997    24117     +120     
==========================================
+ Hits        19192    19308     +116     
- Misses       4805     4809       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Jayllyz Jayllyz force-pushed the feat/config-max-fail branch 2 times, most recently from f1bbf32 to 55932a2 Compare January 11, 2025 17:23
Copy link
Member

@sunshowers sunshowers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR!

Comment on lines 69 to 70
# "max-fail" defines the maximum number of test failures that are allowed before test execution is stopped.
max-fail = 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I think this should actually be part of the fail-fast field. This field currently supports true or false, but it should also support

fail-fast = { max-fail = 1 }

This addresses issues like precedence, and matches the CLI behavior where --fail-fast and --max-fail=1 cannot be specified together.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, Im going to try to do it

@@ -83,4 +130,126 @@ mod tests {
MaxFail::from_str(input).expect_err(&format!("expected input '{input}' to fail"));
}
}

#[test_case(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests are fantastic, thank you!

@Jayllyz
Copy link
Contributor Author

Jayllyz commented Jan 17, 2025

Tested the config on one of my rust repo, it's working good.

Copy link
Member

@sunshowers sunshowers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thank you! I have a couple minor changes I'll take care of.

nextest-runner/src/config/fail_fast.rs Outdated Show resolved Hide resolved
@sunshowers sunshowers force-pushed the feat/config-max-fail branch from 7a022a9 to d3f1eff Compare January 25, 2025 23:51
@sunshowers
Copy link
Member

I've combined max-fail and fail-fast into a single struct, and added tests for error messages (i.e. not just that there are errors).

@sunshowers sunshowers force-pushed the feat/config-max-fail branch from d3f1eff to 18946d0 Compare January 25, 2025 23:57
@sunshowers sunshowers force-pushed the feat/config-max-fail branch from 18946d0 to 229ab71 Compare January 26, 2025 00:02
@sunshowers sunshowers merged commit 7864d98 into nextest-rs:main Jan 26, 2025
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add configuration for max-fail
2 participants