Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

functions to get number of tests, passes, etc. #3

Open
nickrobinson251 opened this issue May 18, 2023 · 5 comments
Open

functions to get number of tests, passes, etc. #3

nickrobinson251 opened this issue May 18, 2023 · 5 comments

Comments

@nickrobinson251
Copy link
Contributor

nickrobinson251 commented May 18, 2023

I found this useful for ReTestItems (e.g. https://github.com/JuliaTesting/ReTestItems.jl/blob/main/test/_integration_test_tools.jl#59), to tests the specifics of what happened when running a testset with multiple tests (similar to #2)

for failures and errors, this is just calling length on #2
but for passes we need to track that separately, since DefaultTestSet doesn't store Pass results (just a count of pass)

@nickrobinson251
Copy link
Contributor Author

and related all*/any* helpers (any_errors(ts) -> Bool, all_passed(ts) -> Bool)

I've seen these kinds of helpers defined e.g. at https://github.com/RelationalAI/rai-test-julia/blob/a317c89c40472996a96bf4f6569029d40abf454f/src/testsets.jl#L105-L115 as well as at https://github.com/JuliaTesting/ReTestItems.jl/blob/498097bd04a9f890b8623c0d0857ab5dd534890d/test/_integration_test_tools.jl#L91

@nickrobinson251 nickrobinson251 changed the title function to get number of tests, passes, etc. functions to get number of tests, passes, etc. May 18, 2023
@sethaxen
Copy link
Collaborator

@nickrobinson251 could you put together a PR(s) with this and #2?

@nickrobinson251
Copy link
Contributor Author

Yes, probably later next week

@sethaxen
Copy link
Collaborator

Great, thanks! In that case I think it makes sense to register the package now, hook it into ChainRulesTestUtils' test suite, and then make a breaking release if your PRs make breaking changes.

@nickrobinson251
Copy link
Contributor Author

yeah, that makes sense to me - thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants