-
Notifications
You must be signed in to change notification settings - Fork 64
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Expand benchmarks to include all tests #216
Comments
Not all the tests should be (or can be) benchmarked though, right? We should probably include a bench + test utility but also allow raw tests. |
I think they all can be benchmarked. |
Some tests evaluate more than one program. Might require some reorg here (for the |
Well it's always either one or two. We could call the benches |
We can pull in the deterministic benchmarks into tests via golden testing (which also could cover the printTerm stuff, in principle). It also solves the problem of performance regressions being overlooked (as a result of no one bothering to consistently check the Hercules CI effects output for benchmark diff), because now the CI would fail and the benchmark goldens will have to be reviewed/updated by the PR author. cf. #123 |
This is only partly done; and will be completed as part of #302 |
We can simply change the testing implementation. See
examples/Utils.hs
.The text was updated successfully, but these errors were encountered: