Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mini-SBIBM #1325

Closed
manuelgloeckler opened this issue Dec 10, 2024 · 1 comment
Closed

Mini-SBIBM #1325

manuelgloeckler opened this issue Dec 10, 2024 · 1 comment
Labels
enhancement New feature or request hackathon improving-tests Everything to do with the sbi test suite

Comments

@manuelgloeckler
Copy link
Contributor

Is your feature request related to a problem? Please describe.
The SBI test suite only tests "general correctness" on rather simple tasks with analytic solutions (gaussian).

It would be very helpful to have a fast but more elaborate mini SBIBM when implementing new methods or adapting implemented ones. Simply to ensure that changes do not have an unintended impact on how well these methods work.

In contrast to the SBIBM, this should have minimal dependencies and, at best, be integrated within the pytest (probably optionally callable with flags).

Describe the solution you'd like
I think the best solution is to implement it as an additional test.
We can use pytest-harvest to collect and show results from these tests after the pytest has run.
These should likely not assert, but should highlight if performance got better/worse.

@manuelgloeckler manuelgloeckler added enhancement New feature or request hackathon labels Dec 10, 2024
@gmoss13 gmoss13 added the improving-tests Everything to do with the sbi test suite label Mar 12, 2025
@janfb
Copy link
Contributor

janfb commented Mar 13, 2025

solved with #1335

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request hackathon improving-tests Everything to do with the sbi test suite
Projects
None yet
Development

No branches or pull requests

3 participants