-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[RFC] unittests: allow running with pytest/unittest directly #13423
base: master
Are you sure you want to change the base?
Conversation
Currently the unittests are not runnable with pytest or unittest without going through the run_unittests.py wrapper. This has that downside that the common "pytest ..." fails and integration with things like VSCode fails too. To work around that we set everything that is needed to run the tests in __init__.py and run_unittests is only one more variant to invoke them by providing different defaults and settings. To make sure that pytest/unittest discover and run_unittests don't diverge implement an automatic test discovery in run_unittests to avoid hardcoding the tests to run there. There shouldn't be any functional changes.
A similar thing could even be done for project tests, in theory, if you dynamically create them and wrap them in a test class: def create_test_case(suffix):
class DynamicTestCase(unittest.TestCase):
def test_example(self):
assert 1 == 1
DynamicTestCase.__name__ = DynamicTestCase.__name__ + suffix
globals()[DynamicTestCase.__name__] = DynamicTestCase
create_test_case("ProjectOne")
create_test_case("ProjectTwo") Which would make it easy to discover tests, run single tests, or even parts of some project in case it is split into different phases. Configuration could be done via env vars, maybe even a dotenv file of some sorts if that makes things easier. |
I'm confused, I always run with just edit: We have the configuration in |
I can't reproduce. Calling pytest gives me am I missing something? |
Ah, yes. looking it at I did put that environment variable in my nix shell configuration. I guess that means all we need to do is actually ensure that the |
Yes, plus the other things I added to I've added some CLI examples for how to run the tests in the original post. |
Yeah, I don't need any of that. |
def init(): | ||
setup_vsenv() | ||
unset_envs() | ||
set_envs() | ||
|
||
init() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't really see a point in having an init() function here? Any particular reason for doing it this way?
My guess would be that it's imitating the use of main() in run_unittests.py but there's a comparatively specific reason why we do that there, and it isn't applicable to __init__.py
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's just my instinct of not calling non-pure functions in the global scope at import time, and limiting it to one place. No other reason.
With pytest it could be done in conftest.py, but that's not a thing with unittest.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is python, not haskell, so I think we are fine on the non-pure functions thing.
Currently the unittests are not runnable with pytest or unittest without going through the run_unittests.py wrapper.
This has that downside that the common "pytest ..." fails and integration with things like VSCode fails too.
To work around that we set everything that is needed to run the tests in __init__.py and run_unittests is only one more variant to invoke them by providing different defaults and settings.
To make sure that pytest/unittest discover and run_unittests don't diverge implement an automatic test discovery in run_unittests to avoid hardcoding the tests to run there. There shouldn't be any functional changes.
All tests can be run the following way with this:
pytest unittests
(new)python3 -m unittest discover unittests '*tests.py' .
(new)./run_unittests.py
Single tests can be run the following way with this:
pytest -k TAPParserTests
(new)python3 -m unittest unittests/taptests.py
(new)./run_unittests.py TAPParserTests