-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New version of vscode doesn't allow pytest plugin proper execution #23323
Comments
I had the same issue and reverted back to the previous version to get my tests running again. |
In my project I use functions to generate tests, and pytest detects them properly. However when I run them from the vscode extension, the number of tests discovered gets multiplied by 50 (600 -> 3000) and it hangs forever at around 600. The should be reproducible by cloning the branch |
I found where the issue comes from in my case. If you want to test if your issue is also coming from those pytest arguments, you can go to PYTHONPATH/vscode_pytest/run_pytest_script.py, you should have a line that looks like:
remove "+ test_ids_from_buffer" and test again. this will remove the test id argument and if you have the same issue it should run all the tests. @LiteApplication I tried to reproduce your issue, but I generate 576 tests, that is what is expected right? |
That's expected indeed @PicouAymeric, I'll try your fix to see if it resolves the issue |
BTW I would be curious to know why the devs chose to change the action behind the run button. |
After some more testing, I found that the bug only crashes the window when "View as tree" is selected, not when "View as list" is. After checking again 576 is the exact number of tests needed, and they all pass with @PicouAymeric's fix. |
Hi everyone- catching up on this now as I was on vacation. Thanks @PicouAymeric We have always passed specific test ids as args- do you have a different version where you think it didn't / which one is this? Are you specifically talking about using the "run" button at the top of the test explorer to run all tests or are you running specific subsections of tests? Also @LiteApplication what do you mean by "running the tests the count does go up to 3k", where are you seeing this number? I will try and repro now and let you know what I find- thanks! |
@eleanorjboyd; I didn't check the version of my Vscode before update, and I don't find any log with that information.
When I click on run on subtest:
When I click on run on test_module.py:
Finally when I click on run on test_function I have:
With the new version of vscode / python extension, this would have been something like:
|
Update: I reinstalled vscode 1.89 and I have the same behaviour, so it seems that it depends only on the Python extension version. |
@PicouAymeric or @FrankCRoth could you send a minimal repro of when you are seeing this problem?
in reference to this, what are your args in your settings? And all of these come from the newer broken extension right, and is this a python print statement (so after we append the testIds) or from the extension before the subprocess gets run? Generally I think some logs would help since that could show me exactly where the problem is happening- could someone add theirs? To do so first set your log level to trace via the |
Hi, Sorry, I just got back from vacation.
|
@PicouAymeric, the issue is that previously on the 2023.8.0 extension was pre-rewrite while now it is post-rewrite and we have changed the entire design of how testing works. To go back to the root of the problem, it doesn't work for the test ids to be passed on your instance because of generated tests by pytest hooks? I can investigate this given the new redesign and consider how we might be able to handle this case within our current design. |
I edited class DeferPlugin:
@pytest.hookimpl(hookwrapper=True)
def pytest_xdist_auto_num_workers(self, config: pytest.Config) -> Generator[None, int, int]:
"""determine how many workers to use based on how many tests were selected in the test explorer"""
return min((yield), len(config.option.file_or_dir)) e.g. this change that renames hookwrapper to wrapper and requires pluggy >= 1.2 pytest-dev/pytest@b41acae I'm using pytest 7; pytest 8 requires the newer pluggy and |
ah yes this would be the same problem as #23707. Yes updating pluggy is the best strategy as the 1.1.0 version was pulled due to being incompatible. We have release some other changes to help with backwards compatibility which will be out on the newest version of python extension (pre-release) tomorrow. Thanks |
After updating vscode some pytest hook functions that were running well before update are not called anymore:
In my case those function are:
Yet I am sure my plugin is loaded since some of its hooks are called without issues:
Executing with pytest.main works perfectly fine.
I can see that behaviour whether I run test collection, run test or run test in debug mode.
Probably the vscode_pytest plugin prevent my functions from being called somehow.
Does anyone has an idea why and how to work it around?
The text was updated successfully, but these errors were encountered: