Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch test runner from nosetest to pytest #6291

Merged
merged 42 commits into from
Dec 19, 2024
Merged

Switch test runner from nosetest to pytest #6291

merged 42 commits into from
Dec 19, 2024

Conversation

cognifloyd
Copy link
Member

@cognifloyd cognifloyd commented Dec 18, 2024

This cherry-picks commits from #6236 from @nzlosh, @FileMagic, and @guzzijones.

This adjust the Makefile and st2-run-pack-tests to use pytest instead of nosetest. We need to switch because nosetest is unmaintained and does not support python 3.10. Goodbye nosetest! 🎉

Several of the tests were adjusted to:

  • use pytest.raises instead of assertRaisesRegex, and
  • use pytest.mark.skipif instead of unittest.skipIf.

Finally, this drops nose (and plugins) from requirements and regenerates lockfiles/st2.lock. I also copied the updated requirement pins from the lockfile into fixed-requirements.txt and related files.

Lockfile diff: lockfiles/st2.lock [st2]

==                    Upgraded dependencies                     ==

  amqp                           5.2.0        -->   5.3.1
  apscheduler                    3.10.4       -->   3.11.0
  argcomplete                    3.5.1        -->   3.5.2
  async-timeout                  4.0.3        -->   5.0.1
  attrs                          24.2.0       -->   24.3.0
  bcrypt                         4.2.0        -->   4.2.1
  certifi                        2024.8.30    -->   2024.12.14
  ciso8601                       2.3.1        -->   2.3.2
  eventlet                       0.37.0       -->   0.38.2
  orjson                         3.10.10      -->   3.10.12
  packaging                      24.1         -->   24.2
  pip                            24.2         -->   24.3.1
  pyspnego                       0.11.1       -->   0.11.2
  python-json-logger             2.0.7        -->   3.2.1
  redis                          5.2.0        -->   5.2.1
  setuptools                     75.2.0       -->   75.3.0
  six                            1.16.0       -->   1.17.0
  tomli                          2.0.2        -->   2.2.1
  virtualenv                     20.27.0      -->   20.28.0
  werkzeug                       3.0.4        -->   3.0.6
  wheel                          0.44.0       -->   0.45.1
  wrapt                          1.16.0       -->   1.17.0

==                     Removed dependencies                     ==

  colorama                       0.4.6
  nose                           1.3.7
  nose-parallel                  0.4.0
  nose-timer                     1.0.1
  rednose                        1.3.0
  termstyle                      0.1.11

A follow-up PR will cherry-pick changes to finally start testing with python 3.10 +.

@cognifloyd cognifloyd added this to the 3.9.0 milestone Dec 18, 2024
@pull-request-size pull-request-size bot added the size/XXL PR that changes 1000+ lines. You should absolutely split your PR into several. label Dec 18, 2024
@cognifloyd cognifloyd self-assigned this Dec 19, 2024
Lockfile diff: lockfiles/st2.lock [st2]

==                    Upgraded dependencies                     ==

  amqp                           5.2.0        -->   5.3.1
  apscheduler                    3.10.4       -->   3.11.0
  argcomplete                    3.5.1        -->   3.5.2
  async-timeout                  4.0.3        -->   5.0.1
  attrs                          24.2.0       -->   24.3.0
  bcrypt                         4.2.0        -->   4.2.1
  certifi                        2024.8.30    -->   2024.12.14
  ciso8601                       2.3.1        -->   2.3.2
  eventlet                       0.37.0       -->   0.38.2
  orjson                         3.10.10      -->   3.10.12
  packaging                      24.1         -->   24.2
  pip                            24.2         -->   24.3.1
  pyspnego                       0.11.1       -->   0.11.2
  python-json-logger             2.0.7        -->   3.2.1
  redis                          5.2.0        -->   5.2.1
  setuptools                     75.2.0       -->   75.3.0
  six                            1.16.0       -->   1.17.0
  tomli                          2.0.2        -->   2.2.1
  virtualenv                     20.27.0      -->   20.28.0
  werkzeug                       3.0.4        -->   3.0.6
  wheel                          0.44.0       -->   0.45.1
  wrapt                          1.16.0       -->   1.17.0

==                     Removed dependencies                     ==

  colorama                       0.4.6
  nose                           1.3.7
  nose-parallel                  0.4.0
  nose-timer                     1.0.1
  rednose                        1.3.0
  termstyle                      0.1.11
only for Makefile-based pytest runs
This is important for the directories where we don't yet have tests.
This is only relevant for when the Makefile runs pytest, because pants
only runs pytest on files that exist, but the Makefile
blindly tries to run on every component directory.
@cognifloyd
Copy link
Member Author

Well that's disappointing. I added pytest-shard for Makefile-based pytest runs because doing only one shard made it really slow. But, now there are 2 nonsensical test failures.

Oddly enough pytest-shard shards per test instead of per file, so tests in the same file are split between shards. When pants does test sharding, it does so at the file level.

For the 2 failures, the AttributeError happens in only those 2 tests. The rest of the tests in that file (and in the same unittest class) all pass in both shards even though many of them use the same model and the same attribute.

What could cause this behavior?!

@cognifloyd
Copy link
Member Author

I inlined the pytest-test-group plugin and modified it to shard the files instead of the tests, which is basically what pants does. So, now we aren't getting those weird failures when tests in the same file get run in different shards/chunks.

@cognifloyd cognifloyd requested a review from a team December 19, 2024 20:11
@cognifloyd cognifloyd merged commit 983f63f into master Dec 19, 2024
48 checks passed
@cognifloyd cognifloyd deleted the pytest branch December 19, 2024 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement external dependency infrastructure: ci/cd size/XXL PR that changes 1000+ lines. You should absolutely split your PR into several. tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants