feat - use fixture plugins instead of conftest files

[diitaz93]

Description

Fix #2095 and continue #2259.

TL;DR

The main conftest.py was too crowded, now the fixtures are separated into different modules available to all tests.

Details

The current test system looks for fixtures defined in the conftest.py files, either in the local directory or in any directory above, but it is impossible to use a fixture defined in a directory below it. For example, a test function in test_module_top.py can't access a fixture defined in tests/subfolder/subsubfolder/conftest.py:

tests
├── conftest.py
├── subfolder
│   ├── sub_subfolder
│   │   ├── __init__.py
│   │   ├── conftest.py
│   │   └── test_module_sub.py
└── test_module_top.py

To solve this, we started writing all our fixtures in the first contest, i.e. tests/conftest.py. This made this file extremely crowded and difficult to navigate.

This PR changes the conftest structure for a plugin structure, in which the fixtures are grouped by topic in a folder called fixture_plugins and imported in the conftest as a plugin. In this way, the structure of the tests is

tests
├── conftest.py
├── fixture_plugins
│   ├── fixture_subfolder
│   │   ├── fixtures_topic_1.py
│   │   └── __init__.py
│   ├── fixtures_topic_2.py
│   └── __init__.py
├── test_subfolder
│   ├── __init__.py
│   └── test_module_sub.py
└── test_module_top.py

and both test_module_top.py and test_module_sub.py have access to all the fixtures defined in fixture_plugins.

Added

• Fixture plugin modules with fixtures in the new folder tests/fixture_plugins/ with the following structure:

tests
├── conftest.py
├── fixture_plugins
│   ├── demultiplex_fixtures
│   │   ├── __init__.py
│   │   ├── flow_cell_fixtures.py
│   │   ├── name_fixtures.py
│   │   ├── path_fixtures.py
│   │   ├── run_parameters_fixtures.py
│   │   └── sample_fixtures.py
│   ├── __init__.py
│   └── timestamp_fixtures.py

Changed

• Moved demultiplexing and timestamp fixtures from conftest.py to separate modules in the new fixture plugin directory.

How to prepare for test

• Ssh to relevant server (depending on type of change)
• Use stage: us
• Paxa the environment: paxa
• Install on stage (example for Hasta):

  bash /home/proj/production/servers/resources/hasta.scilifelab.se/update-tool-stage.sh -e S_cg -t cg -b [THIS-BRANCH-NAME] -a

How to test

• Do ...

Expected test outcome

• Check that ...
• Take a screenshot and attach or copy/paste the output.

Review

• Tests executed by
• "Merge and deploy" approved by
  Thanks for filling in who performed the code review and the test!

This version is a

• MAJOR - when you make incompatible API changes
• MINOR - when you add functionality in a backwards compatible manner
• PATCH - when you make backwards compatible bug fixes or documentation/instructions

Implementation Plan

• Document in ...
• Deploy this branch on ...
• Inform to ...

[seallard]

Nice initiative! Is this a common pattern? And can we still rely on pytests "auto" import of fixtures?

[ChrOertlin]

It looks nicely structured and seems like to improve the ability to lookup fixtures more easily. It would be great to expand this ot other modules. If we can retain all other functionality, maybe the tests/conftest.py will end up containing only imports.

[islean]

I love it! As long as the auto imports work, I love it.

[henrikstranneheim]

Looks like a great improvement!

[diitaz93]

Nice initiative! Is this a common pattern? And can we still rely on pytests "auto" import of fixtures?

I know requesting plugins in the root contest is a common approach, but honestly I am not sure how common it is to request fixtures. The documentation does mention fixtures but very briefly https://docs.pytest.org/en/7.2.x/how-to/writing_plugins.html#requiring-loading-plugins-in-a-test-module-or-conftest-file.

We can rely on the fixtures to be loaded automatically, it is equivalent to writing them down on the root conftest

[diitaz93]

It looks nicely structured and seems like to improve the ability to lookup fixtures more easily. It would be great to expand this ot other modules. If we can retain all other functionality, maybe the tests/conftest.py will end up containing only imports.

Exactly. It is still left to discuss if we would like a hybrid approach with some fixtures in the contests (that could be very specific and only used in one test) or a fixture structure purely based on plugin requests

[diitaz93]

The full migration to plugins will be done in several PRs

[ChrOertlin]

I really like the new structure, it is much easier to see and understand what fixtures we are working with!

[Vince-janv]

This looks great! ⭐

[sonarqubecloud]

Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud