Fixture teardown is called during setup of the next test

[rnetser]

When having 2 tests which call module-scoped yield fixtures and using parametrize, teardown is done between tests, during the setup of the 2nd test (pytest_runtest_setup) and not during pytest_runtest_teardown.
Running with pytest 6.2.5
Is there a way to achieve the desired flow (i.e teardown is done when exiting the fixture, as described below in the flow without parametrize) while using parameterized?

Example test:

import pytest

import logging

LOGGER = logging.getLogger(__name__)


@pytest.fixture(scope="module")
def fix1():
    LOGGER.info("++++++++++++++++++++++++++fix1")
    yield
    LOGGER.info("t1")


@pytest.fixture(scope="module")
def fix2():
    LOGGER.info("++++++++++++++++++++++++++fix2")
    yield
    LOGGER.info("t2")


@pytest.mark.parametrize("fix1, fix2",
    [
    pytest.param(1, 2)
    ],
    indirect=True)
def test1(fix1, fix2):
    LOGGER.info("in test")


def test2(fix1, fix2):
    LOGGER.info("in test")

Output:

tests.test_ruty 2021-11-08 20:27:55 INFO ++++++++++++++++++++++++++fix1
tests.test_ruty 2021-11-08 20:27:55 INFO ++++++++++++++++++++++++++fix2
------------------------------------------------------------------------------------------------------ CALL ------------------------------------------------------------------------------------------------------
tests.test_ruty 2021-11-08 20:27:55 INFO in test

TEST: test1[1-2] STATUS: PASSED
.---------------------------------------------------------------------------------------------------- TEARDOWN ----------------------------------------------------------------------------------------------------

__________________________________________________________________ 1 of 2 completed, 1 Pass, 0 Fail, 0 Skip, 0 XPass, 0 XFail, 0 Error, 0 ReRun ___________________________________________________________________

tests/test_ruty.py 
------------------------------------------------------------------------------------------------------ test2 ------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------ SETUP ------------------------------------------------------------------------------------------------------
tests.test_ruty 2021-11-08 20:27:55 INFO t1
tests.test_ruty 2021-11-08 20:27:55 INFO ++++++++++++++++++++++++++fix1
tests.test_ruty 2021-11-08 20:27:55 INFO t2
tests.test_ruty 2021-11-08 20:27:55 INFO ++++++++++++++++++++++++++fix2
------------------------------------------------------------------------------------------------------ CALL ------------------------------------------------------------------------------------------------------
tests.test_ruty 2021-11-08 20:27:55 INFO in test

TEST: test2 STATUS: PASSED
.---------------------------------------------------------------------------------------------------- TEARDOWN ----------------------------------------------------------------------------------------------------
tests.test_ruty 2021-11-08 20:27:55 INFO t2
tests.test_ruty 2021-11-08 20:27:55 INFO t1

__________________________________________________________________ 2 of 2 completed, 2 Pass, 0 Fail, 0 Skip, 0 XPass, 0 XFail, 0 Error, 0 ReRun ___________________________________________________________________

When running without parametrize:

import pytest

import logging

LOGGER = logging.getLogger(__name__)


@pytest.fixture(scope="module")
def fix1():
    LOGGER.info("++++++++++++++++++++++++++fix1")
    yield
    LOGGER.info("t1")


@pytest.fixture(scope="module")
def fix2():
    LOGGER.info("++++++++++++++++++++++++++fix2")
    yield
    LOGGER.info("t2")


# @pytest.mark.parametrize("fix1, fix2",
#     [
#     pytest.param(1, 2)
#     ],
#     indirect=True)
def test1(fix1, fix2):
    LOGGER.info("in test")


def test2(fix1, fix2):
    LOGGER.info("in test")

Result:

tests.test_ruty 2021-11-08 20:37:23 INFO ++++++++++++++++++++++++++fix1
tests.test_ruty 2021-11-08 20:37:23 INFO ++++++++++++++++++++++++++fix2
------------------------------------------------------------------------------------------------------ CALL ------------------------------------------------------------------------------------------------------
tests.test_ruty 2021-11-08 20:37:23 INFO in test

TEST: test1 STATUS: PASSED
.---------------------------------------------------------------------------------------------------- TEARDOWN ----------------------------------------------------------------------------------------------------

__________________________________________________________________ 1 of 2 completed, 1 Pass, 0 Fail, 0 Skip, 0 XPass, 0 XFail, 0 Error, 0 ReRun ___________________________________________________________________

tests/test_ruty.py 
------------------------------------------------------------------------------------------------------ test2 ------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------ SETUP ------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------ CALL ------------------------------------------------------------------------------------------------------
tests.test_ruty 2021-11-08 20:37:23 INFO in test

TEST: test2 STATUS: PASSED
.---------------------------------------------------------------------------------------------------- TEARDOWN ----------------------------------------------------------------------------------------------------
tests.test_ruty 2021-11-08 20:37:23 INFO t2
tests.test_ruty 2021-11-08 20:37:23 INFO t1

__________________________________________________________________ 2 of 2 completed, 2 Pass, 0 Fail, 0 Skip, 0 XPass, 0 XFail, 0 Error, 0 ReRun ___________________________________________________________________

[bluetech]

There are currently two places which teardown fixtures:

1. In the runner, based on scope (this is done by the SetupState class). E.g., if there was a module fixture, and the next item is in another module. This is not the case here; fix1/fix2 are module fixtures, and the Module is the same, so the runner doesn't know to teardown the fixtures based solely on scope.
2. Dynamically when a fixture is executed, when the fixture noticed that its parametrization has changed. This is the case here.

As you mention, (2) is definitely not how it should be done, since it has a few problems:

1. The teardown happens during the setup phase of the next item, instead of the teardown of this item.
2. The strict stack ordering of setups and teardowns is not maintained -- e.g. fix1 is torn down and set up, then fix2 is torn down and set up, while what should happen is fix1 teardown, fix2 teardown, fix1 setup, fix2 setup.

Fixing this will be a great improvement and will probably fix at least dozen open issues, but will require some research and work -- this is a pretty complex and subtle part of pytest.

[zeevrosental]

@Zac-HD @bluetech , Can we please declare this as a bug and not an enchantment?
will it help getting it fixed in a foreseeable future? :-)

[Zac-HD]

Sure, but the only think that helps issues get fixed is someone volunteering to fix them 🙂

[ilyak90]

@Zac-HD Until it is fixed - is there a workaround for this issue ?

[RonnyPfannschmidt]

It's not a bug per se, it's the expected interaction between parametrization and fixture setup/teardown as currently defined

Making it happen properly requires major changes to fixture definition, setup state and runtestloop

Most of those are pending since the 2016 sprint and at best daunting massive undertakings

[ilyak90]

@RonnyPfannschmidt The problem is that Pytest logs the next teardown as part of next test which means that the logs and the runtime of the teardown are treated as part of next test.

[Anton3]

pytest caches the fixture value according to the scope of the fixture, see my comment here.

In general, pytest does not know in advance when exactly parametrized fixtures will need to be recreated, it only knows that the value will be destroyed within the scope of the fixture.

If it's important to limit teardown of the parametrized fixture to function scope, then you can do so:

@pytest.mark.parametrize(
    "fix1, fix2",
    [
        pytest.param(1, 2)
    ],
    scope="function",  # <=====
    indirect=True,
)
def test1(fix1, fix2):
    LOGGER.info("in test")

Overall, I think the issue should be closed, and the documentation should be improved.

[RonnyPfannschmidt]

i'd like to eventually transfer this issue to a project wrt setupstate in general

back in the 2016 sprint @nicoddemus and a few others discussed replacing the idea of switching from the current mechanism of setup/teardown in runtestprotocol to something more powerful like a execution plan in order to enable better choice in teardown

as mentioned earlier the required refactoring have so far not been completed

however a good amount of work in the underlying tools has been completed