Skip to content

Create a fresh test class instance for each rerun - #340

Open
teddytennant wants to merge 1 commit into
pytest-dev:masterfrom
teddytennant:fix-268-recreate-instance-on-rerun
Open

Create a fresh test class instance for each rerun#340
teddytennant wants to merge 1 commit into
pytest-dev:masterfrom
teddytennant:fix-268-recreate-instance-on-rerun

Conversation

@teddytennant

Copy link
Copy Markdown

pytest gives every test item its own instance of the test class, and caches it on
the item. The rerun loop reuses the same item, so a rerun also reuses the instance
of the attempt that just failed: state written to self (by the test itself or by
a class-defined fixture) survives into the next attempt, which breaks the per-test
isolation pytest otherwise guarantees. This changed with pytest 7.0; with pytest 6.x
each attempt did get a fresh instance.

Reproducer (from #268):

import pytest

class TestExample:
    @property
    def counter(self):
        self.__dict__.setdefault('_counter', 0)
        self._counter += 1
        return self._counter

    @pytest.fixture(autouse=True)
    def some_fixture(self):
        print('SETUP', self.counter)
        yield
        print('TEARDOWN', self.counter)

    @pytest.mark.flaky(reruns=5)
    def test_something(self, param=[]):
        print('TEST', self.counter)
        param.append(0)
        assert len(param) > 2

Before:

SETUP 1
TEST 2
TEARDOWN 3
RSETUP 4
TEST 5
TEARDOWN 6
RSETUP 7
TEST 8
TEARDOWN 9
.

After:

SETUP 1
TEST 2
TEARDOWN 3
RSETUP 1
TEST 2
TEARDOWN 3
RSETUP 1
TEST 2
TEARDOWN 3
.

The fix

_discard_test_class_instance() drops the item's cached _instance and the method
bound to it, as part of the cleanup that already runs just before a rerun is
triggered (after the failed attempt's teardown has completed). pytest recomputes
both lazily on next access via Function._getinstance(), which is exactly the path
used for a normal fresh test item — so no plugin-side reimplementation of
instantiation is needed. Doing it here rather than in a pytest_runtest_setup hook
keeps it next to the other per-rerun cleanup and means it only ever runs when this
plugin has actually decided to rerun.

Deliberately unchanged: nothing touches FixtureDef.cached_result, so fixtures
cached at class, module, package or session scope are still not re-executed on a
rerun. That is what _remove_cached_results_from_failed_fixtures() is careful about
and what test_rerun_on_module_fixture_with_reruns /
test_rerun_on_session_fixture_with_reruns pin — the second new test below pins it
again for this code path specifically, since "recreate the instance" is easy to get
wrong by invalidating fixtures along with it.

The guard returns early when no instance is cached, so plain (non-method) test
functions are unaffected, and unittest.TestCase items are too — pytest's own
TestCaseFunction.teardown() already drops _instance/_obj itself, and the guard
avoids eagerly constructing a throwaway TestCase just to discard it.

Tests

Two tests in tests/test_pytest_rerunfailures.py:

  • test_rerun_recreates_test_class_instance — state set on self by the test and by
    an autouse fixture must not be visible on the next attempt.
  • test_rerun_recreates_instance_without_re_executing_scoped_fixtures — same, plus an
    assertion that the session-, module- and class-scoped fixtures each ran exactly once
    across all three attempts.

Before the fix (source change reverted, tests present):

$ pytest tests/test_pytest_rerunfailures.py -k recreate
FAILED tests/test_pytest_rerunfailures.py::test_rerun_recreates_test_class_instance
FAILED tests/test_pytest_rerunfailures.py::test_rerun_recreates_instance_without_re_executing_scoped_fixtures
2 failed, 147 deselected in 0.18s

with the inner failure being

>       assert not hasattr(self, 'seen_by_test')
E       AssertionError: assert not True

After:

$ pytest tests/test_pytest_rerunfailures.py -k recreate -v
tests/test_pytest_rerunfailures.py::test_rerun_recreates_test_class_instance PASSED
tests/test_pytest_rerunfailures.py::test_rerun_recreates_instance_without_re_executing_scoped_fixtures PASSED
2 passed, 147 deselected in 0.19s

Full suite, Python 3.13, run against every pytest version in the tox matrix
(baseline on master is 147 passed; the two skips on pytest < 9 are the subtests tests):

pytest result
8.2.* 147 passed, 2 skipped
8.3.* 147 passed, 2 skipped
8.4.* 147 passed, 2 skipped
9.0.* 149 passed
9.1.* 149 passed
main 149 passed

pre-commit run --all-files passes.

Fixes #268.

pytest caches the test class instance on the test item, so reruns reused
the instance of the attempt that had just failed. Any state stored on
self -- including attributes set by class-defined fixtures -- leaked
from one attempt into the next, breaking the per-test isolation pytest
otherwise guarantees.

Drop the cached instance (and the method bound to it) as part of the
existing pre-rerun cleanup, so pytest rebuilds both lazily. Cached
results of fixtures with a scope higher than function are untouched, so
those fixtures are still not re-executed on a rerun.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test class instance is not recreated when retrying tests

1 participant