Create a fresh test class instance for each rerun - #340
Open
teddytennant wants to merge 1 commit into
Open
Conversation
pytest caches the test class instance on the test item, so reruns reused the instance of the attempt that had just failed. Any state stored on self -- including attributes set by class-defined fixtures -- leaked from one attempt into the next, breaking the per-test isolation pytest otherwise guarantees. Drop the cached instance (and the method bound to it) as part of the existing pre-rerun cleanup, so pytest rebuilds both lazily. Cached results of fixtures with a scope higher than function are untouched, so those fixtures are still not re-executed on a rerun.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
pytest gives every test item its own instance of the test class, and caches it on
the item. The rerun loop reuses the same item, so a rerun also reuses the instance
of the attempt that just failed: state written to
self(by the test itself or bya class-defined fixture) survives into the next attempt, which breaks the per-test
isolation pytest otherwise guarantees. This changed with pytest 7.0; with pytest 6.x
each attempt did get a fresh instance.
Reproducer (from #268):
Before:
After:
The fix
_discard_test_class_instance()drops the item's cached_instanceand the methodbound to it, as part of the cleanup that already runs just before a rerun is
triggered (after the failed attempt's teardown has completed). pytest recomputes
both lazily on next access via
Function._getinstance(), which is exactly the pathused for a normal fresh test item — so no plugin-side reimplementation of
instantiation is needed. Doing it here rather than in a
pytest_runtest_setuphookkeeps it next to the other per-rerun cleanup and means it only ever runs when this
plugin has actually decided to rerun.
Deliberately unchanged: nothing touches
FixtureDef.cached_result, so fixturescached at class, module, package or session scope are still not re-executed on a
rerun. That is what
_remove_cached_results_from_failed_fixtures()is careful aboutand what
test_rerun_on_module_fixture_with_reruns/test_rerun_on_session_fixture_with_rerunspin — the second new test below pins itagain for this code path specifically, since "recreate the instance" is easy to get
wrong by invalidating fixtures along with it.
The guard returns early when no instance is cached, so plain (non-method) test
functions are unaffected, and
unittest.TestCaseitems are too — pytest's ownTestCaseFunction.teardown()already drops_instance/_objitself, and the guardavoids eagerly constructing a throwaway
TestCasejust to discard it.Tests
Two tests in
tests/test_pytest_rerunfailures.py:test_rerun_recreates_test_class_instance— state set onselfby the test and byan autouse fixture must not be visible on the next attempt.
test_rerun_recreates_instance_without_re_executing_scoped_fixtures— same, plus anassertion that the session-, module- and class-scoped fixtures each ran exactly once
across all three attempts.
Before the fix (source change reverted, tests present):
with the inner failure being
After:
Full suite, Python 3.13, run against every pytest version in the tox matrix
(baseline on master is 147 passed; the two skips on pytest < 9 are the subtests tests):
pre-commit run --all-filespasses.Fixes #268.