Repository navigation
Pytest 8 broke sorting for package-scoped fixtures #12328
Description
Pytest 8 has a regression where the test reordering no longer favours reuse of package-scoped fixtures. If a parametrised package-scoped fixture is used in multiple modules within the same package, then instead of running the whole package with one fixture instantiation before switching to the next, it runs a module at a time, causing the fixture to be torn down and recreated each time.
MWE
conftest.py:
import pytest
@pytest.fixture(scope="package", params=["a", "b", "c"])
def fixture(request):
return request.parampkg/__init__.py: empty
pkg/test_things1.py:
def test1(fixture):
pass
pkg/test_things2.py:
def test2(fixture):
pass
Output of pytest --setup-plan from pytest 8.2.0:
======================================= test session starts ========================================
platform linux -- Python 3.12.3, pytest-8.2.0, pluggy-1.5.0
rootdir: /home/bmerry/work/experiments/NGC-1318-pytest-ordering/simple
collected 6 items
pkg/test_things1.py
SETUP P fixture['a']
pkg/test_things1.py::test1[a] (fixtures used: fixture, request)
TEARDOWN P fixture['a']
SETUP P fixture['b']
pkg/test_things1.py::test1[b] (fixtures used: fixture, request)
TEARDOWN P fixture['b']
SETUP P fixture['c']
pkg/test_things1.py::test1[c] (fixtures used: fixture, request)
pkg/test_things2.py
TEARDOWN P fixture['c']
SETUP P fixture['a']
pkg/test_things2.py::test2[a] (fixtures used: fixture, request)
TEARDOWN P fixture['a']
SETUP P fixture['b']
pkg/test_things2.py::test2[b] (fixtures used: fixture, request)
TEARDOWN P fixture['b']
SETUP P fixture['c']
pkg/test_things2.py::test2[c] (fixtures used: fixture, request)
TEARDOWN P fixture['c']
Whereas pytest 7.4.4 produces:
platform linux -- Python 3.12.3, pytest-7.4.4, pluggy-1.5.0
rootdir: /home/bmerry/work/experiments/NGC-1318-pytest-ordering/simple
collected 6 items
pkg/test_things1.py
SETUP P fixture['a']
pkg/test_things1.py::test1[a] (fixtures used: fixture, request)
pkg/test_things2.py
pkg/test_things2.py::test2[a] (fixtures used: fixture, request)
pkg/test_things1.py
TEARDOWN P fixture['a']
SETUP P fixture['b']
pkg/test_things1.py::test1[b] (fixtures used: fixture, request)
pkg/test_things2.py
pkg/test_things2.py::test2[b] (fixtures used: fixture, request)
pkg/test_things1.py
TEARDOWN P fixture['b']
SETUP P fixture['c']
pkg/test_things1.py::test1[c] (fixtures used: fixture, request)
pkg/test_things2.py
pkg/test_things2.py::test2[c] (fixtures used: fixture, request)
TEARDOWN P fixture['c']
====================================== no tests ran in 0.01s =======================================
If the fixture is not parametric (or has only one parameter), it does correctly re-use the fixture across the whole package, so the issue seems to be related to sorting rather than re-use.
Changing the scope to session gives sensible behaviour, so it seems to be specific to package scope.
System information
pip list:
Package Version
---------- -------
iniconfig 2.0.0
packaging 24.0
pip 24.0
pluggy 1.5.0
pytest 8.2.0
setuptools 69.5.1
wheel 0.43.0
OS: Ubuntu 24.04
- a detailed description of the bug or problem you are having
- output of
pip listfrom the virtual environment you are using - pytest and operating system versions
- minimal example if possible
Activity
bluetech commentedon May 15, 2024
I think I agree this is a regression. This broke with a21fb87 (released in 8.0.0). Specifically, the
reorder_itemsfunction stopped reordering the items in this case. I will take a look.fixtures: fix non-working package-scope parametrization reordering