Skip to content

gh-156082: Do not run tests using the same machine-wide resource at once #156902

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
serhiy-storchaka wants to merge 3 commits into
base: main
Choose a base branch
from
Open

gh-156082: Do not run tests using the same machine-wide resource at once #156902

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
730794c
gh-156082: Do not run tests using the same machine-wide resource at once
serhiy-storchaka Aug 19, 2026
3d90c43
Address the review
serhiy-storchaka Sep 3, 2026
ee13664
Let the tests of different users lock each other out
serhiy-storchaka Sep 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Address the review 
Name the lock files, so that it is clear what left them in the temporary
directory, and say why they are left.  Close the file descriptor whatever
goes wrong.  Call the resource of bigmem tests 'bigmem'.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • Loading branch information
@serhiy-storchaka @claude
serhiy-storchaka and claude committed Sep 3, 2026
commit 3d90c4345c6fd045db1cdffdc9d80384b2b2ec74
19 changes: 13 additions & 6 deletions Lib/test/support/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -339,7 +339,8 @@ def get_resource_value(resource):

# Resources which are machine-wide: two tests using one of them at the same
# time interfere with each other.
EXCLUSIVE_RESOURCES = frozenset({'audio', 'console', 'curses', 'gui'})
EXCLUSIVE_RESOURCES = frozenset({'audio', 'bigmem', 'console', 'curses',
'gui'})

# The environment variable naming the directory in which the test runner keeps
# the lock files. It is set only when tests are run in parallel.
Expand All @@ -359,7 +360,7 @@ def _lock_file(fd):
try:
msvcrt.locking(fd, msvcrt.LK_NBLCK, 1)
except OSError:
# LK_LOCK gives up after ten seconds, so retry ourselves.
# It returns at once if the file is locked, so wait ourselves.
time.sleep(0.1)
else:
return
Expand All @@ -377,16 +378,22 @@ def _open_exclusive_lock(resource):
if not directory:
# The tests are not run in parallel, so there is nothing to exclude.
return None
# The file is left behind: another process may be waiting on a lock on it,
# and it is empty anyway.
path = os.path.join(directory, f'python-test-exclusive-{resource}.lock')
try:
fd = os.open(os.path.join(directory, f'exclusive-{resource}.lock'),
os.O_WRONLY | os.O_CREAT, 0o600)
fd = os.open(path, os.O_WRONLY | os.O_CREAT, 0o600)
except OSError:
return None
try:
_lock_file(fd)
except OSError:
except (ImportError, OSError):
# Locking is not available here, so the tests are not serialized.

@terryjreedy terryjreedy Sep 6, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic and intent of this is not clear to me.

os.close(fd)
return None

@vstinner vstinner Sep 3, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a catch-all "except: os.close(fd); raise" to make sure that the file descriptor is closed if something goes wrong. For example, if "import fcntl" and "import msvcrt" both raise an exception.

except BaseException:
os.close(fd)
raise
return fd


Expand Down Expand Up @@ -1454,7 +1461,7 @@ def wrapper(self):
cls = type(self)
qualname = f'{cls.__qualname__}.{f.__name__}'
# A real run allocates most of the memory of the machine.
with _exclusive_resource('memory'):
with _exclusive_resource('bigmem'):
proc = isolation._start_test(cls.__module__, qualname)
watchdog = _memory_watchdog(proc.pid) if verbose else None
payload, output, returncode = proc.wait(tick=watchdog)
Expand Down
Loading