Skip to content

gh-152635: Raise MemoryError rather than fail assert() when the lock allocation fails in _interpchannels.create() #152642

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sobolevn merged 6 commits into from
Jun 30, 2026
Merged

gh-152635: Raise MemoryError rather than fail assert() when the lock allocation fails in _interpchannels.create() #152642

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
c25ac11
Fix: gh-152635 _interpchannels channel lock alloc failure
stestagg Jun 29, 2026
b61a06f
Add @nomemtest decorator to the test, move the _testcapi call into th…
stestagg Jun 29, 2026
191f46a
self.assertRaises causes an allocation, so the _testcapi call
stestagg Jun 29, 2026
aeb50c0
Remove the test as it's not reliable enough, and testing allocation f…
stestagg Jun 30, 2026
b9182d3
Code review suggestions
stestagg Jun 30, 2026
25b178a
Remove the now-redundant (and incorrect) branch from
stestagg Jun 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 19 additions & 1 deletion Lib/test/test_interpreters/test_channels.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
# Raise SkipTest if subinterpreters not supported.
_channels = import_helper.import_module('_interpchannels')
from concurrent import interpreters
from test.support import channels
from test.support import channels, nomemtest
from .utils import _run_output, TestBase


Expand All @@ -30,6 +30,24 @@ def test_highlevel_reloaded(self):
# See gh-115490 (https://github.com/python/cpython/issues/115490).
importlib.reload(channels)

@nomemtest
def test_lock_allocation_failure(self):
# see gh-152635 (https://github.com/python/cpython/issues/152635)
# The first allocation to happen is the lock, which
# historically triggered an assert if alloc failed.
import _testcapi

cid = None
try:
with self.assertRaises(MemoryError):
_testcapi.set_nomemory(0, 1)

@StanFromIreland StanFromIreland Jun 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The required start/stop vary by platform, for example on Linux on an unpatched main I get:

$ ./python /tmp/repro.py 
Traceback (most recent call last):
  File "/tmp/repro.py", line 6, in <module>
    cid = _channels.create()
          ^^^^^^^^^
MemoryError

I'm also against testing with a loop where we try various numbers.

@stestagg stestagg Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aah thanks, yes that's annoying, I'd hoped that there would be enough consistency for this to just work.

I noticed that gh-151239 didn't include any tests, maybe for this reason, and that the reproducer in the linked gh-150213 didn't catch this issue, I assume only because they started with n=1.

Having a for-loop over ranges in the tests seems quite comprehensive but maybe overkill for what it is?

I'd appreciate any suggestions here!

@StanFromIreland StanFromIreland Jun 30, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, we just don't test this due to the associated complexity.

cid = _channels.create()
finally:
_testcapi.remove_mem_hooks()
if cid is not None:
_channels.close(cid, force=True)
_channels.destroy(cid)


class TestChannels(TestBase):

Expand Down
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Core_and_Builtins/2026-06-29-23-29-14.gh-issue-152635.O21J0O.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Fix a crash caused when running out of memory creating a
:mod:`!_interpchannels` channel. Now a :exc:`MemoryError` is correctly raised.
11 changes: 5 additions & 6 deletions Modules/_interpchannelsmodule.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -354,7 +354,7 @@
#define ERR_CHANNEL_INTERP_CLOSED -4
#define ERR_CHANNEL_EMPTY -5
#define ERR_CHANNEL_NOT_EMPTY -6
#define ERR_CHANNEL_MUTEX_INIT -7
#define ERR_CHANNEL_MUTEX_ALLOC_FAIL -7
Comment thread
stestagg marked this conversation as resolved.
Resolved
#define ERR_CHANNELS_MUTEX_INIT -8
#define ERR_NO_NEXT_CHANNEL_ID -9
#define ERR_CHANNEL_CLOSED_WAITING -10
Expand Down Expand Up @@ -427,9 +427,8 @@
"if not empty (try force=True)",
cid);
}
else if (err == ERR_CHANNEL_MUTEX_INIT) {
PyErr_SetString(state->ChannelError,
Comment thread
stestagg marked this conversation as resolved.
Resolved
"can't initialize mutex for new channel");
else if (err == ERR_CHANNEL_MUTEX_ALLOC_FAIL) {
PyErr_NoMemory();
}
Comment thread
stestagg marked this conversation as resolved.
Resolved
else if (err == ERR_CHANNELS_MUTEX_INIT) {
PyErr_SetString(state->ChannelError,
Expand Down Expand Up @@ -1744,7 +1743,7 @@
{
PyThread_type_lock mutex = PyThread_allocate_lock();
if (mutex == NULL) {
return ERR_CHANNEL_MUTEX_INIT;
return ERR_CHANNEL_MUTEX_ALLOC_FAIL;
Comment thread
stestagg marked this conversation as resolved.
Resolved
}
_channel_state *chan = _channel_new(mutex, defaults);
if (chan == NULL) {
Expand Down Expand Up @@ -2938,7 +2937,7 @@

int64_t cid = channel_create(&_globals.channels, defaults);
if (cid < 0) {
(void)handle_channel_error(-1, self, cid);
(void)handle_channel_error(cid, self, cid);

Check warning on line 2940 in Modules/_interpchannelsmodule.c

View workflow job for this annotation

GitHub Actions / Windows (free-threading) / Build and test (x64, tail-call) 

'function': conversion from 'int64_t' to 'int', possible loss of data [D:\a\cpython\cpython\PCbuild\pythoncore.vcxproj]

Check warning on line 2940 in Modules/_interpchannelsmodule.c

View workflow job for this annotation

GitHub Actions / Windows / Build and test (x64, switch-case) 

'function': conversion from 'int64_t' to 'int', possible loss of data [D:\a\cpython\cpython\PCbuild\pythoncore.vcxproj]

Check warning on line 2940 in Modules/_interpchannelsmodule.c

View workflow job for this annotation

GitHub Actions / Windows (free-threading) / Build and test (x64, switch-case) 

'function': conversion from 'int64_t' to 'int', possible loss of data [D:\a\cpython\cpython\PCbuild\pythoncore.vcxproj]

Check warning on line 2940 in Modules/_interpchannelsmodule.c

View workflow job for this annotation

GitHub Actions / Windows (free-threading) / Build and test (arm64, switch-case) 

'function': conversion from 'int64_t' to 'int', possible loss of data [C:\a\cpython\cpython\PCbuild\pythoncore.vcxproj]

Check warning on line 2940 in Modules/_interpchannelsmodule.c

View workflow job for this annotation

GitHub Actions / Windows / Build and test (arm64, switch-case) 

'function': conversion from 'int64_t' to 'int', possible loss of data [C:\a\cpython\cpython\PCbuild\pythoncore.vcxproj]

Check warning on line 2940 in Modules/_interpchannelsmodule.c

View workflow job for this annotation

GitHub Actions / Windows / Build and test (x64, tail-call) 

'function': conversion from 'int64_t' to 'int', possible loss of data [D:\a\cpython\cpython\PCbuild\pythoncore.vcxproj]

@IvyXu420 IvyXu420 Jul 9, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It causes a warning when compiling CPython using MSVC on Windows x64.
e.g., in GitHub Actions:

[Build CPython: Modules/_interpchannelsmodule.c#L2938]
'function': conversion from 'int64_t' to 'int', possible loss of data [D:\a\cpython\cpython\PCbuild\pythoncore.vcxproj]

@sobolevn sobolevn Jul 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will fix it, thanks for the report!

return NULL;
}
module_state *state = get_module_state(self);
Expand Down
Loading