Repository navigation
[subinterpreters] Improve test coverage for subinterpreters #68741
Description
| BPO | 24553 |
|---|---|
| Nosy | @ncoghlan, @encukou, @skrah, @ericsnowcurrently |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2015-07-03.00:30:03.577>
labels = ['expert-subinterpreters', 'tests']
title = '[subinterpreters] Improve test coverage for subinterpreters'
updated_at = <Date 2020-05-15.00:41:59.313>
user = 'https://github.com/ericsnowcurrently'bugs.python.org fields:
activity = <Date 2020-05-15.00:41:59.313>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Tests', 'Subinterpreters']
creation = <Date 2015-07-03.00:30:03.577>
creator = 'eric.snow'
dependencies = []
files = []
hgrepos = []
issue_num = 24553
keywords = []
message_count = 14.0
messages = ['246107', '246149', '246207', '246208', '246237', '246238', '246253', '246255', '246256', '246257', '246298', '246677', '246695', '319559']
nosy_count = 5.0
nosy_names = ['ncoghlan', 'grahamd', 'petr.viktorin', 'skrah', 'eric.snow']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'test needed'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue24553'
versions = ['Python 3.5', 'Python 3.6']
Activity
ericsnowcurrently commentedon Jul 3, 2015
We do very little testing of subinterpreters in CPython. About all I'm aware of is in test_tracemalloc. I'll be working on improving test coverage as a precursor to fixing some existing bugs.
ncoghlan commentedon Jul 3, 2015
There are some basic tests in test_capi as well:
ericsnowcurrently commentedon Jul 3, 2015
FTR, subinterpreters are already accessible during testing with _testcapi.run_in_subinterp().
That function is used here:
test.support.run_in_subinterp() is used here:
ericsnowcurrently commentedon Jul 3, 2015
Also, I was mistaken about test_tracemalloc.
ericsnowcurrently commentedon Jul 4, 2015
Now I'm wondering what further test coverage we really need...
ncoghlan commentedon Jul 4, 2015
Sorry about the misleading reference to tracemalloc in my email - it was actually test_atexit I was debugging in the PEP-432 branch. tracemalloc only came up in that context because test.support.run_in_subinterp() automatically skips subinterpreter tests when tracemalloc is running. From the function comments:
# Issue bpo-10915, bpo-15751: PyGILState_*() functions don't work with
# sub-interpreters, the tracemalloc module uses these functions internally
As far as test coverage goes, what I would actually like to do is to run regrtest itself in a subinterpreter, and blacklist tests until it passes. This would double the length of a test run, so hiding it behind a resource option would probably be a good idea.
skrah commentedon Jul 4, 2015
Ideally we'd test every C module with the tests executing "in
parallel" (sort of) in multiple interpreters.
I have done so for _decimal, which is mostly okay due to the
thread-local contexts. However, there are a couple of minor
glitches.
The reason I haven't pursued this is that *if* you run _decimal
in multiple interpreters, you get nasty sounding (but harmless)
warnings from libmpdec, which complains about being initialized
multiple times.
So from the absence of bug reports I concluded that no one is
using the sub-interpreter feature with _decimal.
ncoghlan commentedon Jul 4, 2015
Adding Petr Viktorin to the nosy list as well, as the kind of issues Stefan mentions there are the kinds of things that the PEP-489 extension module import issues *didn't* address yet, but we'd like to address in the 3.6 iteration of the multi-phase initialisation support for extension modules.
Petr, for context, this issue is about improving our test coverage for subinterpreter support - see the earlier messages for more details.
skrah commentedon Jul 4, 2015
I think for fast access we need a hybrid solution that allows
static types (heap types slowed down _decimal) *and* cache the
thread local values (like it's currently done for the thread-local
context in _decimal).
Caching the context brought an enormous speedup (about 20%).
I'm not sure if all that can be done in a general API, but it's
clearly the fastest way.
ncoghlan commentedon Jul 4, 2015
That's jumping ahead a little - this issue is only about running the regression test suite from a subinterpreter, and establishing what *already* works properly. Dealing with the fallout of any quirks and outright failures we discover that way will be a different issue :)
skrah commentedon Jul 5, 2015
Nick, this may be a misunderstanding, but you mentioned issues
that PEP-489 did not address yet. The only issue left for _decimal
is the speed issue. Sub-interpreter tests run fine even with the
existing module state API, *if* one is willing to take a speed hit.
[I get that this is outside the scope of this issue, just for
clarification.]
ncoghlan commentedon Jul 13, 2015
As a possible starting point for this, I'll point to https://hg.python.org/cpython/file/02b81a82a57d/Lib/test/__main__.py
Now, I'd expected running that in a process that was *already* running regrtest to fail miserably (it would stomp all over itself).
But what we could potentially do is launch a *clean* subprocess, where the only thing it did was:
I'd currently expect that to fail as well, but I think the failures might be enlightening :)
I'm also not sure we need to integrate this directly into the main regrtest test runner - it could just be a separate submodule invoked like "python -m test.subinterpretertest".
If we later decided to integrate it, then it could go behind a "-usubinterpreter" resource that invoked "test.subinterpretertest" in a subprocess.
skrah commentedon Jul 13, 2015
+1 for python -m test.subinterpretertest. Based on my experiments,
most tests will either fail or leak memory at the very least.
ericsnowcurrently commentedon Jun 14, 2018
Note that bpo-32604 is strongly related.
[-]improve test coverage for subinterpreters[/-][+][subinterpreters] Improve test coverage for subinterpreters[/+]erlend-aasland commentedon Jan 12, 2024
Are there more actionable items left here?