Reference leaks introduced by bpo-30860

@ncoghlan

BPO	31420
Nosy	@ncoghlan, @vstinner, @ericvsmith, @skrah, @ericsnowcurrently
PRs	bpo-30860: Fix a refleak. #3506 bpo-30860: Fix a refleak. #3567

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ericsnowcurrently'
closed_at = <Date 2017-09-12.01:00:56.078>
created_at = <Date 2017-09-11.16:29:21.025>
labels = ['3.7', 'performance']
title = 'Reference leaks introduced by bpo-30860'
updated_at = <Date 2017-09-14.07:36:00.199>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2017-09-14.07:36:00.199>
actor = 'eric.snow'
assignee = 'eric.snow'
closed = True
closed_date = <Date 2017-09-12.01:00:56.078>
closer = 'eric.snow'
components = []
creation = <Date 2017-09-11.16:29:21.025>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 31420
keywords = ['patch']
message_count = 8.0
messages = ['301882', '301883', '301886', '301889', '301891', '301904', '301906', '302154']
nosy_count = 5.0
nosy_names = ['ncoghlan', 'vstinner', 'eric.smith', 'skrah', 'eric.snow']
pr_nums = ['3506', '3567']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue31420'
versions = ['Python 3.7']

The following commands show memory leaks introduced by bpo-30860:

$ ./python -m test -R 3:3 test_atexit -m test.test_atexit.SubinterpreterTest.test_callbacks_leak

$ ./python -m test -R 3:3 test_atexit -m test.test_atexit.SubinterpreterTest.test_callbacks_leak_refcycle

$ ./python -m test -R 3:3 test_threading -m test.test_threading.SubinterpThreadingTests.test_threads_join

$ ./python -m test -R 3:3 test_capi -m test.test_capi.SubinterpreterTest.test_subinterps

The following commands show memory leaks introduced by bpo-30860:

Oh, I'm talking about the commit 2ebc5ce.

Could be the same as bpo-31408.

The leak comes from the following lines of _PySys_EndInit():
---

    PyObject *warnoptions = get_warnoptions();
    if (warnoptions == NULL)
        return -1;
    SET_SYS_FROM_STRING_BORROW_INT_RESULT("warnoptions", warnoptions);

    PyObject *xoptions = get_xoptions();
    if (xoptions == NULL)
        return -1;
    SET_SYS_FROM_STRING_BORROW_INT_RESULT("_xoptions", xoptions);

It's not the first time that I have an issue with these attributes. The last reference weak caused by multiple interpreters was also related to this one if I recall correctly.

See bpo-30598 and my commit 865de27.

Extract of my msg295399: "The problem is that warnoptions is stored in a C global variable *and* in sys.warnoptions of each interpreter. The ownership of this variable is unclear."

Maybe we need a change similar to bpo-28411 (commit 86b7afd) which removed the "modules" field from Py_InterpreterState.

I'm looking into this.

Moving warnoptions (and xoptions) out of PyInterpreterState seems like a good idea to me for the same reasons as applied to sys.modules. The case for these two is even stronger since they are only used in sysmodule.c.

New changeset dae0276 by Eric Snow in branch 'master':
bpo-30860: Fix a refleak. (bpo-3567)
dae0276

Navigation Menu

Repository navigation

Reference leaks introduced by bpo-30860 #75601

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Reference leaks introduced by bpo-30860 #75601

Description

Activity

vstinner commented on Sep 11, 2017

vstinner commented on Sep 11, 2017

skrah commented on Sep 11, 2017

vstinner commented on Sep 11, 2017

vstinner commented on Sep 11, 2017

ericsnowcurrently commented on Sep 11, 2017

ericsnowcurrently commented on Sep 11, 2017

ericsnowcurrently commented on Sep 14, 2017

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions