Repository navigation
Some C-API symbols (e.g. Py_FrozenMain) are not always exported on Unix #88299
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2022-01-17.13:50:21.465>
created_at = <Date 2021-05-14.16:58:44.058>
labels = ['expert-C-API', '3.11']
title = 'Some C-API symbols (e.g. Py_FrozenMain) are not always exported on Unix'
updated_at = <Date 2022-04-01.08:55:02.796>
user = 'https://github.com/vstinner'bugs.python.org fields:
activity = <Date 2022-04-01.08:55:02.796>
actor = 'kulikjak'
assignee = 'none'
closed = True
closed_date = <Date 2022-01-17.13:50:21.465>
closer = 'vstinner'
components = ['C API']
creation = <Date 2021-05-14.16:58:44.058>
creator = 'vstinner'
dependencies = []
files = ['50043']
hgrepos = []
issue_num = 44133
keywords = ['patch']
message_count = 17.0
messages = ['393675', '393676', '393693', '404596', '404723', '404748', '406980', '410401', '410503', '410504', '410756', '410757', '410762', '410776', '410787', '416378', '416475']
nosy_count = 5.0
nosy_names = ['vstinner', 'petr.viktorin', 'miss-islington', 'kulikjak', 'shihai1991']
pr_nums = ['26130', '29148', '29876', '30556', '30636']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue44133'
versions = ['Python 3.11']
Activity
vstinner commentedon Jan 13, 2022
Sadly, Py_FrozenMain() is still missing on Windows. See:
#30556 (comment)
Until Windows also exports the symbol, I don't think that we can add the symbol to the stable ABI.
I prefer to not backport the change since it's always risky to break the build system on a stable branch.
If someone wants to fix the Windows build to also export Py_FrozenMain(), please open a new issue.
I consider that the initial issue is fixed, so I close the issue.
[-]Some C-API symbols (e.g. Py_FrozenMain) are not always exported[/-][+]Some C-API symbols (e.g. Py_FrozenMain) are not always exported on Unix[/+][-]Some C-API symbols (e.g. Py_FrozenMain) are not always exported[/-][+]Some C-API symbols (e.g. Py_FrozenMain) are not always exported on Unix[/+]kulikjak commentedon Jan 17, 2022
Now it started failing on a different place:
======================================================================
FAIL: test_export_symbols (test.test_capi.CAPITest) (name='PyThread_get_thread_native_id')
----------------------------------------------------------------------
Looking at the test, is the expectation that all OSes must implement it since 3.11?
vstinner commentedon Jan 17, 2022
Oh no, I expected that the new way to build Python would also export PyThread_get_thread_native_id() on Solaris. I reopen the issue.
Can you please specify your configure command? Can you check without Python if the symbol is exported or not? If you use --enable-shared, check in libpython, otherwise check in the "python" binary.
kulikjak commentedon Jan 17, 2022
Ah, sorry, I could have described the issue better. It's not a problem with exporting, PyThread_get_thread_native_id() isn't available on Solaris (and possibly other platforms) at all.
https://github.com/python/cpython/blob/main/Include/pythread.h#L28
https://github.com/python/cpython/blob/main/Python/thread_pthread.h#L329
The reason I didn't implement it yet is that Solaris doesn't expose anything like native thread id. We do have functions like
_lwp_self()orpthread_self()orthr_self()but neither of them returns id where "value may be used to uniquely identify this particular thread system-wide". (I presume that means that no other thread of no other process running on a given system would return the same number - all these functions return single digit numbers so there is no way they are unique system wide).If necessary, I guess that such a number can be created by masking pid and thread id together, but then there's a question of how it is supposed to be used (because OS would not understand it).
vstinner commentedon Jan 17, 2022
Oh ok, it's a simple bug in my test. I wrote #74821 to fix it.
vstinner commentedon Jan 17, 2022
New changeset 16901c0 by Victor Stinner in branch 'main':
bpo-44133: Skip PyThread_get_thread_native_id() if not available (GH-30636)
16901c0
encukou commentedon Mar 30, 2022
Jakub, does this mean test_stable_abi_ctypes fails on Solaris?
Are there any other missing functions?
I opened bpo-47169 to improve the situation.
kulikjak commentedon Apr 1, 2022
Yes, it still does, and PyThread_get_thread_native_id is the only symbol missing.
encukou commentedon Apr 29, 2022
PyModule_Create2andPyModule_FromDefAndSpec2are still missing on Arch Linux TraceRefs BuildBot :(https://buildbot.python.org/all/#/builders/82/builds/548
vstinner commentedon Apr 29, 2022
The Py_TRACE_REF macro changes the ABI on purpose, I suggest you to open a separated issue if you plan to implement the stable ABI for this special debug build:
Extract of modsupport.h:
Py_TRACE_REF is incompatible with the stable ABI, extract of object.h:
The PyObject layout is different, it adds two pointers to the structure: _ob_next and _ob_prev. In Python 3.8, I modified the Python debug build to make it ABI compatible with the release build by not defining Py_TRACE_REF if Py_DEBUG is defined.
Someone proposed to reimplement Py_TRACE_REF with a separated list or hash table of all objects (the _ob_prev and _ob_next linked list), rather than modifying PyObject. But nobody implemented it.
Note: Py_TRACE_REF requires additional _Py_AddToAllObjects() and _Py_ForgetReference() function calls in _Py_NewReference() and _Py_Dealloc(). But hopefully, _Py_NewReference() and _Py_Dealloc() are function calls at the ABI level, so the different implementation is not an issue. It seems like the only blocker issue is the PyObject layout change.
encukou commentedon Apr 29, 2022
Ah! I see. Sorry, I just added a quick comment and didn't investigate further immediately.
I reopened #91325 instead :)
vstinner commentedon Apr 29, 2022
No problem, thanks for working on clarifying the stable ABI!