Repository navigation
Convert _hashlib to PEP 489 multiphase initialization #84848
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/tiran'
closed_at = <Date 2020-10-19.17:04:31.425>
created_at = <Date 2020-05-18.13:07:20.001>
labels = ['type-feature', 'library', '3.9']
title = 'Convert _hashlib to PEP 489 multiphase initialization'
updated_at = <Date 2020-10-19.17:04:31.425>
user = 'https://github.com/vstinner'bugs.python.org fields:
activity = <Date 2020-10-19.17:04:31.425>
actor = 'corona10'
assignee = 'christian.heimes'
closed = True
closed_date = <Date 2020-10-19.17:04:31.425>
closer = 'corona10'
components = ['Library (Lib)']
creation = <Date 2020-05-18.13:07:20.001>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 40671
keywords = ['patch']
message_count = 6.0
messages = ['369211', '369216', '369221', '369229', '369862', '369874']
nosy_count = 6.0
nosy_names = ['vstinner', 'christian.heimes', 'petr.viktorin', 'corona10', 'miss-islington', 'shihai1991']
pr_nums = ['20180', '20378', '20381']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue40671'
versions = ['Python 3.9']
Activity
vstinner commentedon May 18, 2020
In Modules/_hashopenssl.c:PyInit__hashlib(), PyModule_AddObject() can be replaced with PyModule_AddType() to add the 3 types: this function does the Py_INCREF for us, which reduces errors if the function fails.
The results of other PyModule_AddObject() calls should also be checked to handle errors.
The following _hashlibstate_global macro is no longer used:
EVPnew() got a module parameter, so it doesn't need _hashlibstate_global anymore.
Dong-hee, Hai: Maybe it's time to convert this module to PEP-489 multiphase initialization?
Hai: do you want to work on a PR?
--
Question: do EVPobject and HMACobject rely on a global state somewhere in OpenSSL? Or is it safe to use them in subinterpreters, with multiple instances of _hashlib?
EVPobject and HMACobject have "lock" member.
Even if converting the module to PEP-489 is not enough to make it compatible with subinterpreters (I don't know, maybe it's enought, see my question), it should help for that :-)
Note: the current code is already quite clean, I like it :-)
cc Petr & Christian.
tiran commentedon May 18, 2020
OpenSSL's EVP and HMAC API depend on global state in OpenSSL, but they are still safe to be used in subinterpreters. OpenSSL ensures that the internal data structures are initialized in a thread-safe manner. The internal state cannot leak across subinterpreters. OpenSSL has process global configuration, but Python's ssl and hashlib module do not expose these features.
The Python wrappers EVPobject, EVPXOFobject and HMACobject are **not** safe to be transfered across subinterpreters. The lock shouldn't matter here. It's used to release the GIL and block other threads from feeding into an object at the same time.
vstinner commentedon May 18, 2020
No Python object must be shared between two interpreter and no object should be transfered form one interpreter to another. That's not specific to _hashlib, but a general guidelines for subinterpreters ;-)
Converting _hashlib should allow to have one _hashlib module instance per interpreter ;-) (Rather than sharing the same instance between all interpereters which caused so many issues!)
shihai1991 commentedon May 18, 2020
Oh, Looks like I am late, Christian have crated a PR ;)
tiran commentedon May 25, 2020
New changeset 20c22db by Christian Heimes in branch 'master':
bpo-40671: Prepare _hashlib for PEP-489 (GH-20180)
20c22db
miss-islington commentedon May 25, 2020
New changeset 1fe1a14 by Miss Islington (bot) in branch '3.9':
bpo-40671: Prepare _hashlib for PEP-489 (GH-20180)
1fe1a14