Repository navigation
Accept mutable bytes-like objects #67190
Description
| BPO | 23001 |
|---|---|
| Nosy | @pitrou, @vstinner, @ethanfurman, @vadmium, @serhiy-storchaka, @MojoVampire |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2015-03-20.07:06:26.080>
created_at = <Date 2014-12-06.14:25:06.583>
labels = ['extension-modules', 'type-feature']
title = 'Accept mutable bytes-like objects'
updated_at = <Date 2015-03-20.10:01:21.895>
user = 'https://github.com/serhiy-storchaka'bugs.python.org fields:
activity = <Date 2015-03-20.10:01:21.895>
actor = 'vstinner'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2015-03-20.07:06:26.080>
closer = 'serhiy.storchaka'
components = ['Extension Modules']
creation = <Date 2014-12-06.14:25:06.583>
creator = 'serhiy.storchaka'
dependencies = []
files = ['37373', '38555']
hgrepos = []
issue_num = 23001
keywords = ['patch', 'needs review']
message_count = 14.0
messages = ['232244', '232246', '232252', '232255', '232264', '233438', '238502', '238598', '238607', '238608', '238628', '238630', '238633', '238635']
nosy_count = 8.0
nosy_names = ['pitrou', 'vstinner', 'Arfrever', 'ethan.furman', 'python-dev', 'martin.panter', 'serhiy.storchaka', 'josh.r']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue23001'
versions = ['Python 3.5']
Activity
serhiy-storchaka commentedon Dec 6, 2014
Some builtins accept only read-only bytes-like objects (PyArg_Parse format codes "s#", "z#", "y", and "y#"). Proposed patch makes them accepting also mutable bytes-like objects such as bytearray.
I'm not sure that all these changes are useful, but in some cases this can get rid of copying binary data created in bytearray (e.g. read by readinto()).
MojoVampire commentedon Dec 6, 2014
In the event of calls back into Python code in multithreaded execution (or GIL release), this would mean you no longer have guarantees as to the contents (or even the validity) of the pointer you get back. I'd think the only safe way to accept mutable buffers would be to use the s*, z*, y* codes, which lock the buffer to prevent resize/destruction. Do we want to open segfault vulnerabilities in arbitrary functions?
serhiy-storchaka commentedon Dec 6, 2014
That is was the patch does. Convert from s# to s* etc.
See also bpo-22896 about potential bugs with the use of pointers to unlocked
buffers.
MojoVampire commentedon Dec 6, 2014
Ah, sorry. Should have examined patch. I thought you were making a change to the behavior of s#, z#, y and y#, not converting actual uses of them. Again, sorry.
pitrou commentedon Dec 7, 2014
+1 on the principle. I haven't looked at the patch.
vadmium commentedon Jan 5, 2015
If this goes ahead, it would be nice adding notes to the documentation saying that bytearray() or whatever was previously not supported. There are APIs in Python 2.6 that had similar treatment with no documentation updates, and I keep being bitten by it.
serhiy-storchaka commentedon Mar 19, 2015
Added versionchanged directives and fixed other documentation.
vadmium commentedon Mar 20, 2015
I had a closer look at the changes, and most of them seem good. However I think I found one leak in fcntl(); see Reitveld.
I noticed there is no test for “ossaudiodev”. Would that be too hard, or is it just an oversight?
serhiy-storchaka commentedon Mar 20, 2015
Thank you for your review Martin. Yes, the problem with fcntl() is not so easy and I'll left it for the separate issue.
It is hard because test_ossaudiodev is not designed to just apply the test with different type of data, and this test doesn't work on my computer at all (no /dev/dsp).
python-dev commentedon Mar 20, 2015
New changeset 17c77938c4e2 by Serhiy Storchaka in branch 'default':
Issue bpo-23001: Few functions in modules mmap, ossaudiodev, socket, ssl, and
https://hg.python.org/cpython/rev/17c77938c4e2
python-dev commentedon Mar 20, 2015
New changeset 11548e1ac920 by Victor Stinner in branch 'default':
Issue bpo-23001: Fix typo
https://hg.python.org/cpython/rev/11548e1ac920
python-dev commentedon Mar 20, 2015
New changeset d478a2a5738a by Victor Stinner in branch 'default':
Issue bpo-23709, bpo-23001: ossaudiodev now uses Py_ssize_t for sizes instead of int
https://hg.python.org/cpython/rev/d478a2a5738a
serhiy-storchaka commentedon Mar 20, 2015
Thanks Victor.
vstinner commentedon Mar 20, 2015
You're welcome, I just worked recently on ossaudiodev for _Py_read/write functions functions. But I'm not able to run ossaudiodev neither :-( I hope that the test runs on some buildbots (FreeBSD?).