Repository navigation
struct.unpack and cStringIO.StringIO don't support new buffer #54421
Description
| BPO | 10212 |
|---|---|
| Nosy | @malemburg, @terryjreedy, @pitrou, @kristjanvalur, @benjaminp, @merwok, @skrah |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2013-03-22.17:25:56.319>
created_at = <Date 2010-10-27.10:36:51.049>
labels = ['extension-modules', 'type-bug']
title = "struct.unpack and cStringIO.StringIO don't support new buffer"
updated_at = <Date 2013-03-22.22:53:58.754>
user = 'https://github.com/kristjanvalur'bugs.python.org fields:
activity = <Date 2013-03-22.22:53:58.754>
actor = 'Arfrever'
assignee = 'none'
closed = True
closed_date = <Date 2013-03-22.17:25:56.319>
closer = 'kristjan.jonsson'
components = ['Extension Modules']
creation = <Date 2010-10-27.10:36:51.049>
creator = 'kristjan.jonsson'
dependencies = []
files = ['19382']
hgrepos = []
issue_num = 10212
keywords = ['patch', 'needs review']
message_count = 11.0
messages = ['119687', '119689', '119750', '119781', '119939', '156389', '156426', '184524', '184549', '184562', '184717']
nosy_count = 10.0
nosy_names = ['lemburg', 'terry.reedy', 'tseaver', 'pitrou', 'kristjan.jonsson', 'benjamin.peterson', 'eric.araujo', 'Arfrever', 'skrah', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue10212'
versions = ['Python 2.7']
Activity
kristjanvalur commentedon Oct 27, 2010
When doing socket IO, it is beneficial to use a bytearra() and then using sock.recv_into() to avoid moving data about.
However, many useful functions still don't accept new style buffers, such as the bytearray and memoryview. In particular, the struct module cannot unpack from them, and the StringIO doesn't accept them as input.
The attached patch adds new-buffer support to the struct module and cStringIO.
pitrou commentedon Oct 27, 2010
You can't add buffer protocol support to cStringIO in a bugfix release, since it would be a new feature.
kristjanvalur commentedon Oct 28, 2010
I disagree. It's not a new feature. We're merely completing an old feature (adding new-style buffers from 3.x to 2.7) that wasn't fully implemented. by the core. The new buffer isn't accepted in a lot of places where you'd expect it to be.
The good alternative, of course, would be to add this to "trunk" but for some reason that is frowned upon.
See also bpo-10211
merwok commentedon Oct 28, 2010
MAL’s viewpoint from msg119721:
“The memoryview object was added to simplify porting applications to Python3. If that backport is incomplete in the sense that the memoryview object is not compatible with the standard Python2 object for such memory views, then I'd consider that a bug.
Without compatibility to the buffer objects, there's no way to make other Python2 buffer interface compatible object compatible to memoryviews.
Alternatively, the Python2 memoryview object implementation could also accept objects with the old buffer interface, much like "s*" does.
Note that the patch needs to check the buffer flags - writing to such buffers is not allowed.”
terryjreedy commentedon Oct 29, 2010
In the absence of doc references in this and bpo-10211 that would *clearly* settle the bug vs. feature issue, it strikes me as a bit murky. So I am inclined to agree with MAL that failure to achieve the stated, documented purpose is a bug. I strongly suspect that if these issues had been filed during the 2.7 beta phase, a fix would have been deemed permissible without much controversy.
kristjanvalur commentedon Mar 20, 2012
The patch is still there. Any new consensus?
terryjreedy commentedon Mar 20, 2012
Stefan, since you have been working on Memoryview, any opinions?
kristjanvalur commentedon Mar 18, 2013
We don't seem to be getting any traction on this. 2.7.4 is to be cut next weekend so it is becoming more urgent :)
terryjreedy commentedon Mar 18, 2013
For the backport policy question, since there seems to be disagreement, consider a post on pydev asking for discussion and if no quick consensus, a ruling from Guido or designee.
tseaver commentedon Mar 19, 2013
Patch looks good to me. I would call it a bugfix, not a feature.
python-dev commentedon Mar 20, 2013
New changeset be4bec689de3 by Kristján Valur Jónsson in branch '2.7':
Issue bpo-10212: Support new buffer interface for struct.unpack and
http://hg.python.org/cpython/rev/be4bec689de3