Skip to content

struct.unpack and cStringIO.StringIO don't support new buffer #54421

Closed
Closed
struct.unpack and cStringIO.StringIO don't support new buffer#54421
Labels
extension-modulesC modules in the Modules dirtype-bugAn unexpected behavior, bug, or error

Description

@kristjanvalur
kristjanvalur
mannequin
opened on Oct 27, 2010
BPO 10212
Nosy @malemburg, @terryjreedy, @pitrou, @kristjanvalur, @benjaminp, @merwok, @skrah
Files
  • newbuffer.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2013-03-22.17:25:56.319>
created_at = <Date 2010-10-27.10:36:51.049>
labels = ['extension-modules', 'type-bug']
title = "struct.unpack and cStringIO.StringIO don't support new buffer"
updated_at = <Date 2013-03-22.22:53:58.754>
user = 'https://github.com/kristjanvalur'

    bugs.python.org fields:

    activity = <Date 2013-03-22.22:53:58.754>
actor = 'Arfrever'
assignee = 'none'
closed = True
closed_date = <Date 2013-03-22.17:25:56.319>
closer = 'kristjan.jonsson'
components = ['Extension Modules']
creation = <Date 2010-10-27.10:36:51.049>
creator = 'kristjan.jonsson'
dependencies = []
files = ['19382']
hgrepos = []
issue_num = 10212
keywords = ['patch', 'needs review']
message_count = 11.0
messages = ['119687', '119689', '119750', '119781', '119939', '156389', '156426', '184524', '184549', '184562', '184717']
nosy_count = 10.0
nosy_names = ['lemburg', 'terry.reedy', 'tseaver', 'pitrou', 'kristjan.jonsson', 'benjamin.peterson', 'eric.araujo', 'Arfrever', 'skrah', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue10212'
versions = ['Python 2.7']

    Activity

    1. kristjanvalur commented on Oct 27, 2010

      @kristjanvalur
      kristjanvalurmannequin
      on Oct 27, 2010
      MannequinAuthor

      When doing socket IO, it is beneficial to use a bytearra() and then using sock.recv_into() to avoid moving data about.
      However, many useful functions still don't accept new style buffers, such as the bytearray and memoryview. In particular, the struct module cannot unpack from them, and the StringIO doesn't accept them as input.

      The attached patch adds new-buffer support to the struct module and cStringIO.

    2. kristjanvalur
      added
      interpreter-core(Objects, Python, Grammar, and Parser dirs)
      type-bugAn unexpected behavior, bug, or error
      on Oct 27, 2010

    3. pitrou commented on Oct 27, 2010

      @pitrou
      pitrou
      on Oct 27, 2010
      Member

      You can't add buffer protocol support to cStringIO in a bugfix release, since it would be a new feature.

    4. merwok
      added
      extension-modulesC modules in the Modules dir
      and removed
      interpreter-core(Objects, Python, Grammar, and Parser dirs)
      on Oct 27, 2010

    5. kristjanvalur commented on Oct 28, 2010

      @kristjanvalur
      kristjanvalurmannequin
      on Oct 28, 2010
      MannequinAuthor

      I disagree. It's not a new feature. We're merely completing an old feature (adding new-style buffers from 3.x to 2.7) that wasn't fully implemented. by the core. The new buffer isn't accepted in a lot of places where you'd expect it to be.

      The good alternative, of course, would be to add this to "trunk" but for some reason that is frowned upon.

      See also bpo-10211

    6. merwok commented on Oct 28, 2010

      @merwok
      merwok
      on Oct 28, 2010
      Member

      MAL’s viewpoint from msg119721:

      “The memoryview object was added to simplify porting applications to Python3. If that backport is incomplete in the sense that the memoryview object is not compatible with the standard Python2 object for such memory views, then I'd consider that a bug.

      Without compatibility to the buffer objects, there's no way to make other Python2 buffer interface compatible object compatible to memoryviews.

      Alternatively, the Python2 memoryview object implementation could also accept objects with the old buffer interface, much like "s*" does.

      Note that the patch needs to check the buffer flags - writing to such buffers is not allowed.”

    7. terryjreedy commented on Oct 29, 2010

      @terryjreedy
      terryjreedy
      on Oct 29, 2010
      Member

      In the absence of doc references in this and bpo-10211 that would *clearly* settle the bug vs. feature issue, it strikes me as a bit murky. So I am inclined to agree with MAL that failure to achieve the stated, documented purpose is a bug. I strongly suspect that if these issues had been filed during the 2.7 beta phase, a fix would have been deemed permissible without much controversy.

    8. kristjanvalur commented on Mar 20, 2012

      @kristjanvalur
      kristjanvalurmannequin
      on Mar 20, 2012
      MannequinAuthor

      The patch is still there. Any new consensus?

    9. terryjreedy commented on Mar 20, 2012

      @terryjreedy
      terryjreedy
      on Mar 20, 2012
      Member

      Stefan, since you have been working on Memoryview, any opinions?

    10. kristjanvalur commented on Mar 18, 2013

      @kristjanvalur
      kristjanvalurmannequin
      on Mar 18, 2013
      MannequinAuthor

      We don't seem to be getting any traction on this. 2.7.4 is to be cut next weekend so it is becoming more urgent :)

    11. terryjreedy commented on Mar 18, 2013

      @terryjreedy
      terryjreedy
      on Mar 18, 2013
      Member

      For the backport policy question, since there seems to be disagreement, consider a post on pydev asking for discussion and if no quick consensus, a ruling from Guido or designee.

    12. tseaver commented on Mar 19, 2013

      @tseaver
      tseaver
      on Mar 19, 2013

      Patch looks good to me. I would call it a bugfix, not a feature.

    13. python-dev commented on Mar 20, 2013

      @python-dev
      python-devmannequin
      on Mar 20, 2013
      Mannequin

      New changeset be4bec689de3 by Kristján Valur Jónsson in branch '2.7':
      Issue bpo-10212: Support new buffer interface for struct.unpack and
      http://hg.python.org/cpython/rev/be4bec689de3

    14. kristjanvalur
      closed this as completedon Mar 22, 2013
    15. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      extension-modulesC modules in the Modules dirtype-bugAn unexpected behavior, bug, or error

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions