Skip to content

sending non-None values makes generator raise StopIteration on next access #90167

Closed
Closed
sending non-None values makes generator raise StopIteration on next access#90167
Labels
3.10 (EOL)end of life3.11only security fixesrelease-blockertype-bugAn unexpected behavior, bug, or error

Description

@Zabolekar
Zabolekar
mannequin
opened on Dec 7, 2021
BPO 46009
Nosy @tiran, @markshannon, @pablogsal, @brandtbucher, @Zabolekar
PRs
  • bpo-46009: Do not exhaust generator when send() method raises #29986
  • [3.10] bpo-46009: Do not exhaust generator when send() method raises (GH-29986). #29988
  • bpo-46009: Remove GEN_START #30367
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2022-01-05.16:02:13.606>
created_at = <Date 2021-12-07.21:53:29.521>
labels = ['type-bug', 'release-blocker', '3.10', '3.11']
title = 'sending non-None values makes generator raise StopIteration on next access'
updated_at = <Date 2022-01-05.16:02:13.606>
user = 'https://github.com/Zabolekar'

    bugs.python.org fields:

    activity = <Date 2022-01-05.16:02:13.606>
actor = 'christian.heimes'
assignee = 'none'
closed = True
closed_date = <Date 2022-01-05.16:02:13.606>
closer = 'christian.heimes'
components = []
creation = <Date 2021-12-07.21:53:29.521>
creator = 'Zabolekar'
dependencies = []
files = []
hgrepos = []
issue_num = 46009
keywords = ['patch']
message_count = 7.0
messages = ['407975', '408013', '408021', '409700', '409770', '409776', '409781']
nosy_count = 5.0
nosy_names = ['christian.heimes', 'Mark.Shannon', 'pablogsal', 'brandtbucher', 'Zabolekar']
pr_nums = ['29986', '29988', '30367']
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue46009'
versions = ['Python 3.10', 'Python 3.11']

    Activity

    1. Zabolekar commented on Dec 7, 2021

      @Zabolekar
      Zabolekarmannequin
      on Dec 7, 2021
      MannequinAuthor

      Tested with Python 3.10.1 on Linux and Python 3.10.0 on Windows.

      The following code prints None in 3.9 and raises StopIteration without any additional information in 3.10:

          def f():
        yield
    
    x = f()
    
    try:
        x.send(0)
    except TypeError as e:
        print(e) # can't send non-None value to a just-started generator
    
    print(next(x))
    2. Zabolekar
      added
      3.10 (EOL)end of life
      type-bugAn unexpected behavior, bug, or error
      on Dec 7, 2021
    3. pablogsal
      added
      release-blocker
      on Dec 7, 2021

    4. markshannon commented on Dec 8, 2021

      @markshannon
      markshannon
      on Dec 8, 2021
      Member

      New changeset 69806b9 by Mark Shannon in branch 'main':
      bpo-46009: Do not exhaust generator when send() method raises (GH-29986)
      69806b9

    5. pablogsal commented on Dec 8, 2021

      @pablogsal
      pablogsal
      on Dec 8, 2021
      Member

      New changeset 99c7232 by Mark Shannon in branch '3.10':
      [3.10] bpo-46009: Do not exhaust generator when send() method raises (GH-29986). (GH-29988)
      99c7232

    6. pablogsal
      closed this as completedon Dec 8, 2021
    7. pablogsal
      closed this as completedon Dec 8, 2021

    8. brandtbucher commented on Jan 4, 2022

      @brandtbucher
      brandtbucher
      on Jan 4, 2022
      Member

      New changeset 31e43cb by Brandt Bucher in branch 'main':
      bpo-46009: Remove GEN_START (GH-30367)
      31e43cb

    9. tiran commented on Jan 5, 2022

      @tiran
      tiran
      on Jan 5, 2022
      Member

      #74552 broke Emscripten WASM builds. I'm getting a "null function or function signature mismatch" error from the WASM engine:

      RuntimeError: null function or function signature mismatch
      at _PyEval_EvalFrameDefault (http://localhost:8000/python.wasm:wasm-function[2383]:0x103a6f)
      at gen_send_ex2 (http://localhost:8000/python.wasm:wasm-function[886]:0x4f35d)
      at gen_iternext (http://localhost:8000/python.wasm:wasm-function[893]:0x4fb1a)
      at builtin_any (http://localhost:8000/python.wasm:wasm-function[3927]:0x198a87)
      at cfunction_vectorcall_O (http://localhost:8000/python.wasm:wasm-function[403]:0x201ae)
      at PyObject_Vectorcall (http://localhost:8000/python.wasm:wasm-function[2426]:0x109dbe)
      at _PyEval_EvalFrameDefault (http://localhost:8000/python.wasm:wasm-function[2383]:0x1062ba)
      at _PyEval_Vector (http://localhost:8000/python.wasm:wasm-function[2380]:0xfc366)
      at PyEval_EvalCode (http://localhost:8000/python.wasm:wasm-function[2379]:0xfc014)
      at _PyConfig_InitPathConfig (http://localhost:8000/python.wasm:wasm-function[6871]:0x2924b0)

      I'm trying to get more debug information now.

    10. tiran
      reopened this on Jan 5, 2022
    11. tiran
      reopened this on Jan 5, 2022

    12. tiran commented on Jan 5, 2022

      @tiran
      tiran
      on Jan 5, 2022
      Member

      I got more debug symbols:

      python.js:235 Uncaught RuntimeError: null function or function signature mismatch
      at _PyEval_EvalFrameDefault (ceval.c:4247)
      at _PyEval_EvalFrame (pycore_ceval.h:48)
      at gen_send_ex2 (genobject.c:219)
      at gen_iternext (genobject.c:583)
      at builtin_any (bltinmodule.c:384)
      at cfunction_vectorcall_O (methodobject.c:516)
      at _PyObject_VectorcallTstate.11 (pycore_call.h:89)
      at PyObject_Vectorcall (call.c:298)
      at _PyEval_EvalFrameDefault (ceval.c:4625)
      at _PyEval_EvalFrame.1 (pycore_ceval.h:48)

      The culprit is

         PyObject *next = (*Py_TYPE(iter)->tp_iternext)(iter);

      in the TARGET(FOR_ITER) target. An additional NULL check resolves the problem.

      index 953876f6226..390400b3246 100644
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -4244,7 +4244,12 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr
             PREDICTED(FOR_ITER);
             /* before: [iter]; after: [iter, iter()] *or* [] */
             PyObject *iter = TOP();
-            PyObject *next = (*Py_TYPE(iter)->tp_iternext)(iter);
+            PyObject *(*iternext)(PyObject *) = *Py_TYPE(iter)->tp_iternext;
+            if (iternext == NULL) {
+                PyErr_BadInternalCall();
+                goto error;
+               }
+            PyObject *next = iternext(iter);
             if (next != NULL) {
                 PUSH(next);
                 PREDICT(STORE_FAST);
    13. tiran
      added
      3.11only security fixes
      on Jan 5, 2022

    14. tiran commented on Jan 5, 2022

      @tiran
      tiran
      on Jan 5, 2022
      Member

      False alarm ... the frozen files for getpath.py were out of date. I can no longer reproduce the issue after a hard git clean and rebuild.

    15. tiran
      closed this as completedon Jan 5, 2022
    16. tiran
      closed this as completedon Jan 5, 2022
    17. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.10 (EOL)end of life3.11only security fixesrelease-blockertype-bugAn unexpected behavior, bug, or error

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions