Skip to content

3.13.0 REPL crashes on double ctrl+z in line overflow #126332

Closed
Closed
3.13.0 REPL crashes on double ctrl+z in line overflow#126332
Labels
3.13only security fixes3.14bugs and security fixesOS-windowstopic-replRelated to the interactive shelltype-bugAn unexpected behavior, bug, or error

Description

@base-13
base-13
opened on Nov 2, 2024

Bug report

Bug description:

The new REPL in 3.13.0 crashes on pressing ctrl+Z twice at the end of a overflowing line, refer to the attachment

2024-11-02.17-22-35.mov

CPython versions tested on:

3.13

Operating systems tested on:

Windows

Linked PRs

Activity

  1. base-13
    added
    type-bugAn unexpected behavior, bug, or error
    on Nov 2, 2024
  2. picnixz
    added
    OS-windows
    topic-replRelated to the interactive shell
    3.13only security fixes
    3.14bugs and security fixes
    on Nov 2, 2024

  3. base-13 commented on Nov 2, 2024

    @base-13
    base-13
    on Nov 2, 2024
    Author

    I found even more weird things which happens when handing line overflows

    2024-11-02.17-36-48.mov

  4. JamesParrott commented on Nov 2, 2024

    @JamesParrott
    JamesParrott
    on Nov 2, 2024
    Contributor

    Something similar happens with a numeric literal:

    Double Ctrl+Z

    Python 3.13.0 (tags/v3.13.0:60403a5, Oct  7 2024, 09:38:07) [MSC v.1941 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> 1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111\11111111111111111111^ZTraceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\__main__.py", line 6, in <module>
    __pyrepl_interactive_console()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\main.py", line 59, in interactive_console
    run_multiline_interactive_console(console)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\simple_interact.py", line 151, in run_multiline_interactive_console
    statement = multiline_input(more_lines, ps1, ps2)
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\readline.py", line 389, in multiline_input
    return reader.readline()
           ~~~~~~~~~~~~~~~^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\reader.py", line 801, in readline
    self.handle1()
    ~~~~~~~~~~~~^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\reader.py", line 784, in handle1
    self.do_cmd(cmd)
    ~~~~~~~~~~~^^^^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\reader.py", line 714, in do_cmd
    self.refresh()
    ~~~~~~~~~~~~^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\reader.py", line 692, in refresh
    self.console.refresh(self.screen, self.cxy)
    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\windows_console.py", line 191, in refresh
    self.__write_changed_line(y, oldline, newline, px)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\windows_console.py", line 238, in __write_changed_line
    and oldline[x_pos] == newline[x_pos]
        ~~~~~~~^^^^^^^
IndexError: string index out of range

    I'm not sure what the intended behaviour is with overflowed lines in the command history, but it does strike me as odd too.

  5. base-13 commented on Nov 2, 2024

    @base-13
    base-13
    on Nov 2, 2024
    Author

    Something similar happens with a numeric literal:

    Double Ctrl+Z

    Python 3.13.0 (tags/v3.13.0:60403a5, Oct  7 2024, 09:38:07) [MSC v.1941 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> 1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111\11111111111111111111^ZTraceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\__main__.py", line 6, in <module>
    __pyrepl_interactive_console()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\main.py", line 59, in interactive_console
    run_multiline_interactive_console(console)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\simple_interact.py", line 151, in run_multiline_interactive_console
    statement = multiline_input(more_lines, ps1, ps2)
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\readline.py", line 389, in multiline_input
    return reader.readline()
           ~~~~~~~~~~~~~~~^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\reader.py", line 801, in readline
    self.handle1()
    ~~~~~~~~~~~~^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\reader.py", line 784, in handle1
    self.do_cmd(cmd)
    ~~~~~~~~~~~^^^^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\reader.py", line 714, in do_cmd
    self.refresh()
    ~~~~~~~~~~~~^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\reader.py", line 692, in refresh
    self.console.refresh(self.screen, self.cxy)
    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\windows_console.py", line 191, in refresh
    self.__write_changed_line(y, oldline, newline, px)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\windows_console.py", line 238, in __write_changed_line
    and oldline[x_pos] == newline[x_pos]
        ~~~~~~~^^^^^^^
IndexError: string index out of range

    I'm not sure what the intended behaviour is with overflowed lines in the command history, but it does strike me as odd too.

    the expression is not the problem, how the REPL is handling overflowing lines is the problem, so it doesn't matter whether its a number literal or a identifier

  6. eendebakpt commented on Nov 2, 2024

    @eendebakpt
    eendebakpt
    on Nov 2, 2024
    Contributor

    I can confirm the issue in current main. In the __write_changed_line method there is a guard for the while loop using x_coord:

            while (
            x_coord < minlen
            and oldline[x_pos] == newline[x_pos]
            and newline[x_pos] != "\x1b"
        ):
            x_coord += wlen(newline[x_pos])
            x_pos += 1

    The wlen method has some special casing for ctrl-z, but that is not taken into account when the length of a string is 1:

    def wlen(s: str) -> int:
    if len(s) == 1:
        return str_width(s)
    length = sum(str_width(i) for i in s)
    # remove lengths of any escape sequences
    sequence = ANSI_ESCAPE_SEQUENCE.findall(s)
    ctrl_z_cnt = s.count('\x1a')
    return length - sum(len(i) for i in sequence) + ctrl_z_cnt

    The following diff avoids the crash:

    diff --git a/Lib/_pyrepl/utils.py b/Lib/_pyrepl/utils.py
index 0f36083b6ff..4651717bd7e 100644
--- a/Lib/_pyrepl/utils.py
+++ b/Lib/_pyrepl/utils.py
@@ -16,7 +16,7 @@ def str_width(c: str) -> int:


 def wlen(s: str) -> int:
-    if len(s) == 1:
+    if len(s) == 1 and s != '\x1a':
         return str_width(s)
     length = sum(str_width(i) for i in s)
     # remove lengths of any escape sequences

  7. eendebakpt commented on Nov 2, 2024

    @eendebakpt
    eendebakpt
    on Nov 2, 2024
    Contributor

    @pablogsal @godlygeek The optimization in wlen for the single character case was added in #120253. Should the special casing for CTRL-Z be taken into account there?

  8. pablogsal commented on Jan 21, 2025

    @pablogsal
    pablogsal
    on Jan 21, 2025
    Member

    Should the special casing for CTRL-Z be taken into account there?

    I think so. @eendebakpt do you mind creating a PR with the fix?

  9. eendebakpt commented on Jan 21, 2025

    @eendebakpt
    eendebakpt
    on Jan 21, 2025
    Contributor

    @pablogsal PR is already open: #126650

  10. pablogsal
    added a commit that references this issue on Jan 21, 2025

    gh-126332: Fix pyrepl crash for double ctrl-z in line overflow (#126650)

    d147e5e
  11. pablogsal
    closed this as completedon Jan 21, 2025
  12. miss-islington
    added a commit that references this issue on Jan 21, 2025

    pythongh-126332: Fix pyrepl crash for double ctrl-z in line overflow (p…

    91eb997
  13. pablogsal
    added a commit that references this issue on Jan 21, 2025

    [3.13] gh-126332: Fix pyrepl crash for double ctrl-z in line overflow (…

    0c70453
  14. vstinner
    added a commit that references this issue on Feb 21, 2025

    gh-126332: Add tests for _pyrepl.utils (#129325)

    0c4248f
  15. miss-islington
    added a commit that references this issue on Feb 21, 2025

    pythongh-126332: Add tests for _pyrepl.utils (pythonGH-129325)

    20e2186
  16. vstinner
    added a commit that references this issue on Feb 21, 2025

    [3.13] gh-126332: Add tests for _pyrepl.utils (GH-129325) (#130414)

    3d88c18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.13only security fixes3.14bugs and security fixesOS-windowstopic-replRelated to the interactive shelltype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions