Skip to content

[Windows] New REPL doesn't allow to input non-ASCII Unicode characters #130804

Closed
Closed
[Windows] New REPL doesn't allow to input non-ASCII Unicode characters#130804
Labels
3.13only security fixes3.14bugs and security fixesOS-windowsstdlibStandard Library Python modules in the Lib/ directorytopic-replRelated to the interactive shelltype-bugAn unexpected behavior, bug, or error

Description

@sergey-miryanov
sergey-miryanov
opened on Mar 3, 2025

Bug report

Bug description:

I got following error when I tried to type Cyrillic characters (tried ч in this example) in the new repl:

.\python.bat
Running Release|x64 interpreter...
Python 3.14.0a5+ (heads/main-dirty:d0eb01c9de9, Mar  3 2025, 23:39:38) [MSC v.1942 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> x
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    x
NameError: name 'x' is not defined
>>> Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "D:\Sources\_pythonish\cpython\Lib\_pyrepl\__main__.py", line 6, in <module>
    __pyrepl_interactive_console()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "D:\Sources\_pythonish\cpython\Lib\_pyrepl\main.py", line 59, in interactive_console
    run_multiline_interactive_console(console)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "D:\Sources\_pythonish\cpython\Lib\_pyrepl\simple_interact.py", line 143, in run_multiline_interactive_console
    statement = multiline_input(more_lines, ps1, ps2)
  File "D:\Sources\_pythonish\cpython\Lib\_pyrepl\readline.py", line 389, in multiline_input
    return reader.readline()
           ~~~~~~~~~~~~~~~^^
  File "D:\Sources\_pythonish\cpython\Lib\_pyrepl\reader.py", line 802, in readline
    self.handle1()
    ~~~~~~~~~~~~^^
  File "D:\Sources\_pythonish\cpython\Lib\_pyrepl\reader.py", line 758, in handle1
    event = self.console.get_event(block=False)
  File "D:\Sources\_pythonish\cpython\Lib\_pyrepl\windows_console.py", line 472, in get_event
    self.event_queue.push(rec.Event.KeyEvent.uChar.UnicodeChar)
    ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Sources\_pythonish\cpython\Lib\_pyrepl\base_eventqueue.py", line 77, in push
    char = bytes(bytearray((ord_char,)))
                 ~~~~~~~~~^^^^^^^^^^^^^
ValueError: byte must be in range(0, 256)

CPython versions tested on:

CPython main branch

Operating systems tested on:

Windows

Linked PRs

Activity

  1. sergey-miryanov
    added
    type-bugAn unexpected behavior, bug, or error
    on Mar 3, 2025

  2. picnixz commented on Mar 3, 2025

    @picnixz
    picnixz
    on Mar 3, 2025
    Member

    Did it previously worked in 3.12 or with the old repl?

  3. picnixz
    added
    stdlibStandard Library Python modules in the Lib/ directory
    topic-replRelated to the interactive shell
    on Mar 3, 2025
  4. picnixz
    changed the title [-]New REPL doesn't allow unicode to input unicode characters[/-] [+]New REPL doesn't allow to input non-ASCII Unicode characters[/+] on Mar 3, 2025
  5. picnixz
    added
    3.13only security fixes
    3.14bugs and security fixes
    on Mar 3, 2025

  6. sergey-miryanov commented on Mar 3, 2025

    @sergey-miryanov
    sergey-miryanov
    on Mar 3, 2025
    ContributorAuthor

    Did it previously worked in 3.12 or with the old repl?

    Will check on 3.12, it definitely worked on 3.8 repl.

    Python 3.8.10 (tags/v3.8.10:3d8993a, May  3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> ч
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'ч' is not defined

  7. sergey-miryanov commented on Mar 3, 2025

    @sergey-miryanov
    sergey-miryanov
    on Mar 3, 2025
    ContributorAuthor

    It works on 3.12 and 3.13

    Running Release|x64 interpreter...
Python 3.12.9+ (heads/3.12-dirty:3614649714a, Mar  4 2025, 00:02:21) [MSC v.1942 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> x
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'x' is not defined
>>> ч
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'ч' is not defined

    Python 3.13.1 (tags/v3.13.1:0671451, Dec  3 2024, 19:06:28) [MSC v.1942 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> x
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    x
NameError: name 'x' is not defined
>>> ч
Traceback (most recent call last):
  File "<python-input-1>", line 1, in <module>
    ч
NameError: name 'ч' is not defined

  8. StanFromIreland commented on Mar 3, 2025

    @StanFromIreland
    StanFromIreland
    on Mar 3, 2025
    Member

    It doesn't occur for me?

    Python 3.14.0a5+ (heads/add-heapq-max-dirty:ddc27f9c385, Mar  1 2025, 14:42:30) [GCC 14.2.1 20250110 (Red Hat 14.2.1-7)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> ч
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    ч
NameError: name 'ч' is not defined
>>> exit

  9. sergey-miryanov commented on Mar 3, 2025

    @sergey-miryanov
    sergey-miryanov
    on Mar 3, 2025
    ContributorAuthor

    @StanFromIreland it seems a windows related.

  10. picnixz
    added
    OS-windows
    on Mar 3, 2025

  11. gvanrossum commented on Mar 3, 2025

    @gvanrossum
    gvanrossum
    on Mar 3, 2025
    Member

    That means it could also be code-page or locale related.

  12. zooba commented on Mar 3, 2025

    @zooba
    zooba
    on Mar 3, 2025
    Member

    We've bypassed the code page for the default REPL since 3.6, so it's almost certainly due to something in the new REPL. Possibly it's gone back to using codepages instead of Unicode?

  13. sergey-miryanov commented on Mar 3, 2025

    @sergey-miryanov
    sergey-miryanov
    on Mar 3, 2025
    ContributorAuthor

    I believe it is from @y5c4l3 a65366e#diff-77cdde1442e2d3a5fe5ee852304d1af6d0a2033e718dc588552d61596cb72100R472

  14. sergey-miryanov
    added a commit that references this issue on Mar 4, 2025

    Merge branch 'main' into pythongh-130804-pyrepl-unicode-chars

    497a0bb
  15. vstinner
    changed the title [-]New REPL doesn't allow to input non-ASCII Unicode characters[/-] [+][Windows] New REPL doesn't allow to input non-ASCII Unicode characters[/+] on Mar 5, 2025
  16. sergey-miryanov
    added a commit that references this issue on Mar 10, 2025

    Merge branch 'main' into pythongh-130804-pyrepl-unicode-chars

    c667049
  17. pablogsal
    added a commit that references this issue on Mar 10, 2025

    gh-130804: Fix support of typing unicode chars in pyrepl (#130805)

    7c98b06
  18. sergey-miryanov
    closed this as completedon Mar 25, 2025

  19. StanFromIreland commented on Mar 29, 2025

    @StanFromIreland
    StanFromIreland
    on Mar 29, 2025
    Member

    It seems this was not fully resolved... #131878

  20. chris-eibl
    added a commit that references this issue on Apr 26, 2025

    Revert "pythongh-130804: Fix support of typing unicode chars in pyrepl (

    a06c42a
  21. ambv
    added 2 commits that reference this issue on May 5, 2025

    [3.13] pythongh-130804: Fix support of typing unicode chars in pyrepl (…

    e980f8b

    [3.13] gh-130804: Fix support of typing unicode chars in pyrepl (GH-1…

    76f52c4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.13only security fixes3.14bugs and security fixesOS-windowsstdlibStandard Library Python modules in the Lib/ directorytopic-replRelated to the interactive shelltype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions