Skip to content

Use current locale for byte regular-expression matching - #9042

Merged
youknowone merged 2 commits into
RustPython:mainfrom
youknowdot:fix-regex-locale-matching
Oct 10, 2026
Merged

Use current locale for byte regular-expression matching#9042
youknowone merged 2 commits into
RustPython:mainfrom
youknowdot:fix-regex-locale-matching

Conversation

@youknowdot

@youknowdot youknowdot commented Oct 10, 2026 •
edited by coderabbitai Bot
Loading

Copy link
Copy Markdown
Contributor

Summary

Extract the Python 3.14-compatible byte regular-expression LOCALE matching fixes from #8954.

  • Use the current LC_CTYPE locale for byte classification and case conversion, including already-compiled patterns, boundaries, backreferences, and character sets.
  • Keep host callbacks outside the no_std engine and preserve the C-locale fallback.
  • Leave Python 3.15 prefixmatch additions out of this independent change.

Existing canonical regex tests cover this behavior; no duplicate snippet tests are added. The exact patched native interpreter unexpectedly passed test_locale_caching, so only its obsolete expected-failure decorator was removed. All canonical assertions and platform skips are preserved.

Validation

Before the executor reset, this exact commit passed the normal commit hooks, VM Clippy, and separate CAPI Clippy. CPython 3.14.7 passed the three selected canonical locale tests and a 39-case comparison matrix; the immutable RustPython baseline failed 13 matrix cases. These are reference/baseline results, not a claim of native runtime success for this head. After executor recovery, a fresh native build of the exact production commit succeeded. The existing test_locale_flag and test_locale_compiled passed; test_locale_caching unexpectedly passed. After removing only its obsolete decorator, all three canonical tests passed normally. Hosted cross-platform CI remains pending.

Remote recovery verified this commit's parent is current main (3af5f8a) and its production diff is limited to five Rust files, with one subsequent obsolete test decorator removed. No test assertions or expected-failure markers are weakened.

AI-assisted with OpenAI Codex. Draft pending platform validation.

Summary by CodeRabbit

  • New Features
    • Regular-expression matching now supports locale-aware character classification and case conversion for character sets, case-insensitive text, word boundaries, and related matching operations.
    • Host locale behavior is used where supported; other configurations retain the default matching behavior.

@youknowdot
Use current locale for byte regular-expression matching
785b1be 
Extract the Python 3.14-compatible LOCALE matching paths from RustPython#8954, keeping host callbacks outside the no_std engine and preserving the C-locale fallback. Existing canonical regex tests cover locale changes and compiled patterns. Keep expected-failure markers until patched native execution proves they are obsolete; omit duplicate snippet tests and Python 3.15 prefixmatch additions.

Assisted-by: OpenAI Codex:model identifier unavailable
@coderabbitai

coderabbitai Bot commented Oct 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration
  • Configuration used: Repository: RustPython/RustPython/.coderabbit.yml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 71658013-d46f-45a9-8d4f-14e3cf6687a2
📥 Commits

Reviewing files that changed from the base of the PR and between 3af5f8a and 2b6cbc0.

⛔ Files ignored due to path filters (1)
  • Lib/test/test_re.py is excluded by !Lib/**
📒 Files selected for processing (5)
  • crates/host_env/src/locale.rs
  • crates/sre_engine/src/engine.rs
  • crates/sre_engine/src/lib.rs
  • crates/sre_engine/src/string.rs
  • crates/vm/src/stdlib/_sre.rs
 ____________________________________
< No tests? Bold. Approval? Also no. >
 ------------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       / 　 づ
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@youknowdot
Enable the repaired locale caching regression
2b6cbc0 
The exact patched native interpreter passes the existing locale flag and compiled-pattern tests and unexpectedly passes locale caching. Remove only the obsolete expected-failure decorator; preserve all canonical assertions and platform skips.

Assisted-by: OpenAI Codex:model identifier unavailable
@github-actions

github-actions Bot commented Oct 10, 2026

Copy link
Copy Markdown
Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[x] lib: cpython/Lib/re
[x] lib: cpython/Lib/sre_compile.py
[x] lib: cpython/Lib/sre_constants.py
[x] lib: cpython/Lib/sre_parse.py
[ ] test: cpython/Lib/test/test_re.py (TODO: 4)
[x] test: cpython/Lib/test/re_tests.py

dependencies:

  • re

dependent tests: (82 tests)

  • re: test_android test_ast test_asyncio test_binascii test_builtin test_bytes test_clinic test_cmd_line test_ctypes test_dict test_dis test_docxmlrpc test_dtrace test_email test_embed test_faulthandler test_filecmp test_fileinput test_fnmatch test_format test_frame test_fstring test_functools test_future_stmt test_generated_cases test_genericalias test_glob test_hashlib test_http_cookiejar test_httplib test_httpservers test_imaplib test_importlib test_inspect test_ipaddress test_launcher test_logging test_mailbox test_mmap test_optparse test_ordered_dict test_platform test_pprint test_pydoc test_pyexpat test_pyrepl test_re test_regrtest test_runpy test_set test_site test_smtplib test_socket test_ssl test_strftime test_string test_strptime test_strtod test_structseq test_symtable test_syntax test_sys test_sysconfig test_tarfile test_tempfile test_tokenize test_tools test_traceback test_turtle test_typing test_unittest test_unparse test_venv test_webbrowser test_winapi test_with test_wsgiref test_xmlrpc test_zipfile test_zipimport test_zoneinfo test_zstd

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

@youknowone
youknowone marked this pull request as ready for review October 10, 2026 10:52
@youknowone
youknowone enabled auto-merge (squash) October 10, 2026 10:53
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 10, 2026 •
edited
Loading

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-10T11:01:22.027124Z 2b6cbc0 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@youknowone
youknowone merged commit fd062ca into RustPython:main Oct 10, 2026
29 of 30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@youknowone youknowone youknowone approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@youknowdot @youknowone