Skip to content

Refactor PyUtf8Str - #6047

Merged
youknowone merged 3 commits into
RustPython:mainfrom
youknowone:wtf8
Jul 30, 2025
Merged

youknowone merged 3 commits into
RustPython:mainfrom
youknowone:wtf8

Conversation

@youknowone

@youknowone youknowone commented Jul 29, 2025 •

Copy link
Copy Markdown
Member

cc @ever0de

I am going to add a new validation step for PyUtf8Str. This is useful for sqlite3

Summary by CodeRabbit

  • New Features

    • Introduced a distinct UTF-8 string type for Python strings, enabling stricter UTF-8 validation and handling.
    • Added new methods for safe downcasting and UTF-8 specific string representations in object and protocol handling.
    • Expanded public interfaces to include UTF-8 string types.
    • Added a macro for retrieving UTF-8 validated identifiers.
  • Bug Fixes

    • Improved error handling for string formatting and output involving invalid UTF-8 or surrogate characters.
    • Enhanced string comparison and attribute handling to support both UTF-8 and WTF-8 encoded strings.
  • Refactor

    • Updated numerous function signatures and internal logic to require or return UTF-8 validated string references, especially in I/O, database, and standard library modules.
    • Simplified and standardized string conversions and representations throughout the system.
    • Refined downcasting mechanisms with VM-based validation for safer type conversions.
  • Style

    • Streamlined code for string formatting, representation, and output for consistency and maintainability.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant