Skip to content

Match common encoding names before normalizing them - #8862

Merged
youknowone merged 1 commit into
RustPython:mainfrom
1ndahous3:codec_name_fast_match
Sep 27, 2026
Merged

Match common encoding names before normalizing them#8862
youknowone merged 1 commit into
RustPython:mainfrom
1ndahous3:codec_name_fast_match

Conversation

@1ndahous3

@1ndahous3 1ndahous3 commented Sep 27, 2026 •
edited by coderabbitai Bot
Loading

Copy link
Copy Markdown
Contributor

Summary

bytes.decode() and str.encode() select the built-in UTF-8, ASCII or Latin-1 codec through FastCodec::classify. Normalizing the default utf-8 spelling allocates a new string on each call before searching the alias lists.

Match utf-8, utf8, UTF-8, ascii, latin-1, latin1 and iso-8859-1 directly before normalization. These spellings retain their existing codec selection; other names follow the existing normalization and lookup path.

Follow-up to #8737, which introduced the native codec fast paths.

Performance

Windows x64 release build; medians of five samples of 200,000 calls.

Operation Before After Speedup
Default str.encode() 182.19 ns 164.86 ns 1.11×
bytes.decode("latin-1") 322.01 ns 285.09 ns 1.13×
str.encode("latin-1") 214.28 ns 187.62 ns 1.14×

AI assistance

Written with Claude Code (claude-opus-5-5) and Codex (GPT-6), reviewed by a human before submission.

Summary by CodeRabbit

  • Bug Fixes
    • Encoding and decoding now consistently recognize common UTF-8, ASCII, and Latin-1 codec spellings, including alternate capitalization and punctuation.
    • Encoding and decoding errors identify these codecs using canonical names.

@1ndahous3
Match common encoding names before normalizing them
d23a999 
Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Codex:GPT-6
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 27, 2026

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

@coderabbitai

coderabbitai Bot commented Sep 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: RustPython/RustPython/.coderabbit.yml

Review profile: CHILL

Plan: Advanced

Run ID: 7ffefc48-e75e-4623-8ad4-b9fd321e3e40

📥 Commits

Reviewing files that changed from the base of the PR and between f871707 and d23a999.

📒 Files selected for processing (2)
  • crates/vm/src/codecs.rs
  • extra_tests/snippets/builtin_str_encode.py

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

📝 Walkthrough

Walkthrough

FastCodec now directly recognizes seven common UTF-8, ASCII, and Latin-1 spellings before normalization. Added tests check encoding and decoding results, plus canonical codec names in errors.

Changes

Codec spelling recognition

Layer / File(s) Summary
Codec classification and test coverage
crates/vm/src/codecs.rs, extra_tests/snippets/builtin_str_encode.py		 FastCodec::classify adds direct matches for seven common encoding spellings. Tests check equivalent encoding and decoding behavior, and canonical codec names in errors.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Refactor

Merge Risk: ⚪ Minimal · up to d23a9

The recognized spellings retain their existing encode and decode behavior; this change only avoids normalization work, so no material merge risk is apparent.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 2 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: matching common encoding names before normalization.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codspeed

codspeed Bot commented Sep 27, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 62 untouched benchmarks
⏩ 4 skipped benchmarks1

Comparing 1ndahous3:codec_name_fast_match (d23a999) with main (f871707)

Open in CodSpeed

Footnotes

  1. 4 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@youknowone
youknowone enabled auto-merge (squash) September 27, 2026 15:08
@youknowone
youknowone merged commit 456a8dc into RustPython:main Sep 27, 2026
31 checks passed
@1ndahous3
1ndahous3 deleted the codec_name_fast_match branch September 27, 2026 17:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@youknowone youknowone youknowone approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@1ndahous3 @youknowone