Skip to content

Keep coroutine generator expressions on the ordinary call path - #9075

Merged
youknowone merged 1 commit into
RustPython:mainfrom
youknowdot:fix-async-genexpr-call-optimization
Oct 10, 2026
Merged

youknowone merged 1 commit into
RustPython:mainfrom
youknowdot:fix-async-genexpr-call-optimization

Conversation

@youknowdot

@youknowdot youknowdot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Extract the Python 3.14 coroutine-generator guard from #8954: coroutine-marked generator expressions must use the ordinary call path rather than the guarded all/any/tuple loop optimization.

The literal mega guard reads the next symbol subtable, which can belong to a nested expression in the outer iterable. This patch instead reuses the existing cursor-preserving comprehension-table lookup and propagates lookup errors. The lookup is computed once; no runtime, opcode, symbol-table construction, or generator lifecycle changes are introduced.

This matches the guard already present in CPython 3.14.7 and 3.14.8. The Python 3.15 list/set optimization additions remain separate.

Validation

  • Independent source/diff review, including lookup cursor restoration, nested-scope selection, error propagation and open-PR overlap.
  • Update two existing structural IR expectations to require the ordinary path; retain CALL and existing synchronous outer-await/list/set controls. Add two compile-only nested-scope fixtures to the same test.
  • Ten ASCII compilation/disassembly cases pass on CPython 3.14.7 and 3.14.8: async-for/await-element cases, nested lambdas/generators, synchronous all/any/tuple, outermost await, and a synchronous outer genexpr over an async inner genexpr.
  • The prior native binary fails the async-case structural expectation. The patched native binary passes all ten cases. No generated function, coroutine, or generator is invoked.
  • Five focused structural codegen tests pass; normal hooks and native build pass; codegen/VM and separately configured C-API Clippy run with warnings denied.

This is compiler-shape compatibility, not a claimed crash repair or measured performance improvement. The full hosted matrix remains required.

Summary by CodeRabbit

  • Bug Fixes
    • Corrected how built-in tuple, all, and any calls handle generator expressions involving coroutines. These expressions now follow the ordinary call path when appropriate, while eligible non-coroutine expressions retain the optimized path. This improves correctness for asynchronous iteration and nested await expressions.

Match Python 3.14 by excluding coroutine-marked generator expressions from builtin call optimization. Reuse the existing cursor-preserving comprehension lookup so nested outer iterable scopes do not select the wrong symbol table; propagate lookup errors.

Assisted-by: OpenAI Codex:model identifier unavailable
@coderabbitai

coderabbitai Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: RustPython/RustPython/.coderabbit.yml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 72657961-7d2c-48c4-8bc2-81c7e68f9d30

📥 Commits

Reviewing files that changed from the base of the PR and between 9d6bf4d and 03a750b.


📒 Files selected for processing (1)
  • crates/codegen/src/compile.rs

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 4 remain after this review.



📝 Walkthrough

Walkthrough

Generator-expression calls to tuple, all, and any now use builtin optimization only when the generator is not marked as a coroutine. Tests cover coroutine generator expressions and the ordinary call path.

Changes

Generator-Call Optimization

Layer / File(s) Summary
Symbol-table-aware optimization and validation
crates/codegen/src/compile.rs
Call detection checks the generator’s symbol table for coroutine status before selecting builtin optimization. Coroutine generator expressions use the ordinary call path. Tests cover async iteration, awaited expressions, and nested generator or lambda iterators. An await only in the outermost iterator does not disqualify optimization.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Suggested reviewers: youknowone


Merge Risk: ⚪ Minimal · up to 03a75

No actionable issue is identified; the change is mergeable after normal checks.

Pre-merge checks | Passed 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: keeping coroutine generator expressions on the ordinary call path instead of the guarded optimization path.
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@youknowone
youknowone marked this pull request as ready for review October 10, 2026 14:36
@youknowone
youknowone enabled auto-merge (squash) October 10, 2026 14:36
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-10T14:40:33.463018Z 03a750b Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 03a750bbf0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

has_common_constant(f, bytecode::CommonConstant::BuiltinAny),
"maybe_optimize_function_call() guards any(genexpr) whether or not the \
generator is a coroutine: {source}"
!has_common_constant(f, bytecode::CommonConstant::BuiltinAny),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Do not flip the existing unit-test assertions

This block reverses the existing structural expectations and the same test adds new source fixtures, but the repository’s test-code policy explicitly prohibits modifying assertions, logic, or test data; this behavior change needs a policy-compliant test strategy or an explicit exception before landing.

AGENTS.md reference: AGENTS.md:L273-L279

Useful? React with 👍 / 👎.

@youknowone
youknowone merged commit 1cbae85 into RustPython:main Oct 10, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants