Skip to content

Add symtable.py from 3.13.5 #6048

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
youknowone merged 11 commits into from
Aug 1, 2025
Merged

Add symtable.py from 3.13.5 #6048

Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
c87884a
Add `symtable.py` from 3.13.5
ShaharNaveh Jul 28, 2025
32d63cc
Update symtable methods
ShaharNaveh Jul 29, 2025
09a7fcf
Mark failing tests
ShaharNaveh Jul 29, 2025
cabfd73
Merge remote-tracking branch 'upstream/main' into update-symtable
ShaharNaveh Jul 30, 2025
796b335
Apply review suggestions
ShaharNaveh Jul 30, 2025
570fb54
Merge remote-tracking branch 'upstream/main' into update-symtable
ShaharNaveh Jul 30, 2025
16f7737
Correct `symbols` attr
ShaharNaveh Jul 30, 2025
433081b
Merge remote-tracking branch 'upstream/main' into update-symtable
ShaharNaveh Jul 31, 2025
75981e3
Readd `nested` attrs
ShaharNaveh Aug 1, 2025
e66d264
Correct `type` return type
ShaharNaveh Aug 1, 2025
8066fe5
Merge remote-tracking branch 'upstream/main' into update-symtable
ShaharNaveh Aug 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update symtable methods
  • Loading branch information
@ShaharNaveh
ShaharNaveh committed Jul 29, 2025
commit 32d63cc2f490acf7a0cb1e4e7950d4c433ff6df8
106 changes: 47 additions & 59 deletions vm/src/stdlib/symtable.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,7 @@ mod symtable {
use crate::{
PyObjectRef, PyPayload, PyRef, PyResult, VirtualMachine, builtins::PyStrRef, compiler,
};
use rustpython_codegen::symboltable::{
CompilerScope, Symbol, SymbolFlags, SymbolScope, SymbolTable,
};
use rustpython_codegen::symboltable::{Symbol, SymbolFlags, SymbolScope, SymbolTable};
use std::fmt;

// Consts as defined at
Expand Down Expand Up @@ -79,6 +77,30 @@ mod symtable {
#[pyattr]
pub const GENERATOR_EXPRESSION: i32 = 2;

#[pyattr]
pub const SCOPE_OFF: i32 = SCOPE_OFFSET;

#[pyattr]
pub const TYPE_FUNCTION: i32 = 0;

#[pyattr]
pub const TYPE_CLASS: i32 = 1;

#[pyattr]
pub const TYPE_MODULE: i32 = 2;

#[pyattr]
pub const TYPE_ANNOTATION: i32 = 3;

#[pyattr]
pub const TYPE_TYPE_VAR_BOUND: i32 = 4;

#[pyattr]
pub const TYPE_TYPE_ALIAS: i32 = 5;

#[pyattr]
pub const TYPE_TYPE_PARAM: i32 = 6;

#[pyfunction]
fn symtable(
source: PyStrRef,
Expand Down Expand Up @@ -117,57 +139,39 @@ mod symtable {

#[pyclass]
impl PySymbolTable {
#[pymethod]
fn get_name(&self) -> String {
#[pygetset]
fn name(&self) -> String {
self.symtable.name.clone()
}

#[pymethod]
fn get_type(&self) -> String {
#[pygetset(name = "type")]
fn typ(&self) -> String {
self.symtable.typ.to_string()
}
Comment on lines +154 to 163

@coderabbitai coderabbitai Bot Aug 1, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Verify integer constant mapping for type property.

The typ() method now returns integer constants instead of strings, which is a significant interface change. The mapping looks reasonable, but the fallback to -1 for unhandled cases may not align with CPython's behavior.

🏁 Script executed:

#!/bin/bash
# Check CPython's SymbolTable.get_type() return values
python3 -c "
import symtable
st = symtable.symtable('def f(): pass', '<string>', 'exec')
print('Module type:', repr(st.get_type()))
for child in st.get_children():
    print('Function type:', repr(child.get_type()))
"

Length of output: 297

Ensure type property matches CPython’s string-based API

CPython’s SymbolTable.get_type() returns strings like "module" and "function", not integer codes. Our Rust typ() currently returns integer constants and falls back to -1 for any unhandled cases, which:

  • Breaks compatibility with existing users expecting string values
  • Omits several type names (e.g. comprehensions, lambdas, generators) that CPython exposes

Please update the implementation to either:

  • Return Python strings matching CPython’s names, or
  • If you intentionally switch to integers, document the change clearly and replace the -1 fallback with a dedicated TYPE_UNKNOWN constant—and add mappings for all types in CPython’s C implementation.

Points to address:

  • Decide on integer vs. string return type and update docs accordingly
  • Remove the // TODO and implement mappings for missing CPython types (generator, lambda, comprehension, etc.)
  • Eliminate the -1 fallback or replace it with a well-defined default constant
🤖 Prompt for AI Agents 
In vm/src/stdlib/symtable.rs lines 154 to 163, the typ() function currently
returns integer constants and uses -1 as a fallback, which breaks compatibility
with CPython's string-based SymbolTable.get_type() API and omits several type
names. To fix this, change the return type to Python strings matching CPython's
type names (e.g., "module", "function", "class", "generator", "lambda",
"comprehension"), remove the -1 fallback, and implement mappings for all missing
CPython types. Also update any related documentation to reflect the string
return type and remove the TODO comment.


#[pymethod]
const fn get_lineno(&self) -> u32 {
#[pygetset]
const fn lineno(&self) -> u32 {
self.symtable.line_number
}

#[pymethod]
const fn is_nested(&self) -> bool {
self.symtable.is_nested
}

#[pymethod]
fn is_optimized(&self) -> bool {
matches!(
self.symtable.typ,
CompilerScope::Function | CompilerScope::AsyncFunction
)
#[pygetset]
fn children(&self, vm: &VirtualMachine) -> PyResult<Vec<PyObjectRef>> {
let children = self
.symtable
.sub_tables
.iter()
.map(|t| to_py_symbol_table(t.clone()).into_pyobject(vm))
.collect();
Ok(children)
}

#[pymethod]
fn lookup(&self, name: PyStrRef, vm: &VirtualMachine) -> PyResult<PyRef<PySymbol>> {
let name = name.as_str();
if let Some(symbol) = self.symtable.symbols.get(name) {
Ok(PySymbol {
symbol: symbol.clone(),
namespaces: self
.symtable
.sub_tables
.iter()
.filter(|table| table.name == name)
.cloned()
.collect(),
is_top_scope: self.symtable.name == "top",
}
.into_ref(&vm.ctx))
} else {
Err(vm.new_key_error(vm.ctx.new_str(format!("lookup {name} failed")).into()))
}
#[pygetset]
fn id(&self) -> usize {
self as *const Self as *const std::ffi::c_void as usize
}
Comment on lines +181 to 184

@coderabbitai coderabbitai Bot Jul 30, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Consider security implications of exposing memory addresses.

The id property exposes raw memory addresses, which could potentially be used for memory layout attacks or information disclosure in security-sensitive contexts.

Consider whether this aligns with CPython's behavior and if there are security implications for your use case:

🌐 Web query:

What does Python's symtable SymbolTable.id property return and are there security considerations?

💡 Result:

The id property (retrievable via get_id()) of Python's symtable.SymbolTable returns the table’s identifier, which is an internal value associated with that specific symbol table instance[1][2][5][6]. This identifier is primarily intended for internal use, such as distinguishing between different symbol table objects generated during code analysis.

Details:

  • What does get_id() return?

    • The value returned by SymbolTable.get_id() is an identifier for that block's symbol table instance. It does not return the name of the function, class, or module—that is provided by get_name(). Instead, it gives an ID unique to the table, but published documentation does not specify if this is guaranteed to be globally unique, stable between runs, or even meaningful outside the context of the current symbol table introspection session[1][2][5][6].
    • Source-level guides and sample code (see [6]) show the use of get_id() alongside type and name to help programmers identify and differentiate symbol tables while traversing nested scopes.

  • Security considerations:

    • There is no evidence in official documentation that the id or the value returned by get_id() contains sensitive information such as memory addresses, code contents, or pointers to user data[1][2][5][6].
    • However, since this is an internal identifier of a compiler structure, it is conceivable (though not specified) that in some Python implementations or versions, this ID could reveal patterns such as traversal order, or, hypothetically, even leak implementation details if misused.
    • In regular usage (introspecting Python source structures for tooling or analysis), exposure of the symbol table ID should not represent a direct security vulnerability, but best practice is to treat all such internal or undocumented identifiers as an implementation detail—do not expose them in sensitive contexts (such as web output, logs with user data, etc.) unless necessary.

Summary Table of Relevant SymbolTable Methods:

Method Purpose
get_id() Returns the symbol table's internal identifier
get_name() Returns the name of the scope (top for module, function/class name otherwise)
get_type() Returns the table's type ('module', 'class', or 'function')
get_lineno() Returns the first line number of the scope

If you require a value that is stable and reveals no internal information, prefer using get_name() or a custom mapping on your analysis layer instead of exposing the value from get_id().

References:
[1][2][5][6]: Python's official symtable module documentation and tutorials.

Citations:

Align SymbolTable.id() with CPython’s get_id(): avoid exposing raw pointers

The current implementation in vm/src/stdlib/symtable.rs returns the struct’s memory address as a usize, which diverges from CPython’s symtable.SymbolTable.get_id() (an internal, opaque identifier) and can leak pointer layouts.

Suggested changes:

  • Replace the pointer cast in fn id(&self) -> usize with a dedicated, incrementing identifier (e.g., an AtomicUsize counter or per‐table UID stored on creation).
  • Ensure IDs are unique within a run but do not expose raw addresses.
  • Update documentation to note that id() returns an internal symbol-table ID, not a memory address.

Locations to update:

  • vm/src/stdlib/symtable.rs, lines ~170–173 (the fn id(&self) definition)

Example diff sketch:

-   #[pygetset]
-   fn id(&self) -> usize {
-       self as *const Self as *const std::ffi::c_void as usize
-   }
+   // Use an internal counter instead of raw pointer to avoid leaking memory addresses
+   #[pygetset]
+   fn id(&self) -> usize {
+       self.inner_id  // populated from a global AtomicUsize at creation
+   }

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents 
In vm/src/stdlib/symtable.rs around lines 170 to 173, the id() method currently
returns the raw memory address cast to usize, which exposes internal pointer
details and differs from CPython's opaque get_id(). To fix this, replace the
pointer cast with a unique internal identifier generated via a static
AtomicUsize counter incremented for each new SymbolTable instance or assign a
unique ID at creation stored in the struct. Update the id() method to return
this stored unique ID instead of the pointer. Also, revise the method's
documentation to clarify that id() returns an internal symbol table identifier,
not a memory address.


#[pymethod]
fn get_identifiers(&self, vm: &VirtualMachine) -> PyResult<Vec<PyObjectRef>> {
#[pygetset]
fn identifiers(&self, vm: &VirtualMachine) -> PyResult<Vec<PyObjectRef>> {
let symbols = self
.symtable
.symbols
Expand All @@ -177,8 +181,8 @@ mod symtable {
Ok(symbols)
}

#[pymethod]
fn get_symbols(&self, vm: &VirtualMachine) -> PyResult<Vec<PyObjectRef>> {
#[pygetset]
fn symbols(&self, vm: &VirtualMachine) -> PyResult<Vec<PyObjectRef>> {
let symbols = self
.symtable
.symbols
Expand All @@ -201,22 +205,6 @@ mod symtable {
.collect();
Ok(symbols)
}

#[pymethod]
const fn has_children(&self) -> bool {
!self.symtable.sub_tables.is_empty()
}

#[pymethod]
fn get_children(&self, vm: &VirtualMachine) -> PyResult<Vec<PyObjectRef>> {
let children = self
.symtable
.sub_tables
.iter()
.map(|t| to_py_symbol_table(t.clone()).into_pyobject(vm))
.collect();
Ok(children)
}
}

#[pyattr]
Expand Down