Allow constructing `PyTraceback` by bschoenmaeckers · Pull Request #5857 · PyO3/pyo3

bschoenmaeckers · 2026-03-06T08:42:15Z

No description provided.

davidhewitt · 2026-03-06T09:34:44Z

Interesting, I would be very curious to know how this looks for injecting Rust call frames into Python tracebacks.

bschoenmaeckers · 2026-03-06T10:46:57Z

Interesting, I would be very curious to know how this looks for injecting Rust call frames into Python tracebacks.

Did some testing usingbtparse:

Traceback (most recent call last):
  File "<unknown>", line 0, in __pthread_cond_wait
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/sys/thread/unix.rs", line 127, in std::sys::thread::unix::Thread::new::thread_start
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/alloc/src/boxed.rs", line 2206, in <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs", line 250, in core::ops::function::FnOnce::call_once{{vtable.shim}}
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/thread/lifecycle.rs", line 89, in std::thread::lifecycle::spawn_unchecked::{{closure}}
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panic.rs", line 359, in std::panic::catch_unwind
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs", line 544, in std::panicking::catch_unwind
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs", line 581, in std::panicking::catch_unwind::do_call
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/panic/unwind_safe.rs", line 274, in <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/thread/lifecycle.rs", line 91, in std::thread::lifecycle::spawn_unchecked::{{closure}}::{{closure}}
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/sys/backtrace.rs", line 166, in std::sys::backtrace::__rust_begin_short_backtrace
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/test/src/lib.rs", line 637, in test::run_test::{{closure}}
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/test/src/lib.rs", line 607, in test::run_test::{{closure}}
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/test/src/lib.rs", line 686, in test::run_test_in_process
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panic.rs", line 359, in std::panic::catch_unwind
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs", line 544, in std::panicking::catch_unwind
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs", line 581, in std::panicking::catch_unwind::do_call
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/panic/unwind_safe.rs", line 274, in <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/test/src/lib.rs", line 686, in test::run_test_in_process::{{closure}}
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/test/src/lib.rs", line 663, in test::__rust_begin_short_backtrace
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs", line 250, in core::ops::function::FnOnce::call_once
  File "/Users/basschoenmaeckers/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs", line 250, in core::ops::function::FnOnce::call_once
    extern "rust-call" fn call_once(self, args: Args) -> Self::Output;
  File "./src/types/traceback.rs", line 230, in pyo3::types::traceback::tests::test_convert_backtrace::{{closure}}
    fn test_convert_backtrace() {
  File "./src/types/traceback.rs", line 231, in pyo3::types::traceback::tests::test_convert_backtrace
    Python::attach(|py| {
  File "./src/marker.rs", line 415, in pyo3::marker::Python::attach
    f(guard.python())
  File "./src/types/traceback.rs", line 232, in pyo3::types::traceback::tests::test_convert_backtrace::{{closure}}
    let backtrace = Backtrace::force_capture();
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/backtrace.rs", line 331, in std::backtrace::Backtrace::create
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/../../backtrace/src/backtrace/mod.rs", line 66, in std::backtrace_rs::backtrace::trace_unsynchronized
  File "/rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/../../backtrace/src/backtrace/libunwind.rs", line 117, in std::backtrace_rs::backtrace::libunwind::trace

davidhewitt · 2026-03-06T13:40:40Z

+            .unwrap_or_else(|| {
+                let frame = PyFrame::new(py, c"<unknown>", c"<unknown>", 0)?;
+                PyTraceback::new(py, None, frame, 0, 0)
+            })


Rather than a null frame, should this just result Result<Option<...>> and let the user handle the empty case?

Fine by me.

bschoenmaeckers · 2026-03-06T13:57:56Z

Interesting, I would be very curious to know how this looks for injecting Rust call frames into Python tracebacks.

We could experiment with creating PyFrame from std::panic::Location::caller and adding that to the current exception traceback. 🤔

bschoenmaeckers · 2026-03-06T18:07:32Z

Might be nice to find a way to do this in abi3 as well. Does someone have an idea how to do that?

Icxolu · 2026-03-07T13:03:18Z

PyTrackback is available in types module. You could call into Python to construct it on the limited abi. I don't think it's possible to create a frame object from the Python side, but if you happen to have one you could create a traceback object this way.

davidhewitt · 2026-03-07T13:28:43Z

We could potentially open an upstream CPython issue asking for a way to create dummy Rust tracebacks in the stable ABI; this sort of mixed-language traceback might be a nice ergonomics win for some projects.

On the other hand, perhaps debugging the Rust frames is most useful when debugging the Rust module itself, at which time it won't be for distribution so using the limited API might be acceptable...

bschoenmaeckers · 2026-03-07T15:00:37Z

PyTrackback is available in types module. You could call into Python to construct it on the limited abi. I don't think it's possible to create a frame object from the Python side, but if you happen to have one you could create a traceback object this way.

I could not find a way to construct a frame with python, so might not be possible.

>>> types.FrameType()
Traceback (most recent call last):
  File "<python-input-5>", line 1, in <module>
    types.FrameType()
    ~~~~~~~~~~~~~~~^^
TypeError: cannot create 'frame' instances

On the other hand, perhaps debugging the Rust frames is most useful when debugging the Rust module itself, at which time it won't be for distribution so using the limited API might be acceptable...

I think for now this is acceptable as a first step.

bschoenmaeckers · 2026-03-09T08:17:40Z

Found a way to make this work on abi3, but it is not going to make you happy. So I will leave it out for now.

    let code = PyCode::compile(py, c"raise Exception()", file_name, PyCodeInput::File)?;
    let kwargs = [
        ("co_firstlineno", line_number.into_py_any(py)?),
        ("co_name", func_name.into_py_any(py)?),
    ]
    .into_py_dict(py)?;

    let traceback = code
        .call_method("replace", (), Some(&kwargs))
        .cast_into::<PyCode>()?
        .run(None, None)
        .unwrap_err()
        .traceback(py)
        .expect("traceback should be present");

    traceback.getattr("tb_frame").cast_into()

bschoenmaeckers · 2026-03-12T08:07:30Z

+    /// Creates a new traceback object from an iterator of frames.
+    ///
+    /// The frames should be ordered from newest to oldest, i.e. the first frame in the iterator
+    /// will be the innermost frame in the traceback.
+    #[cfg(all(not(Py_LIMITED_API), not(PyPy), not(GraalPy)))]
+    pub fn from_frames<'py, I>(
+        py: Python<'py>,
+        frames: I,
+    ) -> PyResult<Option<Bound<'py, PyTraceback>>>
+    where
+        I: IntoIterator,
+        I::Item: IntoPyObject<'py,>

            
+    {
+        frames.into_iter().try_fold(None, |prev, frame| {
+            let frame = frame.into_pyobject(py).map_err(Into::into)?.into_bound();
+            let line_number = frame.line_number();
+            PyTraceback::new(py, prev, frame, 0, line_number).map(Some)
+        })
+    }
+}


I'm not sure about this api yet. Should I remove it from this MR so we can iterate on the design later?

I've removed PyTraceback::from_frames for now. So this is ready for final review.

Seems reasonable to me to play it slow 👍

davidhewitt

Thanks, this seems good to me!

Tpt reviewed Mar 6, 2026

View reviewed changes

Comment thread src/types/traceback.rs Outdated

bschoenmaeckers force-pushed the construct-traceback branch from 608cc0d to e082d4f Compare March 6, 2026 08:51

bschoenmaeckers commented Mar 6, 2026

View reviewed changes

Comment thread src/types/traceback.rs Outdated

bschoenmaeckers force-pushed the construct-traceback branch 2 times, most recently from 911f992 to 5096f1f Compare March 6, 2026 09:15

bschoenmaeckers marked this pull request as ready for review March 6, 2026 10:50

bschoenmaeckers commented Mar 6, 2026

View reviewed changes

Comment thread src/types/traceback.rs Outdated

davidhewitt reviewed Mar 6, 2026

View reviewed changes

bschoenmaeckers force-pushed the construct-traceback branch from 7185363 to d9ca236 Compare March 6, 2026 14:59

bschoenmaeckers mentioned this pull request Mar 7, 2026

Add PyTraceback::append #5867

Draft

bschoenmaeckers mentioned this pull request Mar 10, 2026

Inject rust stack frames into PyErr traceback #5872

Open

bschoenmaeckers force-pushed the construct-traceback branch from 7a18642 to 9f2498b Compare March 11, 2026 08:12

bschoenmaeckers commented Mar 12, 2026

View reviewed changes

bschoenmaeckers added the CI-build-full label Mar 12, 2026

bschoenmaeckers force-pushed the construct-traceback branch from 9f2498b to 4c436a4 Compare March 12, 2026 14:40

bschoenmaeckers removed the CI-build-full label Mar 12, 2026

bschoenmaeckers force-pushed the construct-traceback branch from 4c436a4 to e343e24 Compare March 12, 2026 15:27

bschoenmaeckers added 5 commits March 12, 2026 16:27

Allow constructing PyTraceback

a51b5ab

use classinfo_object

e1ae310

Clarify order of frames

20069b4

changelog

22de2c1

Do not panic when there are no frames

be45e9e

bschoenmaeckers added 3 commits March 12, 2026 16:27

Return None when there are no frames

f6404b8

One less generic parameter

531aae2

cast_into_unchecked

88eaa0d

bschoenmaeckers force-pushed the construct-traceback branch from e343e24 to 88eaa0d Compare March 12, 2026 15:28

bschoenmaeckers added 2 commits March 13, 2026 10:59

Remove PyTraceback::from_frames

67d18b0

Add test & docs for PyFrameMethods

efe5c89

bschoenmaeckers force-pushed the construct-traceback branch from 850ed1c to efe5c89 Compare March 13, 2026 10:04

davidhewitt approved these changes Mar 13, 2026

View reviewed changes

Comment thread newsfragments/5857.added.md Outdated

davidhewitt reviewed Mar 13, 2026

View reviewed changes

Comment thread src/types/frame.rs Outdated

bschoenmaeckers added 2 commits March 13, 2026 16:51

Add private PyCode::empty

c3330fc

Update changelog

67201cc

bschoenmaeckers enabled auto-merge March 13, 2026 15:51

bschoenmaeckers added this pull request to the merge queue Mar 13, 2026

github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Mar 13, 2026

bschoenmaeckers added this pull request to the merge queue Mar 13, 2026

Merged via the queue into PyO3:main with commit 7aa6163 Mar 13, 2026
45 checks passed

bschoenmaeckers deleted the construct-traceback branch March 13, 2026 19:32

Conversation

bschoenmaeckers commented Mar 6, 2026

Uh oh!

Uh oh!

Uh oh!

davidhewitt commented Mar 6, 2026

Uh oh!

bschoenmaeckers commented Mar 6, 2026

Uh oh!

Uh oh!

davidhewitt Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

bschoenmaeckers Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

bschoenmaeckers commented Mar 6, 2026

Uh oh!

bschoenmaeckers commented Mar 6, 2026

Uh oh!

Icxolu commented Mar 7, 2026

Uh oh!

davidhewitt commented Mar 7, 2026

Uh oh!

bschoenmaeckers commented Mar 7, 2026

Uh oh!

bschoenmaeckers commented Mar 9, 2026

Uh oh!

bschoenmaeckers Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

bschoenmaeckers Mar 13, 2026

Choose a reason for hiding this comment

Uh oh!

davidhewitt Mar 13, 2026

Choose a reason for hiding this comment

Uh oh!

davidhewitt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants