Skip to content

Fix PyArg_ParseTuple format mismatch and non-NULL return with exception in stream module - #331

Open
devdanzin wants to merge 2 commits into
python-lz4:masterfrom
devdanzin:fix/stream-format-and-return
Open

devdanzin wants to merge 2 commits into
python-lz4:masterfrom
devdanzin:fix/stream-format-and-return

Conversation

@devdanzin

Copy link
Copy Markdown

Summary

  • Fix _compress_bound: format "OI" expects (PyObject*, unsigned int) but only &input_size (uint32_t*) is provided — the "O" writes a PyObject* into a uint32_t, causing type mismatch and undefined behavior on every call. Changed to "I" to match the identical _input_bound function at line 1091.

  • Fix _compress: when update_context_after_process fails, PyErr_Format sets an exception but goto exit_now returns the already-created py_dest (non-NULL). This violates the Python/C API contract (returning a result with an exception set), causing SystemError. Added Py_CLEAR(py_dest) before the goto.

Context

The "OI" vs "I" bug appears to be a copy-paste error — _input_bound on line 1091 has the correct "I" format for the same single-argument pattern.

Found using cext-review-toolkit.

Note

This PR was authored and submitted by Claude Code (Anthropic).
It was reviewed by a human before submission.

Test plan

  • Existing tests pass
  • _compress_bound now correctly parses a single unsigned int argument
  • _compress returns NULL (not a result) when update_context_after_process fails

devdanzin and others added 2 commits March 31, 2026 22:55
Fix memory and buffer leaks on five error paths:

- block/compress: release source and dict Py_buffers on PyMem_Malloc failure
- block/decompress: release source and dict Py_buffers on PyMem_Malloc failure
- frame/compress: release source Py_buffer on block_checksum version error
- frame/compress_begin: free destination buffer on LZ4F_compressBegin failure
  (same pattern as the bug fixed in 43fe65d, not propagated to this site)
- frame/__decompress: free original destination buffer on PyMem_Realloc failure
  (realloc returns NULL without freeing the original pointer)

Found using cext-review-toolkit (https://github.com/devdanzin/cext-review-toolkit).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…on in stream module

- Fix _compress_bound: format "OI" expects (PyObject*, unsigned int) but only
  &input_size is provided. The "O" writes a PyObject* into a uint32_t —
  type mismatch and memory corruption on every call. Change to "I" to match
  the identical _input_bound function at line 1091.

- Fix _compress: when update_context_after_process fails, PyErr_Format sets
  an exception but goto exit_now returns the already-created py_dest (non-NULL).
  This violates the Python/C API contract (returning a result with an exception
  set), causing SystemError. Clear py_dest before the goto.

Found using cext-review-toolkit (https://github.com/devdanzin/cext-review-toolkit).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

@fallenmi fallenmi left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used Codex to independently review exact base d5daffd against exact head 0957fe1.

  • The base deterministically rejects the valid call _compress_bound(1) with TypeError: function takes exactly 2 arguments (1 given); the head returns 17.
  • The head passed an adversarial parser matrix covering zero, one, the documented max, 32-bit wrap behavior consistent with _input_bound, wrong arity, non-indexable inputs, an __index__ object, 10,000 concurrent calls, and 1,000 repeated boundary rounds under PYTHONMALLOC=debug and -X dev.
  • The exact-head experimental stream suite passed: 5,438 passed and 2,703 skipped. Applying both PR commits cleanly to current master 029f365 preserved the oracle and passed the focused test_stream_0.py suite: 5,400 passed and 2,700 skipped.
  • The inherited error-path releases occur only after their resources are acquired and before the existing immediate returns. The added Py_CLEAR(py_dest) also restores the required NULL-result contract when update_context_after_process() leaves an exception set.
  • git diff --check is clean and all 36 live checks are successful.

Approved for exact head 0957fe1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants