Skip to content

test_zipfile64.[...].test_strip_removed_large_file_with_dd_no_sig fails on BigMem runs #152190

Closed
Closed
test_zipfile64.[...].test_strip_removed_large_file_with_dd_no_sig fails on BigMem runs#152190
Labels
testsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error

Description

@StanFromIreland
StanFromIreland
opened on Jun 25, 2026

Bug report

Bug description:

For example, in this run:

======================================================================
FAIL: test_strip_removed_large_file_with_dd_no_sig (test.test_zipfile64.TestRepack.test_strip_removed_large_file_with_dd_no_sig)
Should scan for the data descriptor (without signature) of a removed
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.diegorusso-aarch64-bigmem.bigmem/build/Lib/test/test_zipfile64.py", line 214, in test_strip_removed_large_file_with_dd_no_sig
    self.assertLess(peak, self.allowed_memory)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 603662 not less than 204800
----------------------------------------------------------------------

I'm also able to reproduce this locally:

$ ./python -m test test_zipfile64 -u extralargefile -v -m test_strip_removed_large_file_with_dd_no_sig
== CPython 3.16.0a0 (heads/main:f7e5cddcbbb, Jun 24 2026, 20:02:47) [GCC 15.2.1 20260123 (Red Hat 15.2.1-7)]
== Linux-7.0.12-101.fc43.x86_64-x86_64-with-glibc2.42 little-endian
== Python build: debug
== cwd: /home/stan/dev/cpython/build/test_python_worker_124469æ
== CPU count: 16
== encodings: locale=UTF-8 FS=utf-8
== resources (1): extralargefile

Using random seed: 2703420052
0:00:00 load avg: 1.79 mem: 27.0 MiB Run 1 test sequentially in a single process
0:00:00 load avg: 1.79 mem: 27.0 MiB [1/1] test_zipfile64
test_strip_removed_large_file_with_dd_no_sig (test.test_zipfile64.TestRepack.test_strip_removed_large_file_with_dd_no_sig)
Should scan for the data descriptor (without signature) of a removed ... FAIL

======================================================================
FAIL: test_strip_removed_large_file_with_dd_no_sig (test.test_zipfile64.TestRepack.test_strip_removed_large_file_with_dd_no_sig)
Should scan for the data descriptor (without signature) of a removed
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/stan/dev/cpython/Lib/test/test_zipfile64.py", line 215, in test_strip_removed_large_file_with_dd_no_sig
    self.assertLess(peak, self.allowed_memory)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 1027764 not less than 204800

----------------------------------------------------------------------
Ran 1 test in 200.611s

FAILED (failures=1)
test test_zipfile64 failed
0:03:20 load avg: 1.57 mem: 32.0 MiB [1/1/1] test_zipfile64 failed (1 failure) in 3 min 20 sec

== Tests result: FAILURE ==

1 test failed:
    test_zipfile64

Total duration: 3 min 20 sec
Total tests: run=1 (filtered) failures=1
Total test files: run=1/1 (filtered) failed=1
Result: FAILURE

This test was added recently in aec0aed by @danny0838 and @gpshead.

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Linked PRs

Activity

  1. StanFromIreland
    added this to Zipfile issues 📦on Jun 25, 2026
  2. StanFromIreland
    added
    type-bugAn unexpected behavior, bug, or error
    testsTests in the Lib/test dir
    on Jun 25, 2026

  3. danny0838 commented on Jun 25, 2026

    @danny0838
    danny0838
    on Jun 25, 2026
    Contributor

    This is probably a side effect of the patch b6937c5, which increased the chunk size from 8 KiB to io.DEFAULT_BUFFER_SIZE (128 KiB on PC) for no-DD scan, while the allowed memory in test_strip_removed_large_file_with_dd_no_sig remains 200 KiB.

    If we agree with the direction of keeping io.DEFAULT_BUFFER_SIZE chunk size, removing the overriding allowed_memory should fix the test error:

    cpython/Lib/test/test_zipfile64.py

    Line 204 in b6937c5

    self.allowed_memory = 200*1024

  4. danny0838
    added 4 commits that reference this issue on Jun 25, 2026

    pythongh-152190: Fix memory checking failure in `test_strip_removed_l…

    b20c49f

    pythongh-152190: Fix memory checking failure in `test_strip_removed_l…

    43b2a63

    pythongh-152190: Fix memory checking failure in `test_strip_removed_l…

    c98f30e

    pythongh-152190: Fix memory checking failure in `test_strip_removed_l…

    34a009b

  5. gpshead commented on Jun 25, 2026

    @gpshead
    gpshead
    on Jun 25, 2026 via email
    Member
  6. danny0838
    added 8 commits that reference this issue on Jun 25, 2026

    pythongh-152190: Fix memory checking failure in `test_strip_removed_l…

    7aea508

    pythongh-152190: Revise comment about the empirical memory threshold

    6547073

    pythongh-152190: Improve memory checking accuracy for `test_zipfile64`

    5335be0

    pythongh-152190: Improve coding style and docstrings

    a5db266

    pythongh-152190: Remove unneeded comments and checks

    0e54ea3

    pythongh-152190: Further optimize tests and tidy code

    ab89bc8

    pythongh-152190: Remove unneeded comments and checks

    05a4ace

    pythongh-152190: Further optimize tests and tidy code

    ded68c8

  7. danny0838 commented on Jul 13, 2026

    @danny0838
    danny0838
    on Jul 13, 2026
    Contributor

    Fixed the error and tidied the code. Is there still any major concern about the PR? (There are test errors but seems irrelevant)

  8. smontanaro commented on Aug 21, 2026

    @smontanaro
    smontanaro
    on Aug 21, 2026
    Contributor

    Confirmed this PR fixes the problem (MacOS and Linux). Can someone push this over the finish line?

  9. gpshead
    added a commit that references this issue on Aug 24, 2026

    gh-152190: Fix memory checking failure in `test_zipfile64.py` (GH-152203

    e5ed2ad
  10. StanFromIreland
    closed this as completedon Aug 27, 2026
  11. github-project-automation
    moved this to Done in Zipfile issues 📦on Aug 27, 2026
  12. clin1234
    added a commit that references this issue on Aug 27, 2026

    pythongh-152190: Fix memory checking failure in `test_zipfile64.py` (p…

    d96b396
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    testsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions