Repository navigation
compression.zstd: non-determinist with zstd_dict #150583
Description
Bug report
Bug description:
It seems that in some cases, compressing with compress(... , zstd_dict=ZstdDict(...).as_digested_dict) is not deterministic, even when the parameters are the same.
Initially reported at Rogdham/pyzstd#66.
Reproducer
The following should print 1 but prints 2 most of the times.
Note: the issue seems somewhat less frequent on free-threaded builds.
from compression import zstd
DICT = b"7\xa40\xec/<\xabJ\t\x10\x10\xdf033\xb3w\n3\xf1x<\x1e\x8f\xc7\xe3\xf1x<\xcf\xf3\xbc\xf7\xd4BAAAAAAAAAAAAAAAAAAAAAAAAA\xa1P(\x14\n\x85B\xa1P(\x14\n\x85\xa2(\x8a\xa2(J)}t\xe1\xe1\xe1\xe1\xe1\xe1\xe1\xe1\xe1\xe1\xe1\xe1\xe1\xe1\xe1\xe1\xe1\xe1\xe1\xf1x<\x1e\x8f\xc7\xe3\xf1x\x9e\xe7y\xef\x01\x01\x00\x00\x00\x04\x00\x00\x00\x08\x00\x00\x00helloworld test data sample foo bar baz hello w"
DATA = b"hello world test"
ZSTD_DICT = zstd.ZstdDict(DICT).as_digested_dict
values = {
zstd.compress(DATA, zstd_dict=ZSTD_DICT)
for _ in range(100_000)
}
print(len(values))CPython versions tested on:
3.14, 3.15, CPython main branch,
Operating systems tested on:
Linux
Activity
gh-150583: Fix zstd compression level with digested ZstdDict (#150586)
[3.14] gh-150583: Fix zstd compression level with digested ZstdDict (G…
[3.15] gh-150583: Fix zstd compression level with digested ZstdDict (G…
emmatyping commentedon Jul 18, 2026
Thanks again for the report and fix!
pythongh-150583: Fix zstd compression level with digested ZstdDict (p…