Repository navigation
Outdated statement in docs comparing pickle with marshal #143928
Description
Activity
@aescarias I'd like to take up this PR
However,
Marshal v3 does support recursion via references (as your example shows), so that bullet is outdated.However sharing question remains—pickle stores shared objects once + rewrites references. Does marshal v3 do object sharing too, or just recursion?
@aescarias I'd like to take up this PR However, Marshal v3 does support recursion via references (as your example shows), so that bullet is outdated.
However sharing question remains—pickle stores shared objects once + rewrites references. Does marshal v3 do object sharing too, or just recursion?
@Yashp002
Indeed. marshal also supports object sharing.>>> import marshal >>> xs = [300, 300, 300] >>> marshal.dumps(xs) b'\xdb\x03\x00\x00\x00\xe9,\x01\x00\x00r\x01\x00\x00\x00r\x01\x00\x00\x00'
You can look at
marshal.cfor the details but, in short, amarshalwriter keeps track of objects by storing a map of objects it has already seen. When first writing a "complex object" (anything but singletons and the actual ref objects), it marks it as "referable" by setting the highest bit of its tag (in marshal, an object has a 1-byte "tag" followed by the value). If you now write an object it has already seen, it instead writes a reference object with an index to the already stored object.So in this example:
\xdb\x03\x00\x00\x00is a list of 3 items (ref. index 0),\xe9,\x01\x00\x00is the small integer 300 (ref. index 1), and finally, the 2 objectsr\x01\x00\x00\x00are references to ref. index 1.(before Python 3.4,
marshal.dumps(xs)outputs
b'[\x03\x00\x00\x00i,\x01\x00\x00i,\x01\x00\x00i,\x01\x00\x00'which repeats the 3 integers)Yes, this statement is outdated. Marshal protocol 3 and newer supports shared objects and recursive collections. The latter is an unintentional side effect -- it is not needed for
.pycfiles. Marshal supports even self-recursive tuples which you cannot normally create (pickle don't support them).If you set
pickler.fast = True(you should not), the pickler will also stop supporting shared objects and recursion.So, I support removing this statement.
On other hand, the second statement can be expanded. Marshal supports only limited set of types, while pickle supports much more builtin and extension types, supports most of user-defined classes by default and allows to add support for classes not supported by default. But this is a different issue.
@picnixz @serhiy-storchaka do you think i could push a PR then, I'll remove the part that says marshal doesn't track serialized objects and would you want the expansion saying while marshal handles it too now, pickle is still superior for above said reasons? ⏫
I would not call pickle superior. If you make a PR, only remove the outdated part. Mentioning more things is another issue as Serhiy said.
- added a commit that references this issue
on Jan 26, 2026
I found this statement in the pickle docs comparing the pickle module with the marshal module:
However, this appears to no longer be the case as marshal version 3 (added in Python 3.4) supports keeping track of objects via references and as such is capable of serializing recursive objects.
(in Python 3.3,
marshal.dumps(xs)fails withValueError: object too deeply nested to marshal)So I don't see any reason for this statement to be in the docs in its current state.
Linked PRs