Repository navigation
cPickle can misread data type #50539
Description
| BPO | 6290 |
|---|---|
| Nosy | @birkenfeld, @mdickinson, @avassalotti |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/avassalotti'
closed_at = <Date 2010-08-05.07:23:49.675>
created_at = <Date 2009-06-15.23:19:52.698>
labels = ['type-bug', 'library']
title = 'cPickle can misread data type'
updated_at = <Date 2010-08-05.07:23:49.673>
user = 'https://bugs.python.org/acjames'bugs.python.org fields:
activity = <Date 2010-08-05.07:23:49.673>
actor = 'alexandre.vassalotti'
assignee = 'alexandre.vassalotti'
closed = True
closed_date = <Date 2010-08-05.07:23:49.675>
closer = 'alexandre.vassalotti'
components = ['Library (Lib)']
creation = <Date 2009-06-15.23:19:52.698>
creator = 'ac.james'
dependencies = []
files = ['14336', '17965']
hgrepos = []
issue_num = 6290
keywords = ['patch']
message_count = 9.0
messages = ['89418', '89421', '89612', '89619', '89620', '89638', '110094', '110097', '112952']
nosy_count = 4.0
nosy_names = ['georg.brandl', 'mark.dickinson', 'alexandre.vassalotti', 'ac.james']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue6290'
versions = ['Python 2.6']
Activity
acjames commentedon Jun 15, 2009
When using cPickle to pickle / unpickle an object instance whose
__dict__ contains a dictionary of NumPy Arrays (on a windows32 system),
some of the array elements have the wrong type raising a ValueError:
could not convert string to float.
On UNIX platform this error does not occur, and the data is read out in
the correct type every time.
Forcing the caller to use pickle.py module instead removed the issue.
Statements about the imprecision of cPickle (such as bpo-1536, 655802),
or its deprecaition (now I can't find where that was mentioned), would
assist. By contrast the current state of the documentation implies that
cPickle is better overall, and thus should be used preferentially.
avassalotti commentedon Jun 16, 2009
Could you provide a test case? The behaviour you are describing sounds
like a bug in cPickle.
acjames commentedon Jun 22, 2009
I have now pinpointed the error to a list of infinities (see attached).
When using pickle.py to read the cPickle'd data we get a different, and
more, informative error:
ValueError: invalid literal for float(): 1.#INF
avassalotti commentedon Jun 23, 2009
Thanks for the test case. I will take a look.
avassalotti commentedon Jun 23, 2009
Could you give me the output of this?
By the way, are you sure this bug occurs on Python 2.6? Python 2.6 uses
a platform-independent float to string converter (i.e.,
PyOS_double_to_string) which shouldn't output stuff like "1.#INF"
Also, can you verify that the bug does not occur with pickle protocol 1
and over?
acjames commentedon Jun 23, 2009
Your test prints:
'(1p1\nF1.#INF\naF-1.#INF\naF-1.IND\na.'
[inf, -inf, nan]
My installation is Python 2.6.2 as currently distributed.
Specifying protocol 1 or 2 does circumvent the error.
Thank you.
mdickinson commentedon Jul 12, 2010
I think this can be closed. It should no longer be a problem in Python 2.7 or Python 3.x, and there's a workaround (use protocol 1 or 2) for Python 2.6.
In theory, it *could* still be fixed for Python 2.6.6, but changing the pickle output in a bugfix release seems like it might be dangerous.
mdickinson commentedon Jul 12, 2010
However, here's a patch. I haven't tested it on Windows.
avassalotti commentedon Aug 5, 2010
It is too late for 2.6.6 now that it is released.