Repository navigation
merge json library with latest simplejson 2.0.x #48386
Description
| BPO | 4136 |
|---|---|
| Nosy | @loewis, @birkenfeld, @rhettinger, @etrepum, @amauryfa, @pitrou, @tiran, @devdanzin, @benjaminp, @djc |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/etrepum'
closed_at = <Date 2009-05-02.12:37:10.550>
created_at = <Date 2008-10-16.22:08:48.956>
labels = ['type-bug', 'library', 'release-blocker']
title = 'merge json library with latest simplejson 2.0.x'
updated_at = <Date 2009-05-02.12:37:10.131>
user = 'https://github.com/etrepum'bugs.python.org fields:
activity = <Date 2009-05-02.12:37:10.131>
actor = 'benjamin.peterson'
assignee = 'bob.ippolito'
closed = True
closed_date = <Date 2009-05-02.12:37:10.550>
closer = 'benjamin.peterson'
components = ['Library (Lib)']
creation = <Date 2008-10-16.22:08:48.956>
creator = 'bob.ippolito'
dependencies = []
files = ['11822', '11870', '13106', '13152', '13647', '13648', '13653']
hgrepos = []
issue_num = 4136
keywords = ['patch']
message_count = 51.0
messages = ['74872', '74874', '74875', '74877', '74929', '74931', '74932', '74933', '74941', '74946', '74947', '75171', '78697', '79054', '79100', '82053', '82218', '82233', '82234', '82252', '82613', '82873', '82874', '82875', '82877', '82887', '82892', '82900', '82937', '83172', '83662', '83703', '83704', '83705', '83706', '83713', '83715', '83721', '85118', '85752', '85753', '85757', '85759', '85775', '85776', '85777', '85778', '85779', '85781', '85814', '86942']
nosy_count = 10.0
nosy_names = ['loewis', 'georg.brandl', 'rhettinger', 'bob.ippolito', 'amaury.forgeotdarc', 'pitrou', 'christian.heimes', 'ajaksu2', 'benjamin.peterson', 'djc']
pr_nums = []
priority = 'release blocker'
resolution = 'accepted'
stage = 'needs patch'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue4136'
versions = ['Python 3.1']
Activity
etrepum commentedon Oct 16, 2008
simplejson 2.0.3 includes major performance enhancements (both in Python
and C) and removes usage of the private sre functionality.
I will need some help with 3.0 since I am not well versed in the changes
to the C API or Python code for that, but merging for 2.6.1 should be no
big deal.
How should I go about this?
vstinner commentedon Oct 16, 2008
Can you write a patch against python trunk ? :-)
etrepum commentedon Oct 16, 2008
Sure, but that doesn't port it to Python 3.0 :)
loewis commentedon Oct 16, 2008
Still, as Victor suggests, the first step for porting it to 3.0
definitely is to produce a patch for the trunk. What the next steps will
be can be discussed when this step has been completed.
etrepum commentedon Oct 17, 2008
patch to r66961 of trunk is attached.
amauryfa commentedon Oct 17, 2008
About the patch: are those lines really needed?
+ PyScannerType.tp_getattro = PyObject_GenericGetAttr;
+ PyScannerType.tp_setattro = PyObject_GenericSetAttr;
+ PyScannerType.tp_alloc = PyType_GenericAlloc;
+ PyScannerType.tp_new = PyType_GenericNew;
+ PyScannerType.tp_free = _PyObject_Del;
I've never used them. What happens if the slots are left empty, and let PyType_Ready() do the rest?
etrepum commentedon Oct 17, 2008
You're probably right, I don't remember what code I was using as a
template for that.
etrepum commentedon Oct 17, 2008
Actually, if I remove those lines from the equivalent module in simplejson
it no longer works properly with Python 2.5.2.
File "/Users/bob/src/simplejson/simplejson/decoder.py", line 307, in
__init__
self.scan_once = make_scanner(self)
TypeError: cannot create 'make_scanner' instances
loewis commentedon Oct 17, 2008
Why aren't the functions pointers in the structs itself?
As a procedural note, it seems like this patch is a complete rewrite of
the module. Do you anticipate further complete rewrites within the next
year? If yes, we should close the issue, and wait for the module to
evolve. If no, I'll try to find some time to review the entire module -
can you then please post the code to Rietveld?
etrepum commentedon Oct 17, 2008
I don't recall exactly why they aren't in the struct itself, it may not
have worked with some compiler on some platform.
It's not really a complete rewrite, the encoding path is largely the
same and the tests haven't changed.
Anyway, there is no further work planned for simplejson. It's done
except for the potential for bug fixes. The only enhancements were
performance related and this is about as fast as it's going to get. The
majority of this work was ready before Python 2.6 was released but it
was frozen so I couldn't get this in.
etrepum commentedon Oct 17, 2008
http://codereview.appspot.com/7311
etrepum commentedon Oct 24, 2008
Attached is a new diff, one byte fix to the float parser when parsing JSON
documents that are just a float (also a test and a version bump).
[-]merge json library with simplejson 2.0.3[/-][+]merge json library with simplejson 2.0.4[/+]34 remaining items
benjaminp commentedon Apr 7, 2009
Here's a half-baked patch against py3k. It resolves all the conflicts
but still has 15 failing tests. Perhaps someone would like to finish it up.
For example, json.dumps(b"hi") works, but not json.dumps([b"hi", "hi"])
pitrou commentedon Apr 7, 2009
There is the problem in the current py3k version of json. b"hi" can be
serialized, but not [b"hi"].
pitrou commentedon Apr 7, 2009
Christian:
in py3k, loads and dumps always seem to operate on/produce str
objects, but encode_basestring_ascii returns a bytes object. Why is that?
what is the use of the encoding argument in py3k? it looks completely
ignored (bytes objects are not allowed as input and never produced as
output)
pitrou commentedon Apr 8, 2009
Updated patch:
involved a lot of code)
To be done:
pitrou commentedon Apr 8, 2009
Here is an updated patch, completely removing the
encodingparameterand fixing docs.
pitrou commentedon Apr 8, 2009
(by the way, all tests pass)
rhettinger commentedon Apr 8, 2009
It would be better to have a patch that diff's from the current 2.7
version than to start with the 3.0 version; otherwise, the two will
never be fully synchronized and some of the choices made in 2.6-to-3.0
will live on forever. The 2.7 version reflects more patch review and
real world usage (from simplejson) than the relatively unexercised 3.0
version.
pitrou commentedon Apr 8, 2009
How am I supposed to produce this patch?
rhettinger commentedon Apr 8, 2009
The idea is to ignore the current 3.0 version and just redo the 2-to-3
conversion from 2.7 and do it well this time. Compute the 3.1 patch as
if the current 3.0 version was blown away (reverted).
tiran commentedon Apr 8, 2009
+1 for Raymond's suggestion
The 3.0 version of json was more like a last minute patch work than
thorough work. You might wanna svn rm the 3.0 code, svn cp the 2.7 code
to the py3k branch and start all over.
djc commentedon Apr 9, 2009
I'll take a stab at doing it Raymond's way this weekend.
benjaminp commentedon May 2, 2009
Since no other patches were proposed, I applied Antoine's patch in r72194.