Repository navigation
Add _PyObject_FastCall() #71315
Description
| BPO | 27128 |
|---|---|
| Nosy | @scoder, @vstinner, @serhiy-storchaka, @1st1, @ztane |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2016-09-01.13:13:54.081>
created_at = <Date 2016-05-26.10:15:56.728>
labels = ['performance']
title = 'Add _PyObject_FastCall()'
updated_at = <Date 2016-09-01.13:13:54.079>
user = 'https://github.com/vstinner'bugs.python.org fields:
activity = <Date 2016-09-01.13:13:54.079>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2016-09-01.13:13:54.081>
closer = 'vstinner'
components = []
creation = <Date 2016-05-26.10:15:56.728>
creator = 'vstinner'
dependencies = []
files = ['43011', '43014', '44041', '44079', '44128']
hgrepos = []
issue_num = 27128
keywords = ['patch']
message_count = 32.0
messages = ['266422', '266424', '266429', '266430', '268057', '268059', '272138', '272139', '272197', '272264', '272268', '272479', '272503', '272884', '272887', '272888', '272891', '273136', '273140', '273143', '273144', '273153', '273166', '273175', '273349', '273350', '273351', '273365', '273371', '273388', '273402', '274123']
nosy_count = 6.0
nosy_names = ['scoder', 'vstinner', 'python-dev', 'serhiy.storchaka', 'yselivanov', 'ztane']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue27128'
versions = ['Python 3.6']
Activity
python-dev commentedon Aug 16, 2016
New changeset 288ec55f1912 by Victor Stinner in branch 'default':
Issue bpo-27128: Cleanup _PyEval_EvalCodeWithName()
https://hg.python.org/cpython/rev/288ec55f1912
New changeset e615718a6455 by Victor Stinner in branch 'default':
Use Py_ssize_t in _PyEval_EvalCodeWithName()
https://hg.python.org/cpython/rev/e615718a6455
vstinner commentedon Aug 16, 2016
Patch version 3: simpler and shorter patch
I also pushed some changes unrelated to fastcall in Python/ceval.c to simplify the patch.
Very few functions are modified (directly or indirectly) to use _PyObject_FastCall():
Much more will come in following patches.
vstinner commentedon Aug 16, 2016
Oh, I now understand. The change makes "namedtuple.attr" slower. With fastcall-3.patch attached to this issue, the fast path is not taken on this benchmark, and so you loose the removed optimization (tuple cached in the modified descriptor function).
In fact, you need the "full" fastcall change to make this attribute lookup *faster*:
https://bugs.python.org/issue26814#msg263999
So yeah, it's better to wait until more changes are merged.
python-dev commentedon Aug 19, 2016
New changeset a1a29d20f52d by Victor Stinner in branch 'default':
Add _PyObject_FastCall()
https://hg.python.org/cpython/rev/a1a29d20f52d
New changeset 89e4ad001f3d by Victor Stinner in branch 'default':
PyEval_CallObjectWithKeywords() uses fast call
https://hg.python.org/cpython/rev/89e4ad001f3d
New changeset 7cd479573de9 by Victor Stinner in branch 'default':
call_function_tail() uses fast call
https://hg.python.org/cpython/rev/7cd479573de9
New changeset 34af2edface9 by Victor Stinner in branch 'default':
Cleanup call_function_tail()
https://hg.python.org/cpython/rev/34af2edface9
New changeset adceb14cab96 by Victor Stinner in branch 'default':
Cleanup callmethod()
https://hg.python.org/cpython/rev/adceb14cab96
New changeset 10f1a4910adb by Victor Stinner in branch 'default':
PEP-7: add {...} around null_error() in abstract.c
https://hg.python.org/cpython/rev/10f1a4910adb
New changeset 5cf9524f2923 by Victor Stinner in branch 'default':
Avoid call_function_tail() for empty format str
https://hg.python.org/cpython/rev/5cf9524f2923
New changeset f1ad6f64a11e by Victor Stinner in branch 'default':
Fix PyObject_Call() parameter names
https://hg.python.org/cpython/rev/f1ad6f64a11e
python-dev commentedon Aug 19, 2016
New changeset 2da6dc1c30d8 by Victor Stinner in branch 'default':
contains and rich compare slots use fast call
https://hg.python.org/cpython/rev/2da6dc1c30d8
New changeset 2d4d40da2aba by Victor Stinner in branch '3.5':
Fix a refleak in call_method()
https://hg.python.org/cpython/rev/2d4d40da2aba
New changeset 5b1ed48aedef by Victor Stinner in branch '2.7':
Fix a refleak in call_method()
https://hg.python.org/cpython/rev/5b1ed48aedef
New changeset df4efc23ab18 by Victor Stinner in branch '3.5':
Fix a refleak in call_maybe()
https://hg.python.org/cpython/rev/df4efc23ab18
New changeset 7669fb39a9ce by Victor Stinner in branch '2.7':
Fix a refleak in call_maybe()
https://hg.python.org/cpython/rev/7669fb39a9ce
python-dev commentedon Aug 19, 2016
New changeset 73b00fb1dc9d by Victor Stinner in branch 'default':
Cleanup call_method() and call_maybe()
https://hg.python.org/cpython/rev/73b00fb1dc9d
New changeset 8e085070ab28 by Victor Stinner in branch 'default':
call_method() and call_maybe() now use fast call
https://hg.python.org/cpython/rev/8e085070ab28
New changeset 2d2bc1906b5b by Victor Stinner in branch 'default':
Issue bpo-27128: Cleanup slot_sq_item()
https://hg.python.org/cpython/rev/2d2bc1906b5b
New changeset 6eb586b85fa1 by Victor Stinner in branch 'default':
Issue bpo-27128: slot_sq_item() uses fast call
https://hg.python.org/cpython/rev/6eb586b85fa1
New changeset 605a42a50496 by Victor Stinner in branch 'default':
Issue bpo-27128: Cleanup slot_nb_bool()
https://hg.python.org/cpython/rev/605a42a50496
New changeset 6a21b6599692 by Victor Stinner in branch 'default':
slot_nb_bool() now uses fast call
https://hg.python.org/cpython/rev/6a21b6599692
New changeset 45d2b5c12b19 by Victor Stinner in branch 'default':
slot_tp_iter() now uses fast call
https://hg.python.org/cpython/rev/45d2b5c12b19
New changeset 124d5d0ef81f by Victor Stinner in branch 'default':
calliter_iternext() now uses fast call
https://hg.python.org/cpython/rev/124d5d0ef81f
New changeset 71c22e592a9b by Victor Stinner in branch 'default':
keyobject_richcompare() now uses fast call
https://hg.python.org/cpython/rev/71c22e592a9b
python-dev commentedon Aug 19, 2016
New changeset 3ab32f7add6e by Victor Stinner in branch 'default':
Issue bpo-27128: _pickle uses fast call
https://hg.python.org/cpython/rev/3ab32f7add6e
vstinner commentedon Aug 19, 2016
Ok, I updated the most simple forms of function calls.
I will open new issues for more complex calls and more sensible parts of
the code like ceval.c.
Buildbots seem to be happy.
python-dev commentedon Aug 19, 2016
New changeset c2af917bde71 by Victor Stinner in branch 'default':
PyFile_WriteObject() now uses fast call
https://hg.python.org/cpython/rev/c2af917bde71
New changeset 0da1ce362d15 by Victor Stinner in branch 'default':
import_name() now uses fast call
https://hg.python.org/cpython/rev/0da1ce362d15
New changeset e5b24f595235 by Victor Stinner in branch 'default':
PyErr_PrintEx() now uses fast call
https://hg.python.org/cpython/rev/e5b24f595235
New changeset 154f78d387f9 by Victor Stinner in branch 'default':
call_trampoline() now uses fast call
https://hg.python.org/cpython/rev/154f78d387f9
New changeset 351b987d6d1c by Victor Stinner in branch 'default':
sys_pyfile_write_unicode() now uses fast call
https://hg.python.org/cpython/rev/351b987d6d1c
New changeset abb93035ebb7 by Victor Stinner in branch 'default':
_elementtree: deepcopy() now uses fast call
https://hg.python.org/cpython/rev/abb93035ebb7
New changeset 2954d2aa4c90 by Victor Stinner in branch 'default':
pattern_subx() now uses fast call
https://hg.python.org/cpython/rev/2954d2aa4c90
vstinner commentedon Aug 20, 2016
I created two new issues:
scoder commentedon Aug 22, 2016
FYI: I copied your (no-kwargs) implementation over into Cython and I get around 17% faster calls to Python functions with 2 positional arguments.
vstinner commentedon Aug 22, 2016
Hey, cool! It's always cool to get performance enhancement without having to break the C API nor having to modify source code :-)
What do you mean by "I copied your (no-kwargs) implementation"? The whole giant patch? Or just a few changes? Which changes?
python-dev commentedon Aug 22, 2016
New changeset 7dd85b19c873 by Victor Stinner in branch 'default':
Optimize call to Python function without argument
https://hg.python.org/cpython/rev/7dd85b19c873
serhiy-storchaka commentedon Aug 22, 2016
The problem is that passing keyword arguments as a dict is not the most efficient way due to an overhead of creating a dict. For now keyword arguments are pushed on the stack as interlaced array of keyword names and values. It may be more efficient to push values and names as continuous arrays (bpo-27213). PyArg_ParseTupleAndKeywords() accepts a tuple and a dict, but private function _PyArg_ParseTupleAndKeywordsFast() (bpo-27574) can be changed to accept positional and keyword arguments as continuous arrays: (int nargs, PyObject **args, int nkwargs, PyObject **kwnames, PyObject **kwargs). Therefore we will be forced either to change the signature of _PyObject_FastCall() and the meaning of METH_FASTCALL, or add new _PyObject_FastCallKw() and METH_FASTCALLKW for support fast passing keyword arguments. Or may be add yet _PyObject_FastCallNoKw() for faster passing only positional arguments without an overhead of _PyObject_FastCall(). And make older _PyObject_FastCall() and METH_FASTCALL obsolete.
There is yet one possibility. Argument Clinic can generate a dict that maps keyword argument names to indices of arguments and tie it to a function. External code should map names to indices using this dictionary and pass arguments as just a continuous array to function with METH_FASTCALL (raising an error if some argument is passed as positional and keyword, or if keyword-only argument is passed as positional, etc). In that case the kwargs parameter of _PyObject_FastCall() becomes obsolete too.
vstinner commentedon Aug 22, 2016
Serhiy: I "moved" your msg273365 to the issue bpo-27809.
scoder commentedon Aug 22, 2016
I copied what you committed into CPython for _PyFunction_FastCall():
cython/cython@8f3d3bd
and then enabled its usage in a couple of places:
cython/cython@a3cfec8
especially for all function/method calls that we generate for user code:
cython/cython@a51df33
Note that PyMethod objects get unpacked into function+self right before the PyFunction_Check(), so the tuple avoidance optimisation also applies to Python method calls.
vstinner commentedon Aug 22, 2016
Ok, I see much better with concrete commits. I'm really happy that
Cython also benefits from these enhancements.
Note: handling keywords is likely to change quickly ;-)
vstinner commentedon Sep 1, 2016
The main features (_PyFunction_FastCall()) has been merged. Supporting keyword arguments is now handled by other issue (see issue bpo-27830). I close this issue.