Skip to content

Argument Clinic: improve generated parser for 1-argument functions #67680

Closed
Closed
Argument Clinic: improve generated parser for 1-argument functions#67680
Assignees
serhiy-storchaka
Labels
buildThe build process and cross-buildtopic-argument-clinictype-featureA feature request or enhancement

Description

@serhiy-storchaka
serhiy-storchaka
opened on Feb 20, 2015
BPO 23492
Nosy @pitrou, @scoder, @larryhastings, @serhiy-storchaka, @MojoVampire
Dependencies
  • bpo-23501: Argument Clinic: generate code into separate files by default
    		•
    Files
  • clinic_meth_o.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2015-04-03.22:00:38.147>
created_at = <Date 2015-02-20.14:40:11.906>
labels = ['type-feature', 'expert-argument-clinic', 'build']
title = 'Argument Clinic: improve generated parser for 1-argument functions'
updated_at = <Date 2015-04-04.14:07:58.523>
user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2015-04-04.14:07:58.523>
actor = 'python-dev'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2015-04-03.22:00:38.147>
closer = 'serhiy.storchaka'
components = ['Build', 'Argument Clinic']
creation = <Date 2015-02-20.14:40:11.906>
creator = 'serhiy.storchaka'
dependencies = ['23501']
files = ['38188']
hgrepos = []
issue_num = 23492
keywords = ['patch']
message_count = 11.0
messages = ['236288', '236313', '236315', '236367', '236372', '236382', '236385', '236388', '236408', '240038', '240072']
nosy_count = 6.0
nosy_names = ['pitrou', 'scoder', 'larry', 'python-dev', 'serhiy.storchaka', 'josh.r']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue23492'
versions = ['Python 3.5']

    Activity

    1. serhiy-storchaka commented on Feb 20, 2015

      @serhiy-storchaka
      serhiy-storchaka
      on Feb 20, 2015
      MemberAuthor

      Proposed patch improve generated parsers for functions with single positional argument. Now they always generated as METH_O and PyArg_Parse() is used to parse single argument.

      To avoid code churn in this and following changes it would be worth to extract all generated code in separated files.

    2. serhiy-storchaka
      added
      buildThe build process and cross-build
      type-featureA feature request or enhancement
      on Feb 20, 2015

    3. larryhastings commented on Feb 20, 2015

      @larryhastings
      larryhastings
      on Feb 20, 2015
      Contributor

      I'm not opposed to the patch in principle. I assume your goal is to make Python faster--do you have any data on how much faster?

      I don't support immediately changing all uses of Argument Clinic to generate their code into a separate file. I would want to see a consensus from the community first. Perhaps we should discuss it (again?) on python-dev?

    4. serhiy-storchaka commented on Feb 20, 2015

      @serhiy-storchaka
      serhiy-storchaka
      on Feb 20, 2015
      MemberAuthor

      This is one step on long way. Second step will be to inline PyArg_Parse for
      some format codes ("i", "U", "y*", "O&", "O!"). Then we could try to expand
      PyArg_ParseTuple, at least for simple common cases. Then
      PyArg_ParseTupleAndKeywords. All this step will produce large diffs for
      generated code.

    5. scoder commented on Feb 21, 2015

      @scoder
      scoder
      on Feb 21, 2015
      Contributor

      Serhiy, I suggest you look at the code that Cython generates for its functions. It has been extensively profiled and optimised (years ago), so generating the same code for the argument clinic should yield the same performance.

      And while I don't have exact numbers at hand, avoiding the tuple packing for the call by passing it into a METH_O function can make a substantial difference. It also kills support for keyword arguments, though.

    6. larryhastings commented on Feb 21, 2015

      @larryhastings
      larryhastings
      on Feb 21, 2015
      Contributor

      Stefan: Serhiy's patch only affects functions taking a single positional-only parameter.

    7. larryhastings commented on Feb 21, 2015

      @larryhastings
      larryhastings
      on Feb 21, 2015
      Contributor

      lgtm

    8. serhiy-storchaka commented on Feb 21, 2015

      @serhiy-storchaka
      serhiy-storchaka
      on Feb 21, 2015
      MemberAuthor

      Serhiy, I suggest you look at the code that Cython generates for its functions. It has been extensively profiled and optimised (years ago), so generating the same code for the argument clinic should yield the same performance.

      Thanks, I'll look on it.

      And while I don't have exact numbers at hand, avoiding the tuple packing for the call by passing it into a METH_O function can make a substantial difference.

      Good idea. Here are samples:

      $ ./python -m timeit "chr(0x20ac)"
Unpatched: 1000000 loops, best of 3: 0.976 usec per loop
Patched:   1000000 loops, best of 3: 0.752 usec per loop

$ ./python -m timeit -s "from cmath import isnan; x = 1j" -- "isnan(x)"
Unpatched: 1000000 loops, best of 3: 0.62 usec per loop
Patched:   1000000 loops, best of 3: 0.386 usec per loop

      Of course for more complex functions the effect is smaller.

    9. serhiy-storchaka commented on Feb 21, 2015

      @serhiy-storchaka
      serhiy-storchaka
      on Feb 21, 2015
      MemberAuthor

      After expanding PyArg_Parse for "i" and "D" codes above tests give following results:

      $ ./python -m timeit "chr(0x20ac)"
1000000 loops, best of 3: 0.558 usec per loop

$ ./python -m timeit -s "from cmath import isnan; x = 1j" -- "isnan(x)"
1000000 loops, best of 3: 0.278 usec per loop

      About twice in comparison with initial variant!

    10. serhiy-storchaka
      self-assigned this
      on Feb 22, 2015

    11. pitrou commented on Feb 22, 2015

      @pitrou
      pitrou
      on Feb 22, 2015
      Member

      Let's make Argument Clinic a fierce optimizer!
      (+1 on this)

    12. serhiy-storchaka
      added
      topic-argument-clinic
      on Feb 25, 2015

    13. python-dev commented on Apr 3, 2015

      @python-dev
      python-devmannequin
      on Apr 3, 2015
      Mannequin

      New changeset e10ad4d4d490 by Serhiy Storchaka in branch 'default':
      Issue bpo-23492: Argument Clinic now generates argument parsing code with
      https://hg.python.org/cpython/rev/e10ad4d4d490

    14. serhiy-storchaka
      closed this as completedon Apr 3, 2015

    15. python-dev commented on Apr 4, 2015

      @python-dev
      python-devmannequin
      on Apr 4, 2015
      Mannequin

      New changeset 973c9ec53bbb by Serhiy Storchaka in branch 'default':
      Fixed the array module broken in issue bpo-23492.
      https://hg.python.org/cpython/rev/973c9ec53bbb

    16. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    buildThe build process and cross-buildtopic-argument-clinictype-featureA feature request or enhancement

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions