Repository navigation
Add introspection information for builtins #63873
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/larryhastings'
closed_at = <Date 2013-11-23.23:39:10.883>
created_at = <Date 2013-11-20.23:52:26.200>
labels = ['interpreter-core', 'type-bug']
title = 'Add introspection information for builtins'
updated_at = <Date 2013-11-23.23:39:10.883>
user = 'https://github.com/larryhastings'bugs.python.org fields:
activity = <Date 2013-11-23.23:39:10.883>
actor = 'larry'
assignee = 'larry'
closed = True
closed_date = <Date 2013-11-23.23:39:10.883>
closer = 'larry'
components = ['Interpreter Core']
creation = <Date 2013-11-20.23:52:26.200>
creator = 'larry'
dependencies = []
files = ['32739', '32763', '32785', '32815']
hgrepos = []
issue_num = 19674
keywords = []
message_count = 16.0
messages = ['203549', '203550', '203551', '203575', '203576', '203583', '203607', '203667', '203672', '203684', '203797', '203827', '203908', '204026', '204120', '204129']
nosy_count = 8.0
nosy_names = ['gvanrossum', 'barry', 'brett.cannon', 'georg.brandl', 'ncoghlan', 'larry', 'python-dev', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue19674'
versions = ['Python 3.4']
Activity
larryhastings commentedon Nov 20, 2013
Let's see if we can get introspection information for builtins using the Clinic for 3.4.
Georg suggested part of the approach while we were hanging out in Tokyo. I'd considered it previously before but dispensed with the idea because it seemed too loopy. Actually it seems to work great.
The approach:
Clinic generates an extra first line for the docstring, that looks like "def (...)\n". (Note: no function name!)
The PyCFunctionObject __doc__ getter detects this line and skips it if present.
Add a new getter to PyCFunctionObject, which I've called "__textsig__", that returns this first line if present in the docstring. (It skips the "def " at the front, and clips it at the newline.)
inspect now notices if it's passed in a PyCFunctionObject. If it gets one, it checks to see if it has a valid __textsig__. If it does, it parses it (using ast.parse) and produces a valid signature.
Advantages of this approach:
It was easy to do and took very few lines of code.
For signatures that are impossible to convert to Clinic, we can
write the metadata by hand.
Disadvantages of this approach:
The next step is probably to convert pydoc to use inspect.signature instead of the manky old methods it currently uses. After that, clean up the patch, and add a unit test or two.
What do you think?
larryhastings commentedon Nov 20, 2013
If you review, you can ignore the changes to the .c files, those only got touched because of the new autogenerated "def (" lines in the docstrings. No other changes there.
larryhastings commentedon Nov 21, 2013
(Well, except for Object/methodobject.c.)
gvanrossum commentedon Nov 21, 2013
I think there's a similar but slightly different convention in Sphinx
autodoc -- the first line of the docstring can be "name(...)" . Isn't that
better than def?
larryhastings commentedon Nov 21, 2013
I was deliberately trying to avoid something that a person might do by accident. Also, "def (" is consistently only five bytes, whereas "pterodactyl (" or whatever will often be much longer, in case static data size is a concern.
But I could switch it to that if you think that's better. Certainly it's a tiny amount more readable.
(I don't think this would make the docstrings compatible with sphinx autodoc though.)
serhiy-storchaka commentedon Nov 21, 2013
Great! See also bpo-16842 and bpo-16638.
ncoghlan commentedon Nov 21, 2013
Sounds good to me (with either Larry's or Guido's spelling).
We should ensure this still works with the config option that disables docstrings entirely.
brettcannon commentedon Nov 21, 2013
For Nick: "still works" as in the metadata is still available without docstrings or "still works" as in it won't crash?
larryhastings commentedon Nov 21, 2013
I changed it to "<callablename>(". Patch attached. Still works.
ncoghlan commentedon Nov 21, 2013
"still works" as in "doesn't crash and the docstrings are still missing to
save memory".
larryhastings commentedon Nov 22, 2013
Anybody have a better name for __textsig__ ?
brettcannon commentedon Nov 22, 2013
__signature_text__? __textsignature__? No need to abbreviate that much since there is no built-in function equivalent and people are not expected to work with it.
larryhastings commentedon Nov 22, 2013
I went with __text_signature__. I also did some more polishing, and added / fixed the unit tests, and even added a Misc/NEWS. Is it good enough to go in before the beta?
ncoghlan commentedon Nov 23, 2013
+1 from me (I see this as the key benefit of argument clinic).
For 3.5, we can look at ducktyping on the attribute, but for now I think
it's worth keeping that text format internal to CPython.
larryhastings commentedon Nov 23, 2013
Fresh patch attached. pydoc now uses inspect.signature instead of inspect.getfullargspec to generate the arguments for the function, and supports builtins. That's everything :D
Planning on checking this in pretty soon, to get it in for beta (which hopefully still gets tagged today).
python-dev commentedon Nov 23, 2013
New changeset 78ec18f5cb45 by Larry Hastings in branch 'default':
Issue bpo-19674: inspect.signature() now produces a correct signature
http://hg.python.org/cpython/rev/78ec18f5cb45