Skip to content

Help text of builtin functions – missing signatures #107526

Closed
Closed
Help text of builtin functions – missing signatures#107526
Assignees
JelleZijlstrasobolevn
Labels
3.12only security fixes3.13only security fixesrelease-blockertype-bugAn unexpected behavior, bug, or error

Description

@frenzymadness
frenzymadness
opened on Aug 1, 2023

I don't understand the deep details here but I think something went wrong in commit bdfb694 when some of the builtin functions where transferred to argument clinic.

The problem I see is in the help texts of those functions. Here are a few examples of help(<function>):

iter

Python 3.11.4

Help on built-in function iter in module builtins:

iter(...)
    iter(iterable) -> iterator
    iter(callable, sentinel) -> iterator
    
    Get an iterator from an object.  In the first form, the argument must
    supply its own iterator, or be a sequence.
    In the second form, the callable is called until it returns the sentinel.

Python 3.12.0b4

Help on built-in function iter in module builtins:

iter(...)
    Get an iterator from an object.

    In the first form, the argument must supply its own iterator, or be a sequence.
    In the second form, the callable is called until it returns the sentinel.

As you can see, the help in 3.12 is talking about two forms but there are no signatures so it doesn't make sense.

next

Python 3.11.4

Help on built-in function next in module builtins:

next(...)
    next(iterator[, default])
    
    Return the next item from the iterator. If default is given and the iterator
    is exhausted, it is returned instead of raising StopIteration.

Python 3.12.0b4

Help on built-in function next in module builtins:

next(...)
    Return the next item from the iterator.

    If default is given and the iterator is exhausted,
    it is returned instead of raising StopIteration.

Again, the help text talks about iterator and default but missing signature means that the reader might not know what those are.

getattr

Python 3.11.4

Help on built-in function getattr in module builtins:

getattr(...)
    getattr(object, name[, default]) -> value
    
    Get a named attribute from an object; getattr(x, 'y') is equivalent to x.y.
    When a default argument is given, it is returned when the attribute doesn't
    exist; without it, an exception is raised in that case.

Python 3.12.0b4

Help on built-in function getattr in module builtins:

getattr(...)
    Get a named attribute from an object.

    getattr(x, 'y') is equivalent to x.y
    When a default argument is given, it is returned when the attribute doesn't
    exist; without it, an exception is raised in that case.

Again, a similar case here.

Linked PRs

Activity

  1. frenzymadness
    added
    type-bugAn unexpected behavior, bug, or error
    on Aug 1, 2023
  2. AlexWaygood
    assigned
    JelleZijlstra
    and
    sobolevn
    on Aug 1, 2023

  3. sobolevn commented on Aug 1, 2023

    @sobolevn
    sobolevn
    on Aug 1, 2023
    Member

    Yes, this is because of NULL default :(

    arg: object = NULL

  4. frenzymadness commented on Aug 1, 2023

    @frenzymadness
    frenzymadness
    on Aug 1, 2023
    ContributorAuthor

    Is there any way to fix it?

  5. sobolevn commented on Aug 1, 2023

    @sobolevn
    sobolevn
    on Aug 1, 2023
    Member

    I've submitted https://discuss.python.org/t/ac-null-defaults-prevent-correct-signatures-lets-add-inspect-unrepresentable-to-fix-this/30753

    I wanted to do it for quite a long time, sorry for ignoring this item in my backlog :)

  6. encukou commented on Aug 1, 2023

    @encukou
    encukou
    on Aug 1, 2023
    Member

    Is this feasible for 3.11?
    If not, should we add the signatures to the doc text manually, in these cases where AC doesn't generate anything?

    FWIW, this breaks Jedi, which currently parses the manually-formatted signatures in docstrings: davidhalter/jedi#1952
    They would be happy to move to inspect.signature once that works. Until it does, parsing __doc__ seems like a sensible workaround, so this might be considered a regression.
    So I'm flagging this as a potential blocker.

  7. encukou
    added
    release-blocker
    needs backport to 3.11only security fixes
    on Aug 1, 2023
  8. ezio-melotti
    added this to Release and Deferred blockers 🚫on Aug 1, 2023
  9. github-project-automation
    moved this to Todo in Release and Deferred blockers 🚫on Aug 1, 2023
  10. AlexWaygood
    added
    3.11only security fixes
    3.12only security fixes
    3.13only security fixes
    and removed
    needs backport to 3.11only security fixes
    on Aug 1, 2023

  11. 1 remaining item

  12. encukou commented on Aug 1, 2023

    @encukou
    encukou
    on Aug 1, 2023
    Member

    I meant 3.12, not 3.11. Got my version numbers mixed up, sorry!

  13. encukou
    removed
    3.11only security fixes
    on Aug 1, 2023

  14. JelleZijlstra commented on Aug 1, 2023

    @JelleZijlstra
    JelleZijlstra
    on Aug 1, 2023
    Member

    The fix @sobolevn proposes doesn't seem feasible for 3.12, as it requires introducing a new object in the inspect module. As a less invasive fix, maybe we can directly add these signatures to the docstrings?

  15. sobolevn commented on Aug 1, 2023

    @sobolevn
    sobolevn
    on Aug 1, 2023
    Member

    @JelleZijlstra that's exactly what I am proposing for 3.11 and 3.12 in #107526 (comment) :)

  16. sobolevn commented on Aug 1, 2023

    @sobolevn
    sobolevn
    on Aug 1, 2023
    Member

    Ok, I think that we would need to revert these changes. Here's why:

    1. We cannot simply add signatures back in AC. Here's what is generated:
    /*[clinic input]
dir as builtin_dir

    arg: object = NULL
    /

dir([object]) -> list of strings

Show attributes of an object.

...
[clinic start generated code]*/

static PyObject *
builtin_dir_impl(PyObject *module, PyObject *arg)
/*[clinic end generated code: output=24f2c7a52c1e3b08 input=2e2cb290445c88f3]*/

    outputs:

    PyDoc_STRVAR(builtin_dir__doc__,
"dir($module, arg=<unrepresentable>, /)\n"
"--\n"
"\n"
"dir([object]) -> list of strings\n"
"\n"
"Show attributes of an object.\n"
"\n"
"...");

    But, that's not what we want!

    1. Argument clinic has only_docstring option for functions, but I will need to rework how it sets this value to True to be able to use it. This would require extra testing and stuff
    2. I hope that https://discuss.python.org/t/ac-null-defaults-prevent-correct-signatures-lets-add-inspect-unrepresentable-to-fix-this/30753 will land for new code, so I won't have to change the clinic code
  17. sobolevn
    added a commit that references this issue on Aug 1, 2023

    pythongh-107526: Revert "pythongh-100357: Convert several functions i…

    db35ad9

  18. davidhalter commented on Aug 2, 2023

    @davidhalter
    davidhalter
    on Aug 2, 2023

    FYI I don't think the effects on Jedi are extreme (speaking as its author). It's just that it's a bit unfortunate that the signatures won't be shown anymore in that case for 3.12. So yes, it's a regression, but not a problematic regression. So if argument clinic helps you here with other issues, it might just be worth keeping it. IMO the more problematic change here might be that the docstrings are just not as good as they were previously. Especially for iter the docstring talks about In the first form, [...]. What is the first form? In the other docstrings there are weird comments as well.

    In the end it would be really nice if these functions were eventually supported by inspect.signature, but I don't think you need to hurry there either.

  19. serhiy-storchaka commented on Aug 9, 2023

    @serhiy-storchaka
    serhiy-storchaka
    on Aug 9, 2023
    Member

    #107782 can help.

  20. JelleZijlstra
    added a commit that references this issue on Aug 21, 2023

    gh-107526: Revert "gh-100357: Convert several functions in bltinsmodu…

    db6dc6c
  21. corona10
    added a commit that references this issue on Aug 21, 2023

    gh-107526: Fix test_module_level_callable_unrepresentable_default (gh…

    04f7875

  22. JelleZijlstra commented on Nov 9, 2023

    @JelleZijlstra
    JelleZijlstra
    on Nov 9, 2023
    Member

    The immediate issue was fixed here. We still have the broader issue that inspect.signature can't handle `', but that's tracked in #87233.

  23. JelleZijlstra
    closed this as completedon Nov 9, 2023
  24. github-project-automation
    moved this from Todo to Done in Release and Deferred blockers 🚫on Nov 9, 2023
  25. sobolevn
    added 2 commits that reference this issue on Feb 20, 2025

    [3.12] pythongh-107526: Revert "pythongh-100357: Convert several func…

    54efeaf

    [3.12] gh-107526: Revert "gh-100357: Convert several functions in blt…

    f841099
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

3.12only security fixes3.13only security fixesrelease-blockertype-bugAn unexpected behavior, bug, or error

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions