Skip to content

Type expression is coerced to a list of parameter arguments in substitution of ParamSpec #88964

Description

@serhiy-storchaka
BPO 44801
Nosy @gvanrossum, @ambv, @serhiy-storchaka, @miss-islington, @Fidget-Spinner
PRs
  • bpo-44801: Check arguments in substitution of ParamSpec in Callable #27585
  • [3.10] bpo-44801: Check arguments in substitution of ParamSpec in Callable (GH-27585) #27598
  • Dependencies
  • bpo-44793: Arguments ignored in substitution in typing.Callable
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-08-04.21:00:34.993>
    created_at = <Date 2021-08-01.07:08:58.391>
    labels = ['type-bug', 'library', '3.10', '3.11']
    title = 'Type expression is coerced to a list of parameter arguments in substitution of ParamSpec'
    updated_at = <Date 2021-08-04.21:00:34.992>
    user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2021-08-04.21:00:34.992>
    actor = 'lukasz.langa'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-08-04.21:00:34.993>
    closer = 'lukasz.langa'
    components = ['Library (Lib)']
    creation = <Date 2021-08-01.07:08:58.391>
    creator = 'serhiy.storchaka'
    dependencies = ['44793']
    files = []
    hgrepos = []
    issue_num = 44801
    keywords = ['patch']
    message_count = 5.0
    messages = ['398687', '398688', '398842', '398925', '398953']
    nosy_count = 5.0
    nosy_names = ['gvanrossum', 'lukasz.langa', 'serhiy.storchaka', 'miss-islington', 'kj']
    pr_nums = ['27585', '27598']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue44801'
    versions = ['Python 3.10', 'Python 3.11']

    Activity

    1. serhiy-storchaka commented on Aug 1, 2021

      @serhiy-storchaka
      MemberAuthor

      Type expression is coerced to a list of parameter arguments in substitution of ParamSpec. For example:

      >>> from typing import *
      >>> T = TypeVar('T')
      >>> P = ParamSpec('P')
      >>> C = Callable[P, T]
      >>> C[int, str]
      typing.Callable[[int], str]

      int becomes [int]. There is even a dedicated test for this.

      But it is not followed from PEP-612. Furthermore, it contradicts one of examples in the PEP:

      >>> class X(Generic[T, P]):
      ...     f: Callable[P, int]
      ...     x: T
      ... 
      >>> X[int, int]  # Should be rejected
      __main__.X[int, int]

      It makes the implementation (at least the code in bpo-44796) more complex and makes the user code more errorprone.

    2. added
      stdlibStandard Library Python modules in the Lib/ directory
      3.11only security fixes
      type-bugAn unexpected behavior, bug, or error
      on Aug 1, 2021
    3. Fidget-Spinner commented on Aug 1, 2021

      @Fidget-Spinner
      Member

      Type expression is coerced to a list of parameter arguments in substitution of ParamSpec.

      It's not, only the repr is like that. Internally it's not coerced.

      >>> C[int, str]
      typing.Callable[[int], str]
      >>> C[int, str].__args__
      (<class 'int'>, <class 'str'>)

      Because Callable's correct form is Callable[[type], type] (where type is not ParamSpec or Concatenate) so the repr reflects that.

      Internally, Callable also flattens the list of args:

      >>> Callable[[int, str], int].__args__
      (<class 'int'>, <class 'str'>, <class 'int'>)

      Because __args__ *must* be hashable and immutable. Otherwise, it will not work with Union or Literal. Some time ago we tried converting to nested tuple. But that was too difficult (and backwards incompatible) because every other typing operation expected __args__ to contain types, not a tuple.

    4. serhiy-storchaka commented on Aug 3, 2021

      @serhiy-storchaka
      MemberAuthor

      Thank you Ken Jin. So the problem is the __args__ (x, str) is interpreted differently depending on x, and after substituting x the interpretation can be changed. If x was ParamSpec, it was interpreted in one way, but if it becomes int, it is now interpreted in other way.

      The solution is to forbid substitution of P with wrong values (not parameters expression). Some normalization is also needed, before and after substitution.

      Other related example is:

      >>> from typing import *
      >>> P = ParamSpec("P")
      >>> class Z(Generic[P]): pass
      ... 
      >>> A = Z[[int]]
      >>> B = Z[int]
      >>> A
      __main__.Z[(<class 'int'>,)]
      >>> B
      __main__.Z[int]
      >>> A.__args__
      ((<class 'int'>,),)
      >>> B.__args__
      (<class 'int'>,)

      It is expected that A and B should the same.

    5. ambv commented on Aug 4, 2021

      @ambv
      Contributor

      New changeset 3875a69 by Serhiy Storchaka in branch 'main':
      bpo-44801: Check arguments in substitution of ParamSpec in Callable (GH-27585)
      3875a69

    6. miss-islington commented on Aug 4, 2021

      @miss-islington
      Contributor

      New changeset 536e35a by Miss Islington (bot) in branch '3.10':
      bpo-44801: Check arguments in substitution of ParamSpec in Callable (GH-27585)
      536e35a

    7. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.10 (EOL)end of life3.11only security fixesstdlibStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or error

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions