Cannot use Literal when I use typing-extension==4.6.0

Initial Checks

I have searched GitHub for a duplicate issue and I'm sure this is something new
I have searched Google & StackOverflow for a solution and couldn't find anything
I have read and followed the docs and still think this is a bug
I am confident that the issue is with pydantic (not my code, or another library in the ecosystem like FastAPI or mypy)

Description

I update my typing extention 4.6.0, I face this error

.venv/lib/python3.9/site-packages/ddtrace/internal/module.py:216: in _exec_module
    self.loader.exec_module(module)
src/entry.py:3: in <module>
    from apis import router as api_router
.venv/lib/python3.9/site-packages/ddtrace/internal/module.py:216: in _exec_module
    self.loader.exec_module(module)
src/apis/__init__.py:5: in <module>
    from .recsys import router as recsys_router
.venv/lib/python3.9/site-packages/ddtrace/internal/module.py:216: in _exec_module
    self.loader.exec_module(module)
src/apis/recsys/__init__.py:3: in <module>
    from .course.endpoints import router as course_router
.venv/lib/python3.9/site-packages/ddtrace/internal/module.py:216: in _exec_module
    self.loader.exec_module(module)
src/apis/recsys/course/__init__.py:1: in <module>
    from . import cell
.venv/lib/python3.9/site-packages/ddtrace/internal/module.py:216: in _exec_module
    self.loader.exec_module(module)
src/apis/recsys/course/cell/__init__.py:1: in <module>
    from . import container
.venv/lib/python3.9/site-packages/ddtrace/internal/module.py:216: in _exec_module
    self.loader.exec_module(module)
src/apis/recsys/course/cell/container.py:4: in <module>
    from core.models.feature_store.cell_dtos import CourseCellDTO
.venv/lib/python3.9/site-packages/ddtrace/internal/module.py:216: in _exec_module
    self.loader.exec_module(module)
src/core/models/feature_store/cell_dtos.py:14: in <module>
    class VocaCellDTO(BaseModel):
pydantic/main.py:197: in pydantic.main.ModelMetaclass.__new__
    ???
pydantic/fields.py:506: in pydantic.fields.ModelField.infer
    ???
pydantic/fields.py:436: in pydantic.fields.ModelField.__init__
    ???
pydantic/fields.py:552: in pydantic.fields.ModelField.prepare
    ???
pydantic/fields.py:668: in pydantic.fields.ModelField._type_analysis
    ???

Example Code

from enum import Enum
from typing import Literal

from pydantic import BaseModel


class MyEum(str, Enum):
    T1 = "T1"
    T2 = "T2"


class MyType(BaseModel):
    type: Literal[MyEum.T1] = MyEum.T1
    dalc: str

then I meet

Traceback (most recent call last):
  File "/Users/project/test.py", line 12, in <module>
    class MyType(BaseModel):
  File "pydantic/main.py", line 197, in pydantic.main.ModelMetaclass.__new__
  File "pydantic/fields.py", line 506, in pydantic.fields.ModelField.infer
  File "pydantic/fields.py", line 436, in pydantic.fields.ModelField.__init__
  File "pydantic/fields.py", line 552, in pydantic.fields.ModelField.prepare
  File "pydantic/fields.py", line 668, in pydantic.fields.ModelField._type_analysis
  File "/Users/project/lib/python3.9/typing.py", line 852, in __subclasscheck__
    return issubclass(cls, self.__origin__)
TypeError: issubclass() arg 1 must be a class

Python, Pydantic & OS Version

pydantic version: 1.10.7
            pydantic compiled: True
               python version: 3.9.13 (main, Aug 25 2022, 18:24:45)  [Clang 12.0.0 ]
                     platform: macOS-12.6.1-arm64-arm-64bit
     optional deps. installed: ['dotenv', 'typing-extensions']

Affected Components

Just started hitting this as well.

I'm getting this too. Even on pydantic v2.

As a workaround:

upgrade to python 3.10. This issue happens on 3.9 but not 3.10; or
change from typing import Literal to from typing_extensions import Literal

The docs for pydantic do say:

This is a new feature of the Python standard library as of Python 3.8; prior to Python 3.8, it requires the typing-extensions package.

But I find that documentation ambiguous. It sounds like the requirement is that typing_extensions merely be installed, and then pydantic will choose it if required. But that doesn't seem to be the case.

So I think we need clearer documentation.

I think the source of the bug is this line:

https://github.com/pydantic/pydantic/blob/bf6b884563e7c635b9d811e97d2fdb51805398b1/pydantic/typing.py#LL412C57-L412C64

from typing_extensions import (
...
    Literal,
...
)
...
def is_literal_type(type_: Type[Any]) -> bool:
    return Literal is not None and get_origin(type_) is Literal

So I think this is a bug in pydantic. is_literal_type is comparing only to typing_extensions.Literal, not typing.Literal.

I think a patch looks something like:

def is_literal_type(type_: Type[Any]) -> bool:
    return Literal is not None and get_origin(type_) in [typing.Literal, typing_extensions.Literal]

Can confirm, the bug appears to occur with typing-extension==4.6.0.

Minimal steps to reproduce the issue (typing_extensions==4.6.0, pydantic==1.10.7, Python 3.8.10):

from typing import Literal
from pydantic import BaseModel


class MyType(BaseModel):
    literal: Literal["foobar"] = "foobar"

raises:
TypeError: issubclass() arg 1 must be a class

@hramezani

Thanks for reporting, @hramezani will take a look 🙏.

I could confirm that this bug is caused by this commit python/typing_extensions@fb37b2e

@mdavis-xyz

@mdavis-xyz thanks for your explanation. seems your suggestion will fix the bug. Do you want to create a patch for fixing? otherwise, I will create a patch.

This has also been reported over at typing_extensions FYI: python/typing_extensions#179.

I think the source of the bug is this line:

https://github.com/pydantic/pydantic/blob/bf6b884563e7c635b9d811e97d2fdb51805398b1/pydantic/typing.py#LL412C57-L412C64
from typing_extensions import (
...
    Literal,
...
)
...
def is_literal_type(type_: Type[Any]) -> bool:
    return Literal is not None and get_origin(type_) is Literal
So I think this is a bug in pydantic. is_literal_type is comparing only to typing_extensions.Literal, not typing.Literal.

I think a patch looks something like:
def is_literal_type(type_: Type[Any]) -> bool:
    return Literal is not None and get_origin(type_) in [typing.Literal, typing_extensions.Literal]

If this analysis is correct (and I haven't checked, but it looks very plausible), you'd probably want to do something like this, in order to retain compatibility with Python 3.7 (where typing.Literal isn't a thing yet):

_Literal_types = {typing_extensions.Literal}

if hasattr(typing, "Literal"):
    _Literal_types.add(typing.Literal)

def is_literal_type(type_: Type[Any]) -> bool:
    return Literal is not None and get_origin(type_) in _Literal_types

(Not sure under what conditions Literal can be None btw? Seems like a slight oddity in the code as it's currently written :)

List of workarounds:

Workaround 1: pin requirement typing_extensions==4.5.0
Workaround 2: use Python >= 3.10.1 (works because typing_extensions has some version-specific logic)
Workaround 3: at the start of your program, before any imports, patch typing.Literal to refer to typing_extensions.Literal. (Full patch example below.) You only need to do this in the initial module.
Workaround 4: replace every from typing import Literal with from typing_extensions import Literal.

The patch of workaround 3:

# ---- Start of patch ----
# Workaround for https://github.com/pydantic/pydantic/issues/5821
#
# Add this code once at the start of your program. It plays nice with mypy.
#
# It patches `typing.Literal` to refer to `typing_extensions.Literal`,
# and thereby avoids a bug in Pydantic<=1.10.7 code for is_literal_type.
import typing_extensions
import typing
typing.Literal = typing_extensions.Literal  # type: ignore
# ---- End of patch ----

# ---- Your program goes here ----
from pydantic import BaseModel
from typing import Literal

class Hello(BaseModel):
    x: Literal['x']   # patch prevents error
    y: typing.Literal['y]   # patch prevents error

Pin typing_extensions < 4.6.0

Pin typing_extensions < 4.6.0 (#14407)

Released.

@dmontagu

@dmontagu @samuelcolvin yall the best!

How about yanking 1.10.7 from pypi? (as in the footsteps of 1.10.3)

But this is not a bug only in 1.10.7 It is present in more versions.

@jnawk

@jnawk @Czaki the reason that 1.10.3 was yanked was because 1.10.4 updated the minimum-allowed-version of typing_extensions. As a result, it was incompatible to use pydantic 1.10.4 with other libraries that had pinned the version of typing_extensions below 4.2.0.

However, the release of pydantic 1.10.8 did not set any new minimum dependency versions (it just fixed the bug). So unless I'm missing something, I don't think the same reasons for yanking the 1.10.3 pydantic version apply here.

deps: floor pydantic

NOJIRA: Pin typing extensions to 4.5.0 to workaround bug in pydantic

deps: floor pydantic

Unpin typing_extensions due to pydantic issue

Pin typing_extensions < 4.6.0 (#14407)

Resolves issue related to use of Literal (#303)

Just to add info: From my tests, this bug is present for all installs I tried with Python 3.7 as well as any combination of the following:

Python 3.8[.11] and 3.9[.16]
typing-extensions >= 4.6
Pydantic 1.8 - 1.10.7

It appears to be not an issue for any version of Pydantic 1.10.8+ or Python 3.10[.6]+. It also appears to not be a problem for typing-extensions 4.4 or 4.5. I did not test Pydantic < 1.8 or typing-extensions < 4.4. I only tested incremental minor versions of typing-extensions (4.6.0, 4.7.0 etc) and the specific Python 3.8 and 3.9 versions noted above.

Navigation Menu

Repository navigation

Cannot use Literal when I use typing-extension==4.6.0 #5821

Initial Checks

Description

Example Code

Python, Pydantic & OS Version

Affected Components

7 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Cannot use Literal when I use typing-extension==4.6.0 #5821

Description

Initial Checks

Description

Example Code

Python, Pydantic & OS Version

Affected Components

Activity

cmditch commented on May 23, 2023

mdavis-xyz commented on May 23, 2023

AiyionPrime commented on May 23, 2023

eachimei commented on May 23, 2023

samuelcolvin commented on May 23, 2023

Czaki commented on May 23, 2023

hramezani commented on May 23, 2023

AlexWaygood commented on May 23, 2023

sietse commented on May 23, 2023

7 remaining items

samuelcolvin commented on May 23, 2023

jessekrubin commented on May 23, 2023

jnawk commented on May 23, 2023

Czaki commented on May 23, 2023

dmontagu commented on May 23, 2023

RandallPittmanOrSt commented on Apr 10, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions