typing.Union does not support attribute assignment post gh-105511

@JelleZijlstra

Bug report

Bug description:

Before #105511, typing.Union is implemented in Python, and custom attributes can be assigned to a Union variable.

gh-105499: Merge typing.Union and types.UnionType #105511

ta = Union[int, str]
ta.__some_attribute__ = (int, str)

However, after #105511, typing.Union is now implemented in C, and no attributes can be assigned to it. The UnionType does not allow subclassing; users cannot bypass this using subclassing.

$ ipython
Python 3.14.0a6+ experimental free-threading build (heads/main:1755157207c, Apr  6 2025, 02:21:10) [Clang 17.0.0 (clang-1700.0.13.3)]
Type 'copyright', 'credits' or 'license' for more information
IPython 9.0.2 -- An enhanced Interactive Python. Type '?' for help.
Tip: Use `object?` to see the help on `object`, `object??` to view it's source

In [1]: import optree

In [2]: optree.PyTree[int]
/Users/PanXuehai/Projects/optree/env/lib/python3.14t/site-packages/IPython/core/interactiveshell.py:3111: SyntaxWarning: 'return' in a 'finally' block
  return result
╭────────────────────────────────────────────────────────────────────── Traceback (most recent call last) ───────────────────────────────────────────────────────────────────────╮
│ in <module>:1                                                                                                                                                                  │
│                                                                                                                                                                                │
│ /Users/PanXuehai/Projects/cpython/.pydev/lib/python3.14t/typing.py:398 in inner                                                                                                │
│                                                                                                                                                                                │
│    395 │   │   @functools.wraps(func)                                                                                                                                          │
│    396 │   │   def inner(*args, **kwds):                                                                                                                                       │
│    397 │   │   │   try:                                                                                                                                                        │
│ ❱  398 │   │   │   │   return _caches[func](*args, **kwds)                                                                                                                     │
│    399 │   │   │   except TypeError:                                                                                                                                           │
│    400 │   │   │   │   pass  # All real errors (not unhashable args) are raised below.                                                                                         │
│    401 │   │   │   return func(*args, **kwds)                                                                                                                                  │
│                                                                                                                                                                                │
│ /Users/PanXuehai/Projects/optree/optree/typing.py:254 in __class_getitem__                                                                                                     │
│                                                                                                                                                                                │
│   251 │   │   │   Deque[recurse_ref],  # type: ignore[valid-type]                                                                                                              │
│   252 │   │   │   CustomTreeNode[recurse_ref],  # type: ignore[valid-type]                                                                                                     │
│   253 │   │   ]                                                                                                                                                                │
│ ❱ 254 │   │   pytree_alias.__pytree_args__ = item  # type: ignore[attr-defined]                                                                                                │
│   255 │   │                                                                                                                                                                    │
│   256 │   │   # pylint: disable-next=no-member                                                                                                                                 │
│   257 │   │   original_copy_with = pytree_alias.copy_with  # type: ignore[attr-defined]                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
AttributeError: 'typing.Union' object has no attribute '__pytree_args__' and no __dict__ for setting new attributes

In [3]: class Foo(type(int | str)):
   ...:     pass
   ...:     
╭────────────────────────────────────────────────────────────────────── Traceback (most recent call last) ───────────────────────────────────────────────────────────────────────╮
│ in <module>:1                                                                                                                                                                  │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: type 'typing.Union' is not an acceptable base type

Repro:

./python -m pip install -v optree
./python -c 'import optree; optree.PyTree[int]'

Source: https://github.com/metaopt/optree/blob/v0.15.0/optree/typing.py#L246-L254

cc @JelleZijlstra

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

Was it designed to support custom attributes? is it a documented behavior? (I don't know so I'm just asking if this was deliberate)

Yeah, don't do that. Setting arbitrary attributes on Union objects (or any other typing objects) was never supported and if you do that sort of thing, be prepared for your code to break.

That said, we could allow setting arbitrary attributes again by adding a managed __dict__ to union objects, at a cost of increasing memory usage for every other user of unions. I'd be curious to hear opinions on whether that's worth it.

at a cost of increasing memory usage for every other user of unions

How much of a cost are we talking about? I think you mentioned something about the fact that unions' size is smaller (or was it larger?) but I can't find the issue (I think I read this yesterday somewhere). If we're only talking about a few bytes when it's not used, then why not. But if we're doubling the union sizes I think it's maybe not worth.

Now, are there other projects for which this could be needed? if there is enough usage, and from large libraries, then maybe it's a possibility? By the way, for what reason do you actually need unions to support assignment?

gh-132139: Document that you can no longer set attributes on unions (#…

pythongh-132139: 3.14 what's new: elaborate on why you can no longer …

gh-132139: 3.14 what's new: elaborate on why you can no longer set `U…

Another interesting side effect of this is for "public API specifications".
Systems like sphinx considers an API public in the current file if the API's __module__ attribute is set to the current file.

We use this in PyTorch in particular to define types, make them public API from sphinx point of view to properly document. An example is here: https://github.com/pytorch/pytorch/blob/aacb9440791f4e9567e2123f19bb062e75c2d242/torch/ao/quantization/__init__.py#L34-L36

The logic at

cpython/Objects/unionobject.c

Lines 400 to 403 in c7d24b8

    
           static const char* const cls_attrs[] = { 
        
                   "__module__",  // Required for compatibility with typing module 
        
                   NULL, 
        
           };

makes the behavior a bit surprising as accessing the __module__ attribute works but trying to set it fails with 'typing.Union' object has no attribute '__module__' and no __dict__ for setting new attributes (the attribute actually does exist).

Curious to hear what you think here?
Are we misguided on trying to set the __module__ attribute as a marker of public APIs for the Union object? Or we should try and find a way to enable this behavior?

@albanD

We use this in PyTorch in particular to define types, make them public API from sphinx point of view to properly document.

@albanD If it is only related to the docs build, I have worked around this by adding a custom typehint_formmatter and saving the Union objects in a weakrefset:

https://github.com/metaopt/optree/blob/ebd1d78d5780b9ad8dbbaba89ecedfee3061d84d/docs/source/conf.py#L195-L213

def typehints_formatter(annotation, config=None):
    if 'optree' not in sys.modules:
        sys.path.insert(0, str(PROJECT_ROOT))


    import optree


    if (
        isinstance(annotation, type(typing.Union[int, str]))
        and typing.get_origin(annotation) is typing.Union
        and annotation in optree.PyTree.__instances__
    ):
        param, name = optree.PyTree.__instances__[annotation]
        if name is not None:
            return f':py:class:`{name}`'


        from sphinx_autodoc_typehints import format_annotation


        return rf':py:class:`PyTree` \[{format_annotation(param, config=config)}]'
    return None

The keys of the WeakKeyDictionary are the set of Union instances.

https://github.com/metaopt/optree/blob/ebd1d78d5780b9ad8dbbaba89ecedfee3061d84d/optree/typing.py#L205-L211
https://github.com/metaopt/optree/blob/ebd1d78d5780b9ad8dbbaba89ecedfee3061d84d/optree/typing.py#L273-L275

The logic at

cpython/Objects/unionobject.c

Lines 400 to 403 in c7d24b8

static const char* const cls_attrs[] = {
"module", // Required for compatibility with typing module
NULL,
};
makes the behavior a bit surprising as accessing the __module__ attribute works but trying to set it fails with 'typing.Union' object has no attribute '__module__' and no __dict__ for setting new attributes (the attribute actually does exist).

__module__ is a class attribute. You cannot assign it via an instance.

If it is only related to the docs build, I have worked around this by adding a custom typehint_formmatter and saving the Union objects in a weakrefset:

The problem is that it's not just that. We also have programmatic checks to ensure that things in the __all__ field matches with the __module__. But since I cannot change the __module__, I have to remove the Union from the __all__ field and thus import * from that module changes behavior.

Also given how it's defined you can't subclass Union to add a __dict__ to it either.

Fix compilation and "import torch" issues for cpython 3.14 (#158184)

To recapitulate why I am not interested in changing this:

This was never supported or documented in any way.
Reliance on this misfeature would have interacted poorly with the cache that existed before 3.14; if multiple modules each create say a Union[int, str] and set its __module__, they would have clobbered each other.
This only worked for some kinds of types supported by the type system; for example, unions created through the | syntax and types.GenericAlias don't support attribute assignments; setting a __module__ attribute on an Annotated object sets its for all Annotated objects.
Adding this now would require adding more memory usage to the union object, so all users would have to pay extra in memory usage.

As an alternative, I'd suggest using the type statement introduced in Python 3.12. Type aliases created through this feature automatically get the __module__ they were created in.

If you need to support 3.11 and earlier, you can use:

if sys.version_info >= (3, 12):
    exec("type Alias = A | B")
else:
    Alias = Union[A, B]
    Alias.__module__ = __name__

Thanks for the in-depth answer. This all makes sense.

If you need to support 3.11 and earlier, you can use:

I'm afraid that doesn't work as the "type" statement will through an error in <=3.11 even if not executed:

$ cat foo.py 
import sys

if sys.version_info >= (3, 12):
    type Alias = A | B
else:
    Alias = Union[A, B]
    Alias.__module__ = __name__
$ python3.10 foo.py 
  File "/home/albandes/local/pytorch/3.10_release_source/foo.py", line 4
    type Alias = A | B
         ^^^^^
SyntaxError: invalid syntax
$ python3.13 foo.py 
$

That is a great feature that would definitely solve our issue otherwise!

Sorry yes, you'd need to use exec. I was thinking about that but forgot to write it when I wrote the comment :). Fixed above.

@JelleZijlstra

Thanks for the quick answer @JelleZijlstra , given that exec makes our security people jumpy, I was trying to go with the following. Any concern with that approach?

if sys.version_info >= (3, 12):
    from typing import TypeAliasType
    Alias = TypeAliasType("Alias", A | B)
else:
    Alias = Union[A, B]
    Alias.__module__ = __name__

types.Union no longer supports attribute assignment

Navigation Menu

Repository navigation

`typing.Union` does not support attribute assignment post gh-105511 #132139

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

11 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	static const char* const cls_attrs[] = {
	"__module__", // Required for compatibility with typing module
	NULL,
	};

typing.Union does not support attribute assignment post gh-105511 #132139

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

picnixz commented on Apr 5, 2025

JelleZijlstra commented on Apr 5, 2025

picnixz commented on Apr 5, 2025

11 remaining items

albanD commented on Jul 12, 2025

XuehaiPan commented on Jul 12, 2025

XuehaiPan commented on Jul 12, 2025

albanD commented on Jul 14, 2025

JelleZijlstra commented on Jul 16, 2025

albanD commented on Jul 16, 2025

JelleZijlstra commented on Jul 16, 2025

albanD commented on Jul 17, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

`typing.Union` does not support attribute assignment post gh-105511 #132139