GH-100930: Implement dataclass code caching - #92650

Could this be further sped up by having dataclasses.py come with a pre-seeded code cache from inlined code that'd already be part of the .pyc file thus avoiding runtime calls to exec() entirely for things that match its shapes?

Could this be further sped up by having dataclasses.py come with a pre-seeded code cache from inlined code that'd already be part of the .pyc file thus avoiding runtime calls to exec() entirely for things that match its shapes?

It could, but I'm not sure I'd want to include that in the initial implementation, mostly because it would be quite a bit of extra code and maintenance burden for a smallish gain that only pays off once (if at all).

What sorts of methods would we pre-generate? __init__, __repr__, and __eq__ for dataclasses with 2 <= n < 6 "simple" members and no defaults, maybe? That's twelve new functions that need to be written and maintained by hand, mostly based on an arbitrary heuristic:

def __init__(self, _field_0, _field_1):
    self._field_0 = _field_0
    self._field_1 = _field_1

_code_cache[<indented function source>] = __init__.__code__

def __repr__(self):
    return self.__class__.__qualname__ + f"(_field_0={self._field_0}, _field_1={self._field_1})"

_code_cache[<indented function source>] = __repr__.__code__

def __eq__(self, other):
    if other.__class__ is self.__class__:
        return (self._field_0, self._field_1) == (other._field_0, other._field_1)
    return NotImplemented

_code_cache[<indented function source>] = __eq__.__code__

<repeat for all other dataclass shapes we want to pre-seed>

Sure, we could rework this to make it more ergonomic:

# XXX: Make sure to regenerate this using
# Tools/scripts/generate_dataclasses_code_cache.py whenever the bytecode or the
# generated methods change!
_MARSHALLED_CODE_CACHE = b"..."
_code_cache = marshal.loads(_MARSHALLED_CODE_CACHE)

But in the end, the pre-seeded cache still runs the risk of falling out of sync with the actual generated methods if they change in the future. If not properly tested and maintained, it could lead to some extremely subtle bugs. So I'm just not sure it's worth it.

Closing, since it looks like the direction we're heading right now is batching up exec calls (#109870), rather than caching them.

The code is still here if we decide to do something like this in the future. :)

i've been investigating dataclass performance. per https://discuss.python.org/t/improving-dataclasses-startup-performance/15442/15, would it be possible to just cache the code generated for the dataclass somewhere, and then it gets compiled to pyc like other python files?