Skip to content
Next Next commit
fix(_state): copy_with's guard refuses a __new__, drops a harmless cl…
…ause

Re-measuring #546's guard after merge, clause by clause over
_UNGUARDED_EFFECT:

- The own-__dataclass_params__ check was the sole refuser only of an
  undecorated subclass borrowing the generated __init__, where replace
  runs that same __init__ and both copies build value=2. Nothing to
  guard, so it goes; the generated-__init__ check still refuses every
  harmful undecorated subclass.
- A __new__ above object passed every clause. replace calls cls(...),
  so it runs there and not in a field copy: a validating __new__ gave
  ValueError from replace and value=-1 from copy_with. It is refused
  now, with a row recording both outcomes.

Each remaining clause is the only one refusing at least one row. No
pipeline class is affected; call counts unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
  • Loading branch information
derek73 and claude committed Sep 29, 2026
commit 61da35a7059e0653e9fce9465e7d891468813d1d
1 change: 1 addition & 0 deletions docs/design/decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -1454,6 +1454,7 @@ Every number below is a py3.11 measurement of 2026-08-31, recomputable with `uv
RAISING OR LOWERING A ROW IS A DECISION, not a maintenance chore. Append here with the interpreter and the harness invocation, as above.

- 2026-09-26 #546 -- the stages copy their state with `_state.copy_with` instead of `dataclasses.replace`, and every row moves DOWN. `dataclasses.replace` walks `fields()` and calls `__init__` on each copy, three frames on 3.11 and 3.12 and four from 3.13 where `copy_with` is one. One parse of the reference name makes 18 copies: six of the state, one each from tokenize, segment, classify, group, assign and post_rules, and twelve of single tokens, six each from classify and assign (`extract_delimited` returns the state unchanged when there is no delimiter, and `script_segment` returns early on ASCII input). That is the whole of the drop, 18 x 2 = 36 frames on 3.11 and 3.12 and 18 x 3 = 54 from 3.13 (recompute: wrap `copy_with` with a counter in the eight stage modules and parse the reference name). A field copy builds what `replace` builds for a dataclass that is decorated itself rather than inheriting the decoration, keeps the generated `__init__`, and has no `__post_init__` and no `init=False` field. `_copyable_fields` checks exactly those four, and `_COPY_FIELDS` runs it over `WorkToken`, `PendingAmbiguity` and `ParseState` at import, so a class that stops qualifying fails there and `copy_with` copies nothing else. `test_the_guard_refuses_a_class_a_field_copy_would_get_wrong` records, for each refused shape, what `replace` builds and what an unguarded copy would build instead. To mypy, `copy_with` is `from dataclasses import replace as copy_with`, which keeps the dataclass plugin's keyword and type checks at every call site; an assignment (`copy_with = dataclasses.replace`) would not, since the plugin keys on the callee's full name (measured: a misspelled field and a wrong-typed value both pass through the assignment and both fail through the import). Measured 2026-09-26 with `uv run python tools/perf/call_count.py --against e0f1a2f`, each row on its own interpreter, parse/facade: 3.11 406/443 → 370/407, 3.12 384/421 → 348/385, 3.13, 3.14 and 3.15 402/439 → 348/385. The rows drop by 40 and 58 rather than 36 and 54 because e0f1a2f already read 4 under every row, inside the band, and the new rows are set to what the harness reads now. `_LINK_BASELINE`'s 64-link clause reads 2587 → 2301 on 3.11. By stage (`--stages`, py3.11, ms per 1000 parses of the reference name): group 19.6 → 17.3, classify 13.4 → 9.8, assign 11.1 → 8.0, tokenize 8.1 → 6.6, post_rules 7.9 → 6.5, segment 2.6 → 1.7. BEHAVIOR IDENTICAL: the differential gate's report at all five baselines matches e0f1a2f's line for line apart from the path header.
- 2026-09-29 -- the #546 guard, re-measured after merge, one clause out and one in. OUT: the check that the class carries its own `__dataclass_params__`. The only shape it refused on its own was an undecorated subclass borrowing the generated `__init__` (`class Sub(Base): __init__ = Base.__init__`), and there `replace` runs that same `__init__`, so `replace` and a field copy both build `value=2`: nothing to guard. Every harmful undecorated subclass writes its own `__init__` or inherits none, and the generated-`__init__` check refuses both. IN: a `__new__` defined anywhere above `object`. `replace` calls `cls(...)`, so that `__new__` runs there and not in a field copy; a frozen dataclass whose `__new__` rejects `value=-1` got `ValueError` from `replace` and `value=-1` from `copy_with`, and the guard accepted it. None of the three pipeline classes defines one. `_UNGUARDED_EFFECT` gains that row, and each of the four clauses left is now the only one refusing at least one row (measured by dropping each in turn over the table). Call counts unchanged, the guard running at import only.

### removed-v1-surface

Expand Down
12 changes: 7 additions & 5 deletions nameparser/_pipeline/_state.py
Original file line number Diff line number Diff line change
Expand Up @@ -218,15 +218,17 @@ class ParseState:
def _copyable_fields(cls: type) -> tuple[str, ...]:
"""The fields `copy_with` carries for `cls`, or TypeError where a
field copy would not build what `dataclasses.replace` builds: the
class must be decorated itself (not inherit the decoration) and keep
the generated `__init__`, with no `__post_init__` and no
`init=False` field."""
params = cls.__dict__.get("__dataclass_params__")
class must keep the generated `__init__` in its own body and
`object.__new__`, with no `__post_init__` and no `init=False`
field."""
init = cls.__dict__.get("__init__")
# dataclasses compiles the __init__ it generates from a string; one
# written in the class body carries its source file instead.
generated = init is not None and init.__code__.co_filename == "<string>"
if (params is None or not generated
# replace calls cls(...), so a __new__ defined above object runs
# there and would not run here.
own_new = any("__new__" in vars(k) for k in cls.__mro__[:-1])
if (not generated or own_new
or hasattr(cls, "__post_init__")
or not all(f.init for f in dataclasses.fields(cls))):
raise TypeError(
Expand Down
12 changes: 12 additions & 0 deletions tests/v2/pipeline/test_state.py
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,16 @@ def __init__(self, value: int) -> None:
self.extra = "set by __init__"


@dataclasses.dataclass(frozen=True)
class _OwnNew:
value: int

def __new__(cls, value: int) -> "_OwnNew":
if value < 0:
raise ValueError("negative")
return super().__new__(cls)


def _derived_with_doubled_set() -> _Derived:
obj = _Derived(1)
object.__setattr__(obj, "doubled", 2)
Expand Down Expand Up @@ -127,6 +137,8 @@ def _outcome(copy: Callable[[], object]) -> object:
"ValueError", {"value": -1}),
("validating __init__", _OwnInit(1), {"value": -1},
"ValueError", {"value": -1}),
("validating __new__", _OwnNew(1), {"value": -1},
"ValueError", {"value": -1}),
("init=False field", _derived_with_doubled_set(), {"value": 3},
{"value": 3, "doubled": 0}, {"value": 3, "doubled": 2}),
("undecorated subclass", _Undecorated(1), {"value": 2},
Expand Down