Skip to content

feat(S2): read an all-caps credential after a comma by default (#564) - #577

Merged
derek73 merged 13 commits into
masterfrom
feat/issue-564-comma-caps-credential
Oct 2, 2026
Merged

derek73 merged 13 commits into
masterfrom
feat/issue-564-comma-caps-credential

Conversation

@derek73

@derek73 derek73 commented Oct 2, 2026 •

Copy link
Copy Markdown
Owner

Closes #564.

Policy.unlisted_caps_suffixes becomes a CapsSuffixes StrEnum, OFF / AFTER_COMMA / EVERYWHERE, defaulting to AFTER_COMMA. The field was new in 2.4 and unreleased, so no deprecation is owed. Passing the old True / False raises a TypeError that names CapsSuffixes.EVERYWHERE / CapsSuffixes.OFF.

Why the comma position is safe by default: the all-caps SURNAME convention (Jean DUPONT, DUPONT, Jean) writes the capitals at the end of a name or before a comma, never after a comma behind a full name. The off default was argued for the convention's positions and was switching this one off along with them.

text 2.3.0 default (AFTER_COMMA)
John Smith, XYZ given XYZ, family John Smith suffix XYZ, reported
Ahmad Jayadi, CHA / John Smith, RAI given CHA / RAI suffix, reported
The Rt Hon Kenneth Clarke QC MP, HMG given HMG suffix QC MP, HMG
John Smith, LEED AP given LEED suffix LEED AP (closes the deviates: #291 marker)
John Smith, PhD XYZ given PhD suffix PhD XYZ
García Márquez, MJ given MJ given MJ (unchanged; undotted paired initials)
Smith, XYZ given XYZ given XYZ (unchanged; one name word)
GISCARD d'ESTAING, VALÉRY / LLOYD FitzGERALD, RONALD / LLOYD WEBBER, ANDREW PhD given name kept given name kept (the name carries no contrast)
John Smith XYZ (no comma) family XYZ family XYZ (trailing slot needs EVERYWHERE)

Decisions (Derek's calls marked in decisions.md#S2)

  • API: a StrEnum, not two booleans or a mixed bool | "comma". OFF exists so a capitalized given name behind a two-word surname can still be kept (García Márquez, GABRIEL → suffix GABRIEL by default, the accepted cost).
  • Two letters: a lone two-capital word is never admitted at the comma. Where the name carries the contrast, the comma decides a two-capital word exactly as Dotted initials after a two-word surname read as a credential: García Márquez, G.J. has no given name #563 decides M.J.. Measured: 0 of 1295 swapped inputs differ.
  • Mixed runs: an unlisted all-caps word is a by-shape member of C1's Should a credential run after a comma keep its comma when the last word is an ambiguous acronym? John Smith, PhD MEng reads first PhD, last John Smith #544 run beside listed credentials. Otherwise adding a credential (PhD XYZ, XYZ Jr.) would flip the reading back to a name.
  • The name's contrast: a word of the name's own before the comma that holds a capital and whose last cased letter is lowercase (read after composing the word, passing over non-letters, caseless letters, and a lowercase letter whose capital is not one letter, like ß), not claimed by the vocabulary as a title, particle, connective, credential or generation. Smith, DiCaprio, IJzerman, al-Rashid, d'Estaing, McDonald and Džokić carry it. A surname written in capitals ends in a capital whatever is glued in front of it (d'ESTAING, al-ASSAD, McDONALD, FitzGERALD, DeVITO, St-PIERRE), so such a record keeps its given name, with no prefix list. Derek reached this through istitle(), "contains a capital", "not all capitals but contains a capital" and "capital then lowercase"; each one's leaks are recorded in decisions.md#S2. The accepted costs: an all-lowercase name, an unlisted mixed-case title (Doña), and in an otherwise all-caps record a glued lowercase ending or a mixed-case hyphenated part (JOHN O'NEILL's, JOHN SMITH-Jones, JEAN-pierre DUPONT).
  • Prerequisite: Why does De La Cruz, Ed lose its given name? #575, without which Van Buren, MARTIN and De La Cruz, MARIA would have lost their given names.

Implementation

  • segment: the all-caps run test and the Should a credential run after a comma keep its comma when the last word is an ambiguous acronym? John Smith, PhD MEng reads first PhD, last John Smith #544 run's caps member share one name_contrast(), which reads the own-words walk case_class already does.
  • classify: tags a caps word in the part a suffix comma opened under the default too, so case repair keeps XYZ (rules.md#R4). A forced-repair test guards this, with its negative control run.
  • Two predicates for two questions: in_any_wordlist (the caps shape's "unlisted") and claimed_as_non_name (which words can carry the contrast; a caller's surname list counts as name text).
  • Cost: every caller asks first, in C, what the predicate would decline anyway. Names with no all-caps word after the comma cost exactly what they do on master (Smith, John, Smith, JOHN, John Smith, PhD, John Smith, MA, John Smith, CPA, Ph. D., MD PhD, and the benchmark reference). John Smith, XYZ goes from 251 to 266. An all-caps record with a credential pays a constant +26. Past that, the contrast test is linear in the words before the comma (about four frames a word) and constant in a word's letters.

Docs and ledgers

rules.md S2/C1/C2/R4, decisions.md#S2 (plus a C1 pointer), mechanisms.md, customize.rst, usage.rst, modules.rst (adds CapsSuffixes), the release log, and all five ledgers. A fix(#564) rule classifies the movers. #342's comma pair (RAI/CHA) reads suffix again by its capitals, so that rule's fields lose given and John Smith, RAI's watched shape is re-recorded.

Verification

  • Tests: full suite 10896 passed; mypy, ruff and the docs doctests are clean.
  • Gate: exits 0 at 1.4.0, 2.0.0, 2.1.0, 2.2.0 and 2.3.0. The radar-unclassified counts are what they were before the change (0/5/6/7/7).
  • Equivalence: reviewers measured OFF against master's default over the corpora, every cases.py text and generated grids of up to 388k names, with 0 differences.
  • Review: a docs review and a code review, then five reviews of fix commits. Every round found defects in the previous fix: the two-letter rule text, case repair, frame guards, a series of leaky definitions of the name's contrast (ending in "a capital and a lowercase last letter"), a surname-list interaction, the lone-capital istitle() pitfall, and caseless letters (ʾ, ʼ, 李) stopping the last-letter scan. All are fixed and pinned by case rows. The last two commits (that caseless-letter fix, and making the scan's fall-through reachable after Codecov flagged it as dead code, with the same answers on 1.4M inputs) were reviewed together: no defects; the decisions.md#S2 recipe now states the extraction regex its input count depends on.

🤖 Generated with Claude Code

derek73 and others added 7 commits October 1, 2026 21:42
Policy.unlisted_caps_suffixes becomes a CapsSuffixes StrEnum --
OFF / AFTER_COMMA / EVERYWHERE -- defaulting to AFTER_COMMA. The field
was new in 2.4 and unreleased; a bool now raises a TypeError naming
both replacements.

The all-caps SURNAME convention ('Jean DUPONT', 'DUPONT, Jean') writes
the capitals at the end of a name or before a comma, never after a
comma behind a full name, so the default reads that one position:
'John Smith, XYZ' gives suffix 'XYZ', as do the corpus names 'Ahmad
Jayadi, CHA', 'John Smith, RAI' and 'The Rt Hon Kenneth Clarke QC MP,
HMG', all read as the given name at 2.3.0. 'John Smith, LEED AP' now
reads suffix too, closing the #291 deviation marker C1 carried. A lone
two-letter word is never admitted at the comma (the undotted form of
#563's paired initials): 'García Márquez, MJ' keeps given 'MJ'.
EVERYWHERE adds the trailing slots, the old True; OFF is 2.3's reading
and the way to keep a capitalized given name behind a two-word surname
('García Márquez, GABRIEL', the accepted cost).

The comma test is on by default, so it checks isupper() and the
two-letter length in C before calling the shared predicate: ordinary
comma names pay no frame. rules.md S2/C1/C2, decisions.md#S2,
mechanisms.md, customize/usage/modules docs, the release log, and the
ledgers at all five baselines follow; the gate exits 0 with radar
counts unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Mixed runs (Derek): an unlisted all-caps word is a by-shape member of
  C1's run beside listed credentials, so 'John Smith, PhD XYZ' and
  'John Smith, XYZ Jr.' read as credential runs where adding a listed
  credential had flipped the part back to a name. A two-letter caps word
  in a run is read as #563 reads a pair ('García Márquez, MJ PhD' keeps
  given 'MJ'; 'MJ JK' flips).
- Case repair: classify tags a caps word in the part a suffix comma
  opened under the default too, so capitalized(force=True) keeps 'XYZ'
  in 'John Smith, XYZ' as EVERYWHERE does.
- Frames: the comma test needs two words before the comma first, so
  'Smith, JOHN' pays nothing (it paid +25); classify skips a listed
  member before the caps predicate.
- rules.md#C1 and the release log state the two-letter rule as the
  code does; new examples pin it and the mixed run. Stale "off by
  default" prose fixed in tests, the case table, mechanisms.md (which
  also named a caller that does not exist) and a guard comment;
  decisions.md#C1 points at the S2 entry; the old recipe is translated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- An all-caps word joins C1's run only where the NAME before the comma
  has a lowercase letter: the first fix let a mixed-case credential
  supply the contrast, so 'LLOYD WEBBER, ANDREW PhD' lost its given
  name. 'García Márquez, JUAN Jr.' (a mixed-case name) stays in the
  accepted cost, now named with a case row.
- rules.md#C1 points two capitals at #563's own sentences instead of
  restating them; the restatement contradicted them. Measured by the
  review: MJ and M.J. flip identically across 4036 inputs.
- rules.md#R4 names the caps shape; a forced-repair test guards the
  classify tagging (an R4 example could not: R5 leaves mixed case
  unrepaired), with its negative control run.
- Every caller of the caps predicate asks, in C, what it would decline
  anyway, so a listed credential pays no frame ('John Smith, CPA' and
  'John Smith, Ph. D.' are back to master's counts).
- decisions.md#S2, the release log and mechanisms.md corrected to
  match, including the third-comma-part render and the cost figures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ords

The second round's contrast test read every token before the comma, so
a maiden clause ('LLOYD WEBBER née Smith, ANDREW PhD') or a title ('Mr
LLOYD WEBBER, ANDREW') supplied the lowercase and an all-caps record
lost its given name again; it also cost a frame per character on an
all-caps record. name_contrast now reads the name's own words before
the comma (own_words, shared with case_class's walk rather than a
second spelling of it), titles and particles aside, in one C-level
comparison, and both caps branches ask it.

The all-caps run's later words get the same C-level prechecks as its
first, so a listed credential never reaches the caps predicate.
rules.md#C1 limits "exactly as paired initials" to mixed-case names and
states the contrast as the name's own words; decisions.md#S2 records
both wrong drafts, the 'De La Cruz García, MARÍA' accepted case, and
the cost (all-caps record +24 constant, no longer per character).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…words

The third round excluded titles and particles by list and missed a
generation ('LLOYD WEBBER Jr., ANDREW', the common SURNAME Jr., GIVEN
record), a connective ('GARCÍA y LÓPEZ, ANDREW') and a title recognized
by shape ('Insp. LLOYD WEBBER, ANDREW'): each lost its given name. The
test is inverted now: the contrast comes only from the name's own words
before the comma that no wordlist claims and no period marks, through
_vocab.in_any_wordlist -- the caps predicate's own "unlisted", lifted
out so both share it. Case rows pin the generation, connective and
later-comma-part shapes.

rules.md#C1 states MJ = M.J. as what it is, the comma's decision where
the name carries the contrast; decisions.md#S2 replaces the relabelled
4036 figure with a measurement on this tree (0 of 1295 with the
contrast, 325 of 925 without, by design) and its recompute.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Four review rounds showed that "a lowercase letter somewhere, less some
exclusions" cannot separate an all-caps record from a mixed-case name:
each draft leaked a class the next found, last glued particles and
unlisted lowercase words ("GISCARD d'ESTAING, VALÉRY", 'HAFEZ al-ASSAD,
BASHAR', 'LLOYD McDONALD, RONALD', 'LLOYD ap RHYS, DAFYDD'), all losing
the given name.

Derek's criterion: the name carries the contrast only through one of
its own words before the comma written in Title case -- S2's "written
the way a name is written" -- with no period and not claimed by the
vocabulary as non-name text. A lowercase-only word never counts, so no
list has to know 'ap' or 'thi'. That question gets its own predicate,
_vocab.claimed_as_non_name, which leaves the surname and bound-given
lists out: a draft sharing in_any_wordlist made a caller's surname list
switch the reading off. Accepted with it: an all-lowercase name ('john
smith, XYZ') reads the listing form, as do Mc/Mac-only names; an
unlisted Title-case title ('Doña') still carries the contrast.

rules.md#C1, the Policy docstring, customize.rst, the release log and
decisions.md#S2 say so; case rows pin the glued particle, the unlisted
lowercase word and the lowercase name, and a unit test the surname
list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`istitle()` accepts a single capital, so an undotted initial carried
the name's contrast and a capitalized-surname record with lowercase
particles lost its given name ('de GAULLE C, CHARLES', 'de la O GARCÍA,
MARÍA'). Title case now needs a lowercase letter behind the capital.

decisions.md#S2 records the whole class istitle() rejects when every
name word is of it (Mc/Mac and other interior capitals, the Dutch IJ,
lowercase-led elisions, a vocabulary claim beside one) rather than
Mc/Mac alone; two case notes and the cost comment state the Title-case
criterion; the Policy docstring and customize.rst add the vocabulary
exclusion.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73 derek73 added this to the 2.4 milestone Oct 2, 2026
@derek73 derek73 self-assigned this Oct 2, 2026
@codecov

codecov Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.90%. Comparing base (7394078) to head (71fea58).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #577      +/-   ##
==========================================
+ Coverage   98.89%   98.90%   +0.01%     
==========================================
  Files          45       45              
  Lines        4070     4118      +48     
==========================================
+ Hits         4025     4073      +48     
  Misses         45       45              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

derek73 and others added 6 commits October 2, 2026 00:08
…se (Derek)

Title case (`istitle()`) read too narrowly: interior capitals and
elisions ('DiCaprio', 'IJzerman', 'al-Rashid', "d'Estaing", 'McDonald')
are mixed-case names and kept a given 'XYZ'. Derek's criterion: a word
of the name carries the contrast if it holds a capital directly
followed by a lowercase letter, a leading Mc/Mac skipped where a capital
follows it (_vocab.written_as_a_name). A lowercase prefix glued to a
surname written in capitals ("d'ESTAING", 'al-ASSAD', 'McDONALD') has
no such pair, so those records keep their given names; a lone capital
and a lowercase-only word have none either.

Weighed and declined, with measurements in decisions.md#S2: "contains
a capital" (any lowercase word in an all-caps record then makes every
word a contrast) and "not all capitals but contains a capital" (takes
d'ESTAING and al-ASSAD). The accepted costs shrink to all-lowercase
names and unlisted mixed-case titles. A parametrized test pins the
predicate; case rows pin McDONALD and DiCaprio.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…letter (Derek)

The capital-then-lowercase test leaked a whole class: any capitalized
prefix glued to a surname written in capitals supplies the pair
('LLOYD FitzGERALD, RONALD', 'PAOLO DeVITO, MARCO', 'JEAN LaFLEUR,
PIERRE', 'DICK VanDYKE, JOHN', 'PAUL DuBOIS, JEAN', 'JEAN St-PIERRE,
MARC', 'LLOYD SMITH-McDONALD, RONALD'), as does ß ('MÜLLER WEIß, HANS');
each lost its given name, the Mc/Mac skip being one member of the class.
It also missed titlecase digraphs ('Džokić Ljubić, XYZ') and cost a frame
per letter.

Derek's criterion: a word carries the contrast if it holds a capital
and its last letter is lowercase, a trailing ß aside. A surname written
in capitals ends in one whatever is glued in front, so no prefix list
is needed; 'McDonald', 'Džokić' and decomposed accents pass. Two C-level
checks, so the per-letter cost is gone (a record with lowercase
particles is a constant +22 over master at any length). Measured against
the previous commit: exactly those eight prefixes and Džokić moved, no
corpus or case-table name, OFF still equal to master.

rules.md#C1, the Policy docstring, customize.rst and the release log
drop the false universal; decisions.md#S2 records the chosen option's
own leaks and the accepted hyphenated case; case rows and a broader
parametrized test pin it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The previous commit tested the last character, so a name typed with
decomposed accents ending in an accented letter lost the contrast
('José André, XYZ', 'Lê Thị Hà, XYZ', 'René Noé, XYZ' read given where
their composed spellings read suffix), as did "Jones'", 'Smith2' and a
glued 'Smith)'. written_as_a_name now composes the word (NFC), passes
over trailing non-letters, and passes over a lowercase letter with no
single capital form (ß, ĸ), which generalizes the ß rule. One call, no
generator.

Measured against the previous commit: the four decomposed-accent names
and 'JOHN Smith)' regain the contrast and the ĸ record keeps its given
name; nothing else moves, and OFF equals master on 4292 inputs.

Also from the review: rules.md#C1 states the last-letter reading and
pins 'MÜLLER WEIß, HANS'; decisions.md#S2 records the true blast radius
of the previous commit (seven prefixes plus ß and Džokić, and
'JOHN O'NEILL's' / 'JEAN-pierre DUPONT' / 'MARY-kate OLSEN', now
accepted costs) with a recompute recipe; the cost prose says what it
is, about four frames per word before the comma and constant in a
word's letters; the fix(#564) ledger comments drop the wording
decisions.md calls wrong.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… recipe

The last-letter scan stopped at a caseless letter, so a mixed-case name
whose words end in one lost the contrast: 'Asmāʾ Wafāʾ, XYZ' and
'JOHN Jonesʼ, XYZ' (U+02BC) kept given 'XYZ' where 'JOHN Jones', XYZ'
read suffix. written_as_a_name now passes over every letter without
case, as it passes over non-letters, so the rule is one sentence: the
last cased letter, read after composing the word, setting aside a
lowercase letter whose capital is not a single letter. rules.md#C1
states exactly that.

decisions.md#S2's recipe compared against "the parent commit" while
its numbers were measured against a16927c, and its grid was unnamed.
It now names both comparators by SHA and lists every input, and was run
as written: against a16927c exactly the intended classes move, and OFF
equals master on all 4286 inputs.

Correction to the previous commit's message: "nothing else moves" also
missed 'JOHN SMIfi' (a ligature), which moved there and is unchanged
here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov flagged the trailing `return False` as the PR's one uncovered
line. It was dead: any word failing `text == text.lower()` holds an
uppercase or titlecase letter (checked over every code point, and NFC
keeps one), and the scan returns on reaching it. The capital test now
sits on the scan's return instead, so a word with no letter carrying
case evidence ('李', '2', 'ß', '') falls through and is pinned by new
rows. Same answers as before on 1,421,107 inputs (the corpora and
case-table words, every code point, 300,000 random mixes); no frame
added, the gate unchanged at all five baselines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of d7f4dac/2479eabb: the recipe's 4286 came from a regex
("([^"\n]{2,60})") the prose did not state; reading its words
literally gave 4273 (tokenize) or 4284 (an escape-aware regex), same
conclusions. The prose now names the regex and records the exact-literal
count beside it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73
derek73 merged commit e56dd29 into master Oct 2, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Should John Smith, XYZ read XYZ as a credential by default, even though John Smith XYZ doesn't?

1 participant