Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions docs/customize.rst
Original file line number Diff line number Diff line change
Expand Up @@ -162,25 +162,25 @@ Fixing the case of a particular word

``capitalization_exceptions`` is the one pair-valued field — each entry
maps a lowercase key to its exact-cased replacement (``"phd"`` →
``"PhD"``), so it isn't a fit for ``add()``/``remove()``. Change it with
``"Ph.D."``), so it isn't a fit for ``add()``/``remove()``. Change it with
``dataclasses.replace()`` instead, and pass the result to
``capitalized()``:

.. doctest::

>>> import dataclasses
>>> from nameparser import parse
>>> str(parse("jane smith dds").capitalized())
'Jane Smith Dds'
>>> str(parse("jane smith dphil").capitalized())
'Jane Smith Dphil'
>>> default = Lexicon.default()
>>> lex = dataclasses.replace(
... default,
... capitalization_exceptions=tuple(default.capitalization_exceptions)
... + (("dds", "DDS"),))
>>> str(parse("jane smith dds").capitalized(lex))
'Jane Smith DDS'
... + (("dphil", "DPhil"),))
>>> str(parse("jane smith dphil").capitalized(lex))
'Jane Smith DPhil'

Note the ``tuple(...) + ...``: assigning a bare ``(("dds", "DDS"),)``
Note the ``tuple(...) + ...``: assigning a bare ``(("dphil", "DPhil"),)``
would *replace* the default exceptions rather than extend them, so
``"phd"`` and the rest would stop being fixed.

Expand Down
8 changes: 8 additions & 0 deletions docs/design/decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -579,6 +579,10 @@ Excluded (SUFFIX_ACRONYMS — esq, removed 2026-09-08, #316/#489 bundle):

- esq is OUT of SUFFIX_ACRONYMS and must not be put back by a sweep that finds "Esq." parsing and assumes the acronym set is what carries it. The SUFFIX_WORDS membership carries every single-token spelling — Esq, Esq., ESQ, esq — and stops being inert with the acronym entry gone. What the acronym entry uniquely covered is the multi-dot spelling, so "John Smith E.S.Q." reads family E.S.Q. where every release since 1.4.0 read suffix; that is the whole of the cost and it is a spelling nobody writes. The criterion is #suffix-acronym-collisions', asked of the WORD rather than of the frequency: does the entry describe the word or the machinery. Esquire is a contraction, not an initialism, so the initialism set was never its home; it arrived in the 2019-12-11 bulk Wikipedia post-nominal import (af5bdab, #93) and was never reviewed. This SUPERSEDES the dual-membership entry that stood here from the 2.2 cycle, which called the acronym membership load-bearing and the word membership inert and concluded that the singleton "is why the two sets cannot carry a disjointness assert". The assert now exists — `not (SUFFIX_ACRONYMS & SUFFIX_WORDS)` in suffixes.py's guard block — and it is what the removal buys: the two sets normalize differently (the word test strips edge periods, the acronym test strips all of them), so a word in both is matched by two rules and which one fired is unreadable from outside. AGENTS.md's esq gotcha, the two defending comment blocks in suffixes.py, the `# NOT asserted:` note and the `suffix_acronym_multidot_spelling` case row all retire with it; a row pinning `John Smith E.S.Q.` → family replaces the last of those. Measured over the corpus glob as it stood at the change (1123 distinct names, 1263 rows; the docs commit that follows adds this bundle's rules.md examples and makes it 1136 and 1284): exactly one moves, `John Smith E.S.Q.` itself, and it diffs at every baseline. Classified a behavior change, a 2.x parity break, on all four ledgers.

Excluded (SUFFIX_ACRONYMS — ph, removed 2026-09-13, #459/#521):

- ph is OUT of SUFFIX_ACRONYMS and must not be put back by a sweep that finds "Ph. D." parsing and assumes the acronym set is what carries it. The split spelling is merged by group's own rule (the `PH`/`D` regexes of `_pipeline/_vocab.py`, v1's fix_phd; see #phd-merge, whose "merged back by vocabulary" means that regex stage and not a lexicon lookup), which never consults acronym membership, and the single-token spellings `phd`/`Ph.D.` are the `phd` entry's. The fragment arrived in the 2019-12-11 bulk Wikipedia post-nominal import (af5bdab, #93) and was never reviewed. It did two things. One was to let the merged `Ph. D.` token pass the acronym test on its first piece, which nothing read until #459's all-caps repair did — with `ph` in the set that repair read `john smith ph. d.` as `John Smith PH. D.` on the default path, not only under force, and that is what forced the removal. The other, found by the docs review of the fix-up (2026-09-22) and not at the removal, was to read a bare trailing `Ph.` — no `D.` behind it — as a credential: `John Smith Ph.` gave family Smith, suffix Ph. on every release from 1.4.0 through 2.3.0 (wheels measured), and out of the set it falls to the positional read, middle Smith, family Ph., suffix empty; `Smith, John Ph.` moves the same way, to middle Ph. The cost is that CLASS and not one spelling, measured 2026-09-22 on HEAD and on the five wheels with `Lexicon.default().add(suffix_acronyms={"ph"})` as the comparator: the acronym test strips periods, so the undotted `John Smith Ph`/`PH`/`ph` move identically; inside a credential run `Ph.` no longer joins it, so `John Smith Ph. MD` reads family Ph., suffix MD, and `John Smith MD Ph.` reads middle `Smith MD`, family Ph., the real credential leaving the suffix field with the fragment; `John Smith Ph. Jr.` keeps only the Jr.; and the comma form `Smith, Ph.` reads title Ph. where 2.2.0 and 2.3.0 read suffix. `John Smith P.H.` does not move, the by-shape class reading it either way. Every shape in the class is a bare `ph` with no `D.` behind it, which nobody writes — the same criterion as esq's, asked of the machinery rather than a surname: the entry described the merge, not a word. Classified a behavior change, a 2.x parity break, on all five ledgers (`change(suffix-acronym-collisions) ph leaves the acronym set`), with the cases.py row `removed_ph_fragment_leaves_a_bare_trailing_ph_a_name_word` written to carry the plain shape; the ledger rule is literal to that one name, so a future row on any other shape of the class arrives unexplained at every baseline and is classified by widening that rule, not filed as a regression. No corpus name had any shape of the class, which is why the removal measured as 0 role movers at review time (2026-09-11, 1143 names), the corpus-blindness AGENTS.md warns of. Under `capitalized()` the 13 `PH. D.` shapes the repair had moved under `force=True` go back to `Ph. D.`; `john smith phd` and `ph.d.` still give `Ph.D.`.

### suffix-acronym-collisions — the trailing-position collision class, decided (2026-09-07, #342/#454)

Closes #342 (a wordlist question) and #454 (a rules.md question) together, because they are the same question asked of two different words and answering one without the other would leave the criterion half-stated. No parser code moves in either. Two existing forks carry the whole thing and both were confirmed on the pre-bundle tree with a throwaway override before any wordlist was edited: `Parser(lexicon=Lexicon.default().remove(suffix_acronyms={"rai","cha"}).add(suffix_acronyms_ambiguous={"ba"})).parse(text)`.
Expand Down Expand Up @@ -1325,6 +1329,10 @@ R3's earlier history is under `decisions.md#R2`, which this entry does not repea

- 2026-08-29 — WHY THE BOUNDARY WENT UNNOTICED UNTIL #407, which is where a future reader should look for it. For an ALL-PARTICLE part the other three tag-driven views give the same answer through `replace()` and `revise()` alike: measured over `de la`, `van der`, `do`, `de` and `van de la`, all five agree on `family_particles=''`, on a `family_base` holding the whole part, and on initials from every word. They converge because an UNTAGGED part and a MARKED all-particle part reach the same place by different routes — untagged, no word is recognized as a particle; marked, none is ACTING as one — and all three views only ask which words are particles. Case repair is the one view that asks a second question, since it must also decide whether to lowercase, so it is where the two routes first come apart. The mirror case confirms the reading: on a MIXED part the convergence is the other way round — `de la vega` and `van der berg` diverge in all three views between `replace()` and `revise()` (`replace()` reports particles `''` and base `'de la vega'` where `revise()` reports `'de la'` and `'vega'`) and AGREE on case repair, R4's all-particle clause not reaching them. So before #407 the distinction was invisible on exactly the shape the clause is about, and visible only on shapes the clause does not govern.

- 2026-09-13 #459 (landed 2026-09-22, PR #521) — DECIDED: a credential acronym the exceptions map does not carry is an initialism, so a single-case word the parse put in the SUFFIX role from `suffix_acronyms` repairs to its all-caps spelling rather than a title-cased one (the clause in `_render._cap_word`, after the exceptions-map lookup and before the Mac/Mc rule — the order is load-bearing, since `mcse` is in the acronym set and matches the Mac/Mc shape, and reads `MCSE` only because the acronym clause is asked first; rules.md#R4 states the precedence and pins it with a row). The scope is narrow on purpose. The exceptions map is consulted first, so its five entries — the spellings `str.capitalize()` gets wrong, `md` → M.D. and `phd` → Ph.D. because they are punctuated, and `ii`/`iii`/`iv` → II/III/IV because a numeral is written all-caps and they are `suffix_words` the acronym clause would never see — keep theirs and the clause never touches them. The repair is gated on the SUFFIX role, so a word that is in the acronym vocabulary but parsed as an ordinary name word (`anh van do` → `Anh Van Do`) still repairs as that name word -- the gate is the whole reason the fix is safe on surnames that share a spelling with a credential. Because it is the ROLE and not a parse tag, the gate also reaches a suffix spliced in as raw text: `parse("john smith").replace(suffix="mba").capitalized()` gives `MBA` like the parsed name does, which is the contrast R4's Accepted paragraph draws against the all-particle clause. The same PR removed `ph` from `suffix_acronyms`; see `Excluded (SUFFIX_ACRONYMS — ph ...)`, beside the esq block, for why. The wider design the issue proposed (letter masks, `md` leaving the map, the given-role half of `QC MP`) stays on the rescoped #459; this clause is the narrow part #459 already accepts, not a re-litigation of those.
Reach, population first, measured 2026-09-22 on the PR branch merged with master at 23e52dc6, over the deduped corpus glob (1338 distinct names): 178 names carry acronym vocabulary in a suffix token and 30 of those are written in a single case, and 22 move on the default `capitalized()` path, 119 under `force=True`. Recompute by exec'ing the pre-change `_cap_word` -- `git show 23e52dc6:nameparser/_render.py`, the master this branch merged, which carries #397/#461's generation gate but not this clause -- into `nameparser._render`'s namespace and swapping it in for the changed one in a single process, then diffing `str(capitalized(parse(n), None, force=...))` over the glob. The digits have moved twice and neither move is the clause's: the PR's first head (2026-09-11, `ph` still in the set, 1143 names) measured 5 default and 71 forced; the commit that dropped `ph` took the 13 `PH. D.` forced movers back to `Ph. D.` and left 111 in the population, 5 default and 58 forced; and the merge brought in the corpus rows of PR #530 (#289/#516), PR #532 (#531) and PR #534 (#533), whose single-case trailing `MA` credentials (`DOE, JOHN MA`, `jane doe nee smith ma`, `JOHN SMITH, MA`, the CJK `田中 太郎, MA` family; their mixed-case twins hold still under R5) account for 15 of the 17 further default movers; the other two are `STEVEN HARDMAN, MD, DO, DDS` and `john smith mcse`, the row this PR's own rules.md amendment adds (`McSe` under the pre-change function, which is the precedence claim above measured). The ratio is the stable claim: every single-case name with an acronym suffix the map does not carry moves, and nothing else does.
Accepted costs, deferred to the rescoped #459 rather than relitigated here: the all-caps default reaches words conventionally written mixed-case -- `bsc`/`msc` read `BSC`/`MSC` under `force=True`, and `Dr. med. univ. Margit Popp, MSc` is a corpus name that reads `MSC` under `force=True` only, its mixed case holding it back on the default path under R5 -- which the letter-mask design #459 defers is meant to recover; `ii`/`iii`/`iv` are `suffix_words` rather than acronyms and need the exceptions map precisely because the clause would not see them there; and because the ambiguous five (`ba`, `do`, `ed`, `jd`, `ma`) are in `suffix_acronyms`, the clause moves WHICH parse triggers the repair rather than preventing it -- `john smith ed` → `John Smith ED`, `john smith ba` → `BA`, and `smith, ms.` → `MS.` on the default path. That is the same #342/#454-class cost #459 already accepts, and the alternative (reading classify's `vocab:suffix` tag) was measured and costs `jd` → `Jd`, because `jd` carries `vocab:suffix-ambiguous` and not `vocab:suffix`, so the role gate is the right instrument.

- 2026-09-20 #461/#397 — THE GROUNDING MOVED, AND THEN A DEFECT THE GROUNDING EXPOSED. R4's sentence rested on R3 by name ("the carve-out R3 states for initials"), and R3's carve-out is conditional now while R4's is not, so the cross-reference is CUT and replaced by this rule's own reason: a connective that initials because it joins nothing is still not written the way a name is written. Measured 2026-09-20, plain and forced, over `juan y`, `john and jane smith`, `duke of edinburgh`, `juan de y`, `juan y garcia`, `JUAN Y GARCIA`, `josep carod i rovira`, `JOSEP CAROD I ROVIRA` and — under `add(particles={"y"})` — `Anh y Van`: all nine are byte-identical to the same call at the parent, re-measured after the repair below and still byte-identical, and `tests/test_capitalization.py` is green.
WHAT THE PLAIN CALL DOES, stated over a population rather than over nine names, because "capitalized() does not move" was drafted for this bullet and is false as written. Swept 2026-09-20 over the 1558 non-empty corpus-union-cases names under eight configurations against the parent commit, `capitalized()` moves on SIX distinct names and every one of them is a name whose ROLES moved under the join — it renders different fields rather than treating a word differently. The correct statement is therefore: where only `initials()` moves, `capitalized()` is byte-identical; where the roles move, it follows them.
WHAT THE FORCED CALL DID, AND THE DEFECT THAT WAS FOUND WRITING THIS RECORD. The sweep above was run for the PLAIN claim and turned up a second answer: `capitalized(force=True)` moved on ten further names whose roles, initials and plain repair all stayed put, every one a lower-case `i` — `parse("Carod i")` forced gave "Carod i" where 1.4.0 and 2.3.0 gave "Carod I", and so did "John Quincy Smith i", "Josep Carod i", "Lluis Carod i", "Josep Lluis Carod i III", "Josep Lluis Carod i V", "Carod y de Rovira i", "Rovira, Josep Carod i Jr.", "Carod i Rovira" and "Josep i Rovira". Nine of those were a DEFECT and are REPAIRED here; the remaining two are the rule.
Expand Down
24 changes: 19 additions & 5 deletions docs/design/rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -2173,21 +2173,35 @@ R4. Rationale: case repair is a display concern, applied only on
conventions rather than by the bearer's. A spelling written in a
single case is repaired even where its bearer meant it, because
nothing in the text marks it as a choice; where the text does
mark one, R5 defers to it.
mark one, R5 defers to it. A credential acronym the exceptions map
does not carry is an initialism, so a single-case word the parse
put in the suffix role from the acronym vocabulary repairs to its
all-caps spelling rather than a title-cased one, and that repair
outranks the Mac/Mc convention where a word fits both (MCSE, not
McSe); a word in that vocabulary that parsed as an ordinary name
word repairs as that name word, and a suffix word that is neither
an acronym nor an exceptions-map entry -- the generational `jr`,
`sr` -- keeps its title case.
"juan mcdonald" → capitalized="Juan McDonald"
"Juan McDonald" → capitalized_forced="Juan McDonald"
"ANH DO" → capitalized="Anh Do"
"anh van do" → capitalized="Anh Van Do"
"john smith phd" → capitalized="John Smith Ph.D."
"john smith mba" → capitalized="John Smith MBA"
"john smith mcse" → capitalized="John Smith MCSE"
"john smith jr" → capitalized="John Smith Jr" · boundary
"John Quincy Smith i" → capitalized_forced="John Quincy Smith I"
"Carod i" → capitalized_forced="Carod I"
"Smith, John, and" → capitalized_forced="John Smith and"
"Doe, Jane, and Jr." → capitalized_forced="Jane Doe and Jr."
"juan de la vega" → capitalized="Juan de la Vega" · boundary
Accepted: the clause reaches a part the parser read. A field
spliced in as raw text after the parse carries no reading of its
own, so a family set that way to "de la" stays lowercase where
those same two words parsed from a name are repaired to "De La".
Accepted: the all-particle clause reaches a part the parser read.
A field spliced in as raw text after the parse carries no reading
of its own, so a family set that way to "de la" stays lowercase
where those same two words parsed from a name are repaired to "De
La". The acronym repair is the contrast: it asks the role and the
vocabulary and not a reading, so a suffix spliced in as "mba" is
repaired to "MBA" exactly as a parsed one is.
That is the boundary between splicing text into a field and
revising a field through the parser — revise() classifies the
value, so the repair follows it — rather than a gap between them.
Expand Down
Loading
Loading