diff --git a/AGENTS.md b/AGENTS.md index 55bd1486..95f6b211 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -380,7 +380,7 @@ Add a dedicated `copy.deepcopy()` round-trip test for it too (see `test_regexes_ **`_normalize` must reach a fixed point** — storage and match-time share the one fold, and `Lexicon.__setstate__` re-validates, so a value that changes on re-normalization changes under its owner. `strip().strip(".")` alone is not idempotent (`'. a .'` → `' a '` → `'a'`). The loop is the fix; keep any new stripping inside it. **Anything built on `_normalize` must converge too** — `_fold_words` runs `_normalize` per word and DROPS the words that fold away (`_title_key` is that list space-joined, and `_run_addresses_by_given` reads the list itself, so its last-word arm is the last word of the FOLDED key by construction); keeping the empty slot stored `'lt .'` as `'lt '`, a key match-time can never rebuild (so the entry is silently inert) and `__setstate__` rejects on the next round-trip as "not written by this version". -**Perf regressions are caught by the scaling test, not the absolute-time ones** — `tests/v2/test_benchmark.py::test_parse_cost_grows_no_worse_than_linearly` times a repeated unit at n vs 4n over ten shapes (one per pipeline inner loop) and bounds the ratio; the `_thousand_names` tests use constant-size, delimiter-free input and are structurally blind to a complexity regression. Two rules when touching it: calibrate `_MAX_RATIO` against the WEAKEST quadratic's signal (a mixed quadratic surfaces far below the textbook 16×, so the operating point `_BASE` matters more than the bound), and confirm a planted regression fails it across REPEATED runs — one failure is a coin-flip on a timing test. The ten shapes cover different dimensions (segment count only via `commas`, intra-piece accumulation only via `particles`/`conjunctions`, non-ASCII input only via `honorifics` — the other nine are pure ASCII, so `script_segment` returns at its bail and the CJK stages go unmeasured); measure before pruning one. A stage gated on an opt-in `Policy` field needs a `_POLICY_SHAPES` entry instead, since bare `parse()` never enters it — and that table's rows carry a **reachability probe** run before the measurement, because a precedence change can quietly stop the shape reaching the stage and leave a green test measuring a no-op (`_POLICY_SHAPES` is also asserted non-empty: an empty `parametrize` is a skip, not a failure, so deleting its last row would retire the guard silently). +**Perf regressions are caught by the scaling test, not the absolute-time ones** — `tests/v2/test_benchmark.py::test_parse_cost_grows_no_worse_than_linearly` times a repeated unit at n vs 4n over eleven shapes (one per pipeline inner loop) and bounds the ratio; the `_thousand_names` tests use constant-size, delimiter-free input and are structurally blind to a complexity regression. Two rules when touching it: calibrate `_MAX_RATIO` against the WEAKEST quadratic's signal (a mixed quadratic surfaces far below the textbook 16×, so the operating point `_BASE` matters more than the bound), and confirm a planted regression fails it across REPEATED runs — one failure is a coin-flip on a timing test. The eleven shapes cover different dimensions (segment count only via `commas`, intra-piece accumulation only via `particles`/`conjunctions`, non-ASCII input only via `honorifics` — the other ten are pure ASCII, so `script_segment` returns at its bail and the CJK stages go unmeasured); measure before pruning one. **A shape the CLOCK cannot reach needs a FRAME-count guard instead**, which is the second scaling test in that file (`test_a_trailing_credential_run_does_not_cost_exponentially`, #531): where the defect is an exponential rather than a quadratic, the input length that separates the curves on a timing test does not finish, so the guard counts frames over 8 units against 16 and bounds THAT ratio. One pair does not see every curve, and the fix round for #531 measured why: at 2× the input the per-member LINEAR work swamps a quadratic (2.08× for a genuine one against 1.73× clean), so that pair guards the exponential alone and a second, longer pair — 16 against 64, where the same quadratic reads 7.42× against 3.53× clean — is what can see one. Assert them in that order: an exponential never returns from the longer run, so the cheap pair has to have failed first. Frame counts do not move under load, so this shape needs no repeated-run calibration — but it does need the same reachability assertion `_POLICY_SHAPES` rows carry, since the walk under measurement runs only while every unit still reads as a credential. A stage gated on an opt-in `Policy` field needs a `_POLICY_SHAPES` entry instead, since bare `parse()` never enters it — and that table's rows carry a **reachability probe** run before the measurement, because a precedence change can quietly stop the shape reaching the stage and leave a green test measuring a no-op (`_POLICY_SHAPES` is also asserted non-empty: an empty `parametrize` is a skip, not a failure, so deleting its last row would retire the guard silently). **Expected-failure tests use `@pytest.mark.xfail`** — the conftest parametrized fixture breaks `@unittest.expectedFailure`; always use `@pytest.mark.xfail` instead. diff --git a/docs/customize.rst b/docs/customize.rst index 13ce608a..7cef3b90 100644 --- a/docs/customize.rst +++ b/docs/customize.rst @@ -452,7 +452,10 @@ listed below. as a credential where the position allows it: ``"John Smith X.Y.Z."`` gives suffix ``X.Y.Z.`` while ``"Jack X.Y.Z."`` keeps family ``X.Y.Z.``, and either reading - is reported. Case is irrelevant — the periods are the signal. + is reported. The family-comma form is one of those positions + since 2.4: ``"Doe, John X.Y.Z."`` gives suffix ``X.Y.Z.`` + while ``"Doe, X.Y.Z."`` keeps given ``X.Y.Z.``. + Case is irrelevant — the periods are the signal. Whole-token vocabulary still wins (``M.A.``, ``Ph.D.``), and a single trailing period is not this shape (``"John Smith Xyz."`` keeps family ``Xyz.``). Two further @@ -474,7 +477,9 @@ listed below. - Reads an unlisted all-caps word of two or more letters, with no period in it, in a name written in more than one case as a credential where the position allows it: ``"John Smith XYZ"`` - gives suffix ``XYZ``. Defaults to ``False``, and deliberately: + gives suffix ``XYZ``, and since 2.4 so does the family-comma + form ``"Doe, John XYZ"``. Defaults to ``False``, and + deliberately: an all-caps surname is a real writing convention that shape cannot separate from a credential, so ``"Jean Pierre DUPONT"`` gives family ``Pierre``, suffix ``DUPONT`` with this on. Off, diff --git a/docs/design/decisions.md b/docs/design/decisions.md index 1fcbd1e6..d5f2ecad 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -141,6 +141,7 @@ Open: [#380](https://github.com/derek73/python-nameparser/issues/380) covers "Be WHAT REVIEW FOUND, because the corpus could not. Four agents on PR #472 found the site firing outside the order it was reasoned from, in two directions, and the differential corpus reported neither: 52 default-order names (a conjunction stops a particle's forward chain and leaves it standing in a middle — `Maria Luisa y de la Cruz` gave family 'de la Cruz' where master gives 'la Cruz') and 366 FAMILY_FIRST_GIVEN_LAST folds of a particle that does not end the name (`de Anke van y` gave family 'van de Anke' with the given name still behind it). Of 1099 corpus names only 6 hold a conjunction-then-particle adjacency and none in this shape, so the REACH recipe honestly reported 0 default-order movers throughout. The count was right and the population was blind; "none under the default order" was a property of the corpus, not of the code. Both are now `· boundary` examples in rules.md so the doc runner holds them. The repair is one clause — the run must END the name — and it subsumes the order test a first repair added: measured over 542,592 generated parses, an explicit `state.order[0] is Role.FAMILY` never decides anything the ends-the-name test has not already decided, so it is not in the code. What IS kept is `state.order is not None`, which is what keeps this site off the family-comma path (0 co-firings with P6's comma site over ~892,000 parses). A stale index list was repaired with it: O1 and O2 retag between roles without recomputing, and this site was the first consumer of `families` after them (33,840 stale observations instrumented). No shipped input reaches a wrong answer through it — no `_TURKIC`/`_EAST_SLAVIC`-matching word is particle vocabulary — so the fix is unwitnessed by any test and is recorded here instead. +- 2026-09-18 #531 (cross-amendment) — THE PRECEDENCE OVER S2 GAINS ONE EXCEPTION, AND THE RUN WALK GAINS ONE STAND-DOWN CONDITION. An all-caps member of the AMBIGUOUS credential class, in a name written in more than one case, reads as the credential and this attachment stands down: `Doe, John DO` is suffix `DO` where `Doe, John Do` and `SMITH, JOHN DO` still attach. The argument, the accepted cost (`SMITH, JOHN DO` paired with `NASCIMENTO, EDSON ARANTES DO`) and the two carve-outs are in decisions.md#S2's 2026-09-18 `do` bullet; what belongs here is the CONDITION, which is narrowed to the ambiguous tag rather than to the suffix role: a suffix-roled single-token piece carrying that tag is not part of the run, while `vd` and `mc` — unambiguous suffix vocabulary, also particles, also suffix-roled — stay inside it and are unaffected. Keying on the role instead would have stood the attachment down for them too, and keying on nothing would have let it silently override the slot's reading (`Doe, John DO` read family `DO Doe` with the condition absent, verified). The 2026-08-18 entry's "assign reads a trailing `vd`/`mc`/`do` as a post-nominal, and the attachment overrides that after a comma" is unchanged for `vd` and `mc` and now has this one exception for `do`. ### M2 — the maiden-marker rule @@ -217,6 +218,7 @@ Declined: - 2026-07-17 (M12, Derek-approved) — ma and do joined the ambiguous acronym set because both are common surnames; the two-word "Jack Ma" is kept intact by S2's words-to-spare guard, while the periods gate governs the dotted spellings ("M.A." counts unambiguously). Documented side effect: parenthesized bare "(MA)"/"(DO)" no longer escape to suffix as in 1.x. - 2026-09-15 #289 — AMENDS the sentence above for one spelling and leaves it standing for the other. "The two-word 'Jack Ma' is kept intact by S2's words-to-spare guard" is still true of `Jack Ma`, and false of `Jack MA`: an all-caps member of the ambiguous set inside a mixed-case name is now read as a credential with nothing to spare, so `Jack MA` gives given 'Jack', suffix 'MA' — v1's reading was last 'MA', so this is a deliberate parity break and not a restoration. The periods gate is untouched and still governs the dotted spellings. The two ledger comments that quote this sentence carry the same amendment. See #S2. +- 2026-09-18 #531 — AND `do` IS NOW SPELLING-DEPENDENT AFTER A FAMILY COMMA, which neither sentence above anticipated: it is the one member of this set that is also particle vocabulary, so `Doe, John DO` reads suffix `DO` (capitals, mixed-case name) while every other spelling attaches to the family under rules.md#P6. The pairing and its accepted cost are in #S2's 2026-09-18 `do` bullet. ### vocabulary-collisions — when a word earns the ambiguous marking @@ -598,6 +600,8 @@ for n in ('Smith, John','Smith, XYZ'): print(n, calls_for(off.parse, n), calls_f - 2026-09-18 — THE PREDICTION THAT DID NOT SURVIVE ITS OWN MEASUREMENT, kept because the failure mode is this log's recurring one. A review round called THREE `one_case` plumbing sites in `_group.py` inert and proposed removing them. Measured with a runtime wrapper over 1,642 names under six policies — the default, both family-first orders, strict commas, and each 2.4 switch flipped — 9,852 parses: the maiden walk's PAIR is inert, 0 parses move, and it is inert BY CONSTRUCTION rather than by corpus luck (`numeral_only` answers off `peeled.numeral`, and the numeral fork is decided before the peel reads a lean). The chain-tail measure is not: dropping it there moves 18, on `John van der Berg Ma`, `John de Ma` and `Freiherr von Berg MA` under every one of the six. Both figures are at the call sites. The population is the DISTINCT union, in file order, of every `tools/differential/corpus*.jsonl` entry, every `tests/v2/cases.py` text, and `tests/test_variations.TEST_NAMES` with the three comma permutations `test_variations_of_TEST_NAMES` builds (no-comma, family-comma, and suffix-comma where the name has a suffix), empty strings dropped; the wrapper patches the call sites in a copy of the module source and rebinds `_pipeline.STAGES`, which binds stage functions at import and would otherwise leave the mutation inert. This entry first carried 2,338 names and 14,028 parses, which that recipe does not reproduce by any reading of it — the distinct union is 1,642, summing the three sources' distinct counts gives 2,202, and not deduping at all gives 2,690 — so both figures were re-measured on this tree in the verification round of 2026-09-18 and restated here and at the call sites. The FINDING was unharmed: 0 and 18 reproduce exactly, on the same three names, under all six policies, and the control that drops all four sites at once moves those same 18, which is what shows the wrapper can report a difference at all. The argument was right for two sites out of three and would have taken a third with it. - 2026-09-18 — THE COMMA HARNESS AND THE FRAME BAND, AFTER THE ROUND. Unchanged at the default: 412/449 on the reference name (`uv run python tools/perf/call_count.py`), and the module-`parse` totals above hold exactly — `John Smith` 176, `Smith, John` 207, `John Smith, Jr.` 233, `Smith, John, Jr.` 257, `Smith, MA` 225. WITH SWITCH B ON the two figures recorded above each drop by one: `Smith, John` 213 → 212 and `Smith, XYZ` 245 → 244, `Parser().parse` on both sides as before. The cause is the dead caps branch leaving `ambiguous_class_candidate`: `segment`'s single-token call used to reach a `caps_shape_candidate` call that could only ever answer False, and now does not. Recompute with the recipe above. - 2026-09-18, VERIFICATION ROUND — THE REPORT INVENTORY AN UPGRADER SEES, 2.3.0 → TREE, AND ITS RECIPE. docs/release_log.rst's comma-report bullet claimed TWO reports go away and named `Steven Hardman, MD, DO, DDS` and `John Smith, MD, R.A.I.`. The second is false and was never measured against the wheel: `John Smith, MD, R.A.I.` reports NOTHING at 2.3.0, having carried a `comma-structure` flag only inside this arc's own development, where rules.md#S3's narrow retirement raised it and the C2 amendment quieted it again. Over the whole corpus exactly ONE report is lost, and one more is SWAPPED. Measured: LOST — `comma-structure` ×1 (`Steven Hardman, MD, DO, DDS`), `given-or-family` ×2 (`Jack X.Y.I.`, and `e and e`, which is PR #527's conjunction change and not this arc's). GAINED — `suffix-or-name` ×37, `conjunction-or-initial` ×9 (#527's again), `given-or-family` ×2 (`Jack MA`, `Jack MA.`). So `Jack X.Y.I.` is the only name whose report CHANGED KIND on this arc's account, `given-or-family` → `suffix-or-name`, and the bullet now names it as the swap rather than counting it among the losses. Recompute: collect the corpus glob through compare.py's `_load_entries` deduped by name, parse each name on the tree and on the 2.3.0 wheel (a PEP 723 `uv run --no-project` script pinning `nameparser==2.3.0`, per AGENTS.md's warning that a same-tree comparison reports parity), and take the per-name multiset difference of the `ambiguities` kind values in both directions. The kind counts are multisets, not name counts — `e and e` contributes two `conjunction-or-initial` gains on its own — which is why a bullet quantified from the gate's per-rule name lists would not answer this question. +- 2026-09-18 (Derek), #531 — CAPITALS DECIDE FOR `do`, AND THE PARTICLE RULE KEEPS EVERY OTHER SPELLING. `do` is the one member of the ambiguous credential class (`ba`, `do`, `ed`, `jd`, `ma` — recompute with `sorted(Lexicon.default().suffix_acronyms_ambiguous)`) that is also particle vocabulary, so the trailing slot of the given part and P6's attachment want the same word. Where the writing carries a positive credential lean — an all-caps member in a name written in more than one case — the credential reading wins and P6 stands down: `Doe, John DO` reads suffix `DO` and reports `suffix-or-name`. Every other spelling attaches exactly as it did before, with P6's own `particle-or-given` and no second report: `Doe, John do`, `Doe, John Do`, `DOE, JOHN DO`, `doe, john do` all read family `do Doe` in their own case. THE PAIRING IS THE ARGUMENT, and the accepted cost and the reason are the two halves of it. In ONE CASE the rule cannot tell `NASCIMENTO, EDSON ARANTES DO` from `SMITH, JOHN DO` and reads both as the particle — right about the Brazilian record, wrong about the osteopath, whose `DO` 1.4.0 read as a suffix; both report `particle-or-given`, which is how a caller finds the second. In MIXED CASE it can, and does: `Nascimento, Edson Arantes do` keeps family `do Nascimento` while `Doe, John DO` gives suffix `DO`. Accepted rather than repaired: the one-case reading is the commoner of the two collisions, and the Nascimento record is the name a wrong answer would damage. The other four members are not particles and are unaffected. This is the exception rules.md#P6's "this attachment outranks the suffix reading (S2)" clause now names. Implementing it takes TWO carve-outs rather than one, which is worth recording because the second is in the REPORT and not the reading: the reading carve-out takes a particle-tagged member only on a POSITIVE credential lean, since `Doe, John do` leans nothing and the positional reading would otherwise take it — the wrong ANSWER, not merely a stray report; and the emitter beside it skips a particle-tagged member it did not itself take, since an emitter gated on tag membership alone reported both kinds on every `do` row. Where P6 owns the fork, P6 reports it, which is mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE read strictly. ONE SHAPE THE PAIRING DOES NOT REACH — a shape, not a spelling, and the difference is the review finding: ANY particle or particle chain standing in front of the caps `DO` takes it out of this slot, because grouping's prefix chain merges the two into ONE particle piece before the slot exists and there is then no lone member for the capitals to lean on. It is a fact about this WORD being particle vocabulary itself, not about particles generally — the caps `MA` in `Doe, John van MA` is not swallowed, reads as the credential, and leaves `van` to attach behind it. `Doe, John van DO` reads family `van DO Doe`, and so do the `de`, `von`, `del`, `la` and `van der` spellings of it (measured 2026-09-19 — parse them and read `family`). Pre-existing and unmoved by #531; `Doe, John van DO` is the case row and rules.md#P6's example line, and P6's statement carries the carve-out. The mirror shape is silent rather than merely unreached and is recorded on `AmbiguityKind.SUFFIX_OR_NAME`'s fourth boundary: a particle BEHIND the member ends the given part where this walk reads it, so `Doe, John MA do` keeps middle `MA` beside family `do Doe` though the capitals would otherwise have taken it, the attachment that moves `do` running a stage later. +- 2026-09-18 (Derek), #531 — THE COMMA REPORT'S REACH NOW INCLUDES THE GIVEN SEGMENT'S TRAILING SLOT, AND THE OPEN FOLLOW-UP IS CLOSED. The 2026-09-18 bullet above headed THE COMMA REPORT'S REACH IS THE FIRST POST-COMMA PIECE recorded this slot as an open maintainer decision and named the two questions it turned on — whether a middle initial's neighbourhood should start reporting (a noise judgement, rules.md#A1) and whether the silence was a 1.4 parity gap. Both are answered here, and that bullet stands as it landed. The slot now reads and reports: `Doe, John MA` gives suffix `MA` and `Doe, John Ma` keeps middle `Ma`, each saying which way it went. Derek chose to restore the ROLE and report both ways — one rule for both spellings — over a report-only change and over a capitals-only one, because the issue exists in the first place because two spellings of one name disagree. The noise question was settled by MEASURING THE DISAGREEMENT rather than by argument: over a generated sweep of 78 pairs — five listed members and two by-shape tokens in three cased spellings each, plus five controls in one spelling apiece, so 26 words against three name shapes — 48 pairs disagreed about whether the word was a credential or a name, and every one of the 48 disagreed in the same direction, the comma form declining what the comma-less form took. After this change 3 disagree and all three are the lower-case `do` rows P6 owns. The sweep and its allowlist are `tests/v2/test_properties.py`; a slot that answers differently from the same name written without a comma is not a quiet slot, it is an inconsistent one. Recorded under `3-0-reevaluations`' standing rule because v1 parity is LOAD-BEARING for one half of this and explicitly NOT for the other: `Doe, John MA` reads suffix `MA` on the 1.4.0 wheel, so the bare-acronym half RESTORES v1's role and the report is all that is new there — its 1.4.0 ledger rule neither retired nor narrowed to `_ambiguities` (nothing below baseline 2.0 can diff on that pseudo-field) but was RE-POINTED to the four names where the writing declines the credential, which keep the 2.0-era middle name against v1. `Doe, John X.Y.Z.` reads middle `X.Y.Z.` at 1.4.0 too, so the dotted half LEAVES v1 and carries a 1.4.0 ledger rule of its own; it moves to match the comma-less `John Doe X.Y.Z.` and rules.md#S3's shape rule, not to restore anything. BLAST RADIUS, stated the way this log's own rule asks: twenty corpus names move, and only TWO of them were in any corpus before this branch (`Doe, John MA` and `Doe, John X.Y.Z.`, both admitted by #530's own arc) — the other eighteen are this change's own case rows, so what the differential measures on pre-existing data is two names, and the population the rule reaches is a SHAPE (every family-comma listing whose given part ends in a member of this class) that the corpora barely sample. ACCEPTED COSTS, all measured on the differential corpora: `SMITH, JOHN DO` keeps family `DO SMITH` where 1.4.0 read suffix `DO`, paired with the Nascimento record in the bullet above; one-case and caseless names take the credential with no case evidence at all, so `DOE, MARY JO MA`, `doe, john ma`, `田中, 太郎 MA` and `김, 민준 MA` all read a suffix, which is what 1.4.0 read for each of the four — "caseless is inert" holds for the LEAN and not for the outcome, since `is_one_case` answers True for a script with no case and the positional reading then decides; `Doe, John van MA` loses its middle to the family, reading family `van Doe`, suffix `MA` with two reports where it read middle `van MA` in silence, which is `Berg, Jan van Jr.`'s reading arriving through a shape it could not reach before (Derek accepted it 2026-09-18 as P6 working correctly rather than as a cascade to carve out); `Doe, John Prof. MA` gains a TITLE role `Prof.` never had, H5's transparency reaching it once `MA` leaves the walk, landing it on the same answer as `Doe, John MA Prof.`; `Smith, LEED AP` moves under the default-off caps switch alone, to given `LEED`, family `Smith`, suffix `AP` with two reports, so no default reading is at stake; and a genuine middle name that is also a class member is now a credential wherever it ends the given part and is not Title-cased — `DOE, JOHN ED` reads suffix `ED` — which is the same cost the comma-less form has carried since 2.0. NOT REPAIRED HERE and left open: the maiden walk claims `Smith MA` whole in `Doe, Jane nee Smith MA` before this slot exists, so the rule cannot reach it and the name stays silent with maiden `Smith MA` — the same gap #530's close-out saw from the other side with `John Smith nee Jones R.A.I.`, and it is out of scope for #531. ### indic-honorifics — the renunciate class and the Indic honorific vocabulary (2026-09-06, #346/#344/#343) diff --git a/docs/design/mechanisms.md b/docs/design/mechanisms.md index d8474950..67db3320 100644 --- a/docs/design/mechanisms.md +++ b/docs/design/mechanisms.md @@ -197,4 +197,5 @@ Problem shape. A test pins an ordering, a sort, a dedup or a partition, and its - A detector that re-implements a rule's grouping will get the grouping wrong. Derive the boundary from the same vocabulary the rule reads, not from the half you happen to be thinking about: walking a particle run over the NEVER-GIVEN set alone (the rule chains through ANY particle) split "de la Vega" after "de la" and reported 50 false movers for #364, where the true count is one. Both wrong answers were plausible and printed cleanly. - Guard the whole family, parametrize over it: a defect on one of N parallel entry points hides behind a per-example test — three times in one session (a guard on one class of two, a decode hint on 3 of 5 entry points, a sync roster missing 4 copies) — and a {class}×{field}×{bad-value} parametrization is what caught each. - A growth guard needs calibration, not just existence: benchmark guards that compare n vs 4n catch the quadratic the absolute-time tests are blind to, but calibrate against the WEAKEST signal you must detect and confirm a planted regression fails across repeated runs, not once — a stochastic check "verified" on one sample verifies nothing. +- A frame BUDGET pinned on one reference name cannot see algorithmic blowup; a frame RATIO over two input sizes can. #531's first implementation had a trailing walk re-enter the predicate that owns it, so a run of k ambiguous credentials cost 2**k — `'Doe, John ' + 'MA '*24` took about 6s, wall clock, where the parent commit took 0.24ms (two runs of that one measurement, 5.8s and 5.9s, which is the spread the guard below quotes them at) — while every absolute figure beside the change stayed correct and unmoved: `tools/perf/call_count.py`'s 412/449 on its reference name, and +0 frames on a comma name carrying no class member. The budget is a claim about ONE input and the defect is a claim about a family of them, so no budget is the instrument. The guard that catches it counts frames at 8 members against 16 and bounds THAT ratio (`tests/v2/test_benchmark.py::test_a_trailing_credential_run_does_not_cost_exponentially`): 2,347 → 394,671 frames unfixed, 168x, against 865 → 1,497 and 1.73x fixed, with the bound at 6x. ONE pair does not see every curve, though, and the fix round measured how badly: at 2× the input the per-member linear work swamps a quadratic, so the per-member memo the first fix shipped — a genuine quadratic — reads 2.08x at that pair, inside any bound this one can carry. Separating those needs 4× the input, where the same memo reads 7.42x against the linear 3.53x, so the guard carries a second pair (16 against 64) and asserts the cheap one FIRST: an exponential does not return from a run of 64. Frames rather than a clock for two reasons — the recursion IS frame entries, so the count is the defect rather than a proxy, and the input length that separates an exponential from a linear on a timing test does not finish. It needs the reachability assertion a `_POLICY_SHAPES` row carries, for the same reason: the walk under measurement runs only while every unit still reads as a credential, and a precedence change that routes them elsewhere leaves a green test measuring a walk that no longer happens. - Mind the optional-extra environment split: a local venv's incidental namedivider makes `if available` branches run PRESENT locally and ABSENT in CI, so a locally-green suite proves nothing about the no-extra path (this broke #337's first landing). Run the decisive check in both states or gate the example. diff --git a/docs/design/rules.md b/docs/design/rules.md index 21cf0010..c9f7c7d8 100644 --- a/docs/design/rules.md +++ b/docs/design/rules.md @@ -752,7 +752,18 @@ P6. Rationale: a particle ending the name has nothing to link the word is BOTH a particle and suffix vocabulary, this attachment outranks the suffix reading (S2): a trailing abbreviation after a family comma is the tussenvoegsel far more - often than the decoration it collides with. + often than the decoration it collides with. One exception, and it + is where the capitals speak: a word of the AMBIGUOUS credential + class, written in capitals in a name written in more than one + case, reads as the credential and this attachment stands down — + unless a particle stands immediately in front of it, the two + being one particle run by then, which this rule takes whole. + Every other spelling of such a word attaches as it did before, + and the kind rule below gives it this rule's particle fork rather + than S2's credential one. In a name written wholly in one case + the two readings cannot be told apart and the particle keeps it, + which is right about a Portuguese record and wrong about a + credential; the report is how a caller finds the second. "Jong, Anke de" → family="de Jong" "Beethoven, Ludwig van" → family="van Beethoven" "Berg, Jan vd" → family="vd Berg" @@ -761,6 +772,10 @@ P6. Rationale: a particle ending the name has nothing to link "Beethoven, Ludwig van" → family_base="Beethoven" "Beethoven, Ludwig van" → family_particles="van" "Nguyen, Van" → given="Van" · boundary + "Doe, John DO" → suffix="DO" · boundary + "Doe, John Do" → family="Do Doe" + "SMITH, JOHN DO" → family="DO SMITH" + "Doe, John van DO" → family="van DO Doe" Without a comma, a declared family-first order has named the family in the same way and the attachment fires there too — but only where the run ENDS the name and stands in a MIDDLE — the one @@ -916,9 +931,47 @@ S2. Rationale: generational suffixes and credentials are recognized to spare — as the second of two words it stays the family name — and at the slots that report, either reading carries the ambiguity flag. Those slots are the trailing slot of a name, the - first slot after a family comma, and the segments beyond it; the - trailing slot of the GIVEN part after a family comma is not one - of them, and takes its reading in silence (S3). + first slot after a family comma, the trailing slot of the GIVEN + part after that comma, and the segments beyond it. + After a family comma, a word of this class ending the GIVEN part + is read as the comma-less spelling reads a word ending the name, + and the count is not what decides it there. The comma has already + named the family and the first name word after it is the given + name, so the words to spare are there by construction and the + count says nothing: a word that ENDS that part reads as the + credential unless its WRITING says otherwise, and a word that + does not end it is never asked. "Ending the given part" reaches + past the credentials behind it and past a trailing title, which + is transparent to this reading as it is to the rest of S2's (H5); + a name word behind the word ends the reach, and so does a + PARTICLE the suffix vocabulary does not also claim: it belongs to + the family the comma already named and is taken there by a rule + that runs after this reading is made (P6), so `Doe, John MA do` + keeps its middle name though the capitals would otherwise have + taken the word. A particle the suffix vocabulary DOES claim is + looked past like any other credential, so `Doe, John MA vd` + reads suffix `MA`, with `vd` attaching behind it. Where the + reach ends, the word is an ordinary middle name, read in silence. + A particle in FRONT of the word takes it out of this slot as + well, and takes the word behind it too: where what follows the + particle reads as a name rather than a credential the two are + one name, so neither word is asked and neither reports — + `Doe, John van Ma` reads middle `van Ma` and `Doe, John DO Ed` + middle `DO Ed`, where `Doe, John van MA` reads suffix `MA` and + `Doe, John DO` alone reads the credential, both reporting. + Every word this slot does read reports the fork whichever way it + went, so a run of members all read as credentials reports once + for each, as the same words do without the comma — while a + member the writing keeps as a name + stops the reading there, and whatever stands in front of it is + name text, asked nothing and reporting nothing. One member of + this class is particle vocabulary as well, and where it stands + alone at this slot P6 decides it: the capitals take it as the + credential and every other spelling attaches to the family, + reported there as P6's fork rather than as this one. Behind + another particle it does not stand alone — the two are one + particle run by then — and the run attaches whatever the capitals + say (P6). Written case is the other evidence, and it speaks only in a name written in more than one case: there a member of the ambiguous set written in capitals reads as the credential even with no @@ -926,9 +979,12 @@ S2. Rationale: generational suffixes and credentials are recognized not wholly lower reads as the name even with words to spare. A name written wholly in one case says nothing about any word in it, and the count decides alone; so does a script with no case - to write in. After a family comma this evidence is SECOND: the - count of name words before the comma decides first (C1), and the - case is read only where that count leaves the word a name. + to write in. After a family comma this evidence is SECOND at the + FIRST slot after it: the count of name words before the comma + decides there first (C1), and the case is read only where that + count leaves the word a name. At the trailing slot of the given + part the comma has already settled the count, so the writing is + the only evidence there is. An unlisted word joins this same ambiguous class by SHAPE where the caller asks for it. Two or more period-separated chunks is one such shape, admitted by default (S3); an unlisted all-caps @@ -954,8 +1010,18 @@ S2. Rationale: generational suffixes and credentials are recognized "John Smith Ma" → family="Ma" "Smith, MA" → suffix="MA" "Smith, Ma" → given="Ma" - "Doe, John MA" → middle="MA" · boundary - "Doe, John MA" → ambiguities=() · boundary + "Doe, John MA" → suffix="MA" + "Doe, John Ma" → middle="Ma" · boundary + "Doe, John MA Smith" → middle="MA Smith" · boundary + "Doe, John DO" → suffix="DO" + "SMITH, JOHN DO" → family="DO SMITH" · boundary + "Doe, John MA JD" → ambiguities=("suffix-or-name", "suffix-or-name") + "Doe, John MA Ma" → middle="MA Ma" · boundary + "Doe, John MA Ma" → ambiguities=("suffix-or-name",) · boundary + "Doe, John van Ma" → middle="van Ma" · boundary + "Doe, John van Ma" → ambiguities=() · boundary + "Doe, John DO Ed" → middle="DO Ed" · boundary + "Doe, John DO Ed" → ambiguities=() · boundary "John Smith XYZ" → family="XYZ" "John Smith XYZ" unlisted_caps_suffixes-on → suffix="XYZ" "Jean DUPONT" unlisted_caps_suffixes-on → family="DUPONT" @@ -1006,7 +1072,7 @@ S2. Rationale: generational suffixes and credentials are recognized and unchanged (decisions.md#v1-xfail-triage: `king` stays a title, for the addressing forms). "Dr Jr" → suffix="Jr" - history: decisions.md#S2 · interacts: H1, H2, H3, H5, C1, S3, P2, P5 · implemented: nameparser/_pipeline/_classify.py, nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py, nameparser/_pipeline/_vocab.py + history: decisions.md#S2 · interacts: H1, H2, H3, H5, C1, S3, P2, P5, P6 · implemented: nameparser/_pipeline/_classify.py, nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py, nameparser/_pipeline/_vocab.py S3. Rationale: credentials are often written run together with periods; the chunks between the periods are what carry the @@ -1022,11 +1088,10 @@ S3. Rationale: credentials are often written run together with ambiguous acronym is (S2): a credential where the name has words to spare, a name word where it does not, either reading reported at the slots S2 reports at, and the same at a comma — - which means the FIRST piece after a family comma and the part - before a SUFFIX comma. The part before a FAMILY comma never - reports, the comma having already named it the family; neither - does a word trailing the given part after one, where this - reading is taken silently. Case says nothing here — the + which means the FIRST piece after a family comma, the word + trailing the given part after one, and the part before a SUFFIX + comma. The part before a FAMILY comma never reports, the comma + having already named it the family. Case says nothing here — the periods are the evidence — and three shapes are outside it: a single trailing period is not this shape at all, a chunk that is not wholly alphabetic is no acronym letter, and a word carrying @@ -1046,7 +1111,7 @@ S3. Rationale: credentials are often written run together with "Jack X.Y.I." → family="X.Y.I." · boundary "John Smith Xyz." → family="Xyz." · boundary "John Smith 1.4" → family="1.4" · boundary - "Doe, John X.Y.Z." → middle="X.Y.Z." · boundary + "Doe, John X.Y.Z." → suffix="X.Y.Z." Accepted: the initialless-script clause carries no example line of its own. Every input that exercises it composes a script that writes no abbreviations with a period that only a Latin @@ -1308,9 +1373,11 @@ C1. Rationale: a credential run after the comma means the name is in count leaves the word a name — one name word before the comma — is the case read, capitals in a mixed-case name making it the credential there too (S2). A decision either way at this comma - is reported, which is the one place the comma's own decision is - reported; an attachment decided after a family comma (P6) - reports on its own. C2's comma-structure flag reports what the + is reported. It is one of TWO places the comma's own decision is + reported, the other being the word trailing the given part after + it (S2), which is a second decision about a second word and never + the same fork twice; an attachment decided after a family comma + (P6) reports on its own. C2's comma-structure flag reports what the parse could not recognize, not a fork it called. By default a recognized suffix word counts even written like an initial ("V."), while strict mode vetoes diff --git a/docs/release_log.rst b/docs/release_log.rst index c5be2352..cdbb1818 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -16,7 +16,9 @@ Release Log - **New Policy field unlisted_caps_suffixes, off by default: an opt-in reading for an unlisted all-caps credential.** It reaches the core parser only -- ``Parser(policy=Policy(unlisted_caps_suffixes=True))`` -- since the field has no v1 ``Constants`` manager. With it on, ``.parse("John Smith XYZ")`` gives given ``John``, last ``Smith``, suffix ``XYZ``, and ``.parse("John Smith, XYZ")`` gives the same three fields. It is off by default because an all-caps surname is a real writing convention that shape cannot separate from a credential: ``Jean DUPONT``, ``Minjun KIM`` and ``Jean Pierre DUPONT`` are surnames in French and Korean records, and the last of those gives given ``Jean``, last ``Pierre``, suffix ``DUPONT`` with the switch on. Off, nothing changes and nothing is reported -- 1.4.0's reading for that whole class. Neither of the two new fields reaches the v1 ``Constants`` API, as ``lenient_comma_suffixes`` does not: a ``HumanName`` tracks the parser's own DEFAULTS, so the dotted reading above (default on) reaches it while this one (default off) cannot be turned on from there. See the ``S2`` entry of ``docs/design/decisions.md`` (closes #516) - - **The comma's own decision about an ambiguous credential is now reported.** ``parse("Smith, MA").ambiguities`` names ``suffix-or-name``, and so does every other decision at the ambiguous credential class -- before or after a comma, in either direction, with no new ``AmbiguityKind`` (the family-comma attachment fork already reported this way, e.g. ``parse("Berg, Jan vd")``). One report per decision: ``Smith, Ma`` reports that the word was kept as the given name just as ``Smith, MA`` reports that it was taken as a credential. The reading a SURNAME PARTICLE swallows is reported too, which no release before this one did: ``John van der Berg Ma`` gives last ``van der Berg Ma`` and names ``suffix-or-name``, where the chain took a word the credential reading had considered. ONE report goes away, because a comma segment the parser reads as a credential run is no longer called unrecognized: ``Steven Hardman, MD, DO, DDS`` no longer reports ``comma-structure``, on its written case. That is the whole of the losses over the differential corpora -- ``John Smith, MD, R.A.I.`` is quieted on its shape by the same change, but it never reported at 2.3.0 either, having only carried the flag inside this release's own development. The other movement an upgrader sees is a SWAP rather than a loss: ``Jack X.Y.I.`` reported ``given-or-family`` at 2.3.0 and reports ``suffix-or-name`` here, the dotted retirement above having handed it to the ambiguous class. Everything else at this class is a GAIN, which is what the rest of this bullet describes. Where a report is NOT emitted is a boundary rather than an oversight: a credential trailing the GIVEN part of a family-comma listing is read as a middle name silently, as every 2.x release has read it -- ``Doe, John MA`` gives middle ``MA`` and reports nothing. 1.4.0 read that one as a suffix, so the silence is a 2.0-era reading this release leaves where it found it, recorded open in the ``S2`` entry; the dotted sibling ``Doe, John X.Y.Z.`` is a middle name at 1.4.0 too. See the ``S2`` and ``C1`` entries of ``docs/design/decisions.md`` + - **The comma's own decision about an ambiguous credential is now reported.** ``parse("Smith, MA").ambiguities`` names ``suffix-or-name``, and so does every other decision at the ambiguous credential class -- before or after a comma, in either direction, with no new ``AmbiguityKind`` (the family-comma attachment fork already reported this way, e.g. ``parse("Berg, Jan vd")``). One report per decision: ``Smith, Ma`` reports that the word was kept as the given name just as ``Smith, MA`` reports that it was taken as a credential. The reading a SURNAME PARTICLE swallows is reported too, which no release before this one did: ``John van der Berg Ma`` gives last ``van der Berg Ma`` and names ``suffix-or-name``, where the chain took a word the credential reading had considered. ONE report goes away, because a comma segment the parser reads as a credential run is no longer called unrecognized: ``Steven Hardman, MD, DO, DDS`` no longer reports ``comma-structure``, on its written case. That is the whole of the losses over the differential corpora -- ``John Smith, MD, R.A.I.`` is quieted on its shape by the same change, but it never reported at 2.3.0 either, having only carried the flag inside this release's own development. The other movement an upgrader sees is a SWAP rather than a loss: ``Jack X.Y.I.`` reported ``given-or-family`` at 2.3.0 and reports ``suffix-or-name`` here, the dotted retirement above having handed it to the ambiguous class. Everything else at this class is a GAIN, which is what the rest of this bullet describes. The one slot this bullet left silent no longer is: the bullet below closes it, and a credential trailing the GIVEN part of a family-comma listing now reads as a credential and reports either way. See the ``S2`` and ``C1`` entries of ``docs/design/decisions.md`` + + - **Fix a credential ending the given part of a family-comma listing being read as a middle name in silence.** ``HumanName("Doe, John MA")`` gives first ``John``, last ``Doe``, suffix ``MA``, where 2.0 through 2.3 gave middle ``MA`` -- and 1.4.0 gave the suffix, so this restores v1's reading for that half. The comma has already named the family and the first word after it is the given name, so the words-to-spare count that governs the comma-less form is satisfied by construction and the writing decides alone: ``Doe, John Ma`` keeps middle ``Ma``, written the way a name is written, and ``Doe, John Ed`` keeps middle ``Ed``. Either reading is now reported, and the report belongs to the SPELLING rather than to the fields -- a declined name re-rendered without its comma, ``John Ma Doe``, re-parses to those same three fields and reports nothing, the word no longer standing where the question is asked. A name word behind the credential still ends its reach and stays silent -- ``Doe, John MA Smith`` gives middle ``MA Smith`` and reports nothing -- while a credential run or a trailing title is transparent to it: ``Doe, John MA PhD`` gives suffix ``MA PhD`` and ``Doe, John MA Prof.`` gives title ``Prof.`` with suffix ``MA``. Two second-order movements an upgrader may see, both consequences of the word leaving the given part rather than of this rule reaching further: ``Doe, John Prof. MA`` now gives title ``Prof.`` where it gave middle ``Prof. MA``, the trailing-title chain reaching a word the credential used to hide; and ``Doe, John van MA`` gives last ``van Doe`` with suffix ``MA`` where it gave middle ``van MA``, the surname-particle rule reaching a particle the same way. A name written wholly in one case says nothing either way and takes the credential, which is what 1.4.0 read: ``DOE, MARY JO MA``, ``doe, john ma``, ``田中, 太郎 MA`` and ``김, 민준 MA`` all give a suffix. The unlisted dotted spelling moves with them without the parity claim -- ``Doe, John X.Y.Z.`` gives suffix ``X.Y.Z.`` where 1.4.0 and 2.3.0 both gave a middle name -- to match the comma-less ``John Doe X.Y.Z.``. One word is carved out: ``do`` is the only member of this class that is also a surname particle, so capitals decide it and the particle reading keeps every other spelling. ``Doe, John DO`` gives suffix ``DO``, while ``Doe, John do``, ``Doe, John Do``, ``DOE, JOHN DO`` and ``doe, john do`` are unchanged and keep the particle-or-given report they already had. In a name written wholly in one case the two cannot be told apart, so ``SMITH, JOHN DO`` keeps last ``DO SMITH`` as ``NASCIMENTO, EDSON ARANTES DO`` does -- right about the Portuguese record, wrong about the osteopath, and the report is how a caller finds the second. See the ``S2`` and ``P6`` entries of ``docs/design/decisions.md`` (closes #531) **Additions** diff --git a/nameparser/_pipeline/_assign.py b/nameparser/_pipeline/_assign.py index 96160213..4320b6c1 100644 --- a/nameparser/_pipeline/_assign.py +++ b/nameparser/_pipeline/_assign.py @@ -45,12 +45,14 @@ particles_ambiguous token with more pieces following ("Van Johnson", and since #367 "Dr. Van Johnson" too, a title no longer displacing the particle out of that position) -- whatever role name_order assigns. -Emits SUFFIX_OR_NAME at FOUR sites: the trailing roman numeral, each +Emits SUFFIX_OR_NAME at FIVE sites: the trailing roman numeral, each ambiguous acronym the trailing peel had to resolve, the bare-suffix carve-out where an input that is nothing but post-nominal vocabulary -gets its first word made into the name (H4's suffix half, #491), and +gets its first word made into the name (H4's suffix half, #491), -- since #289 -- the FAMILY-COMMA path's own read of the first -post-comma piece. Further emitters of the same kind live in +post-comma piece, and -- since #531 -- the class member ENDING that +path's given part, which the first-piece emitter could never reach. +Further emitters of the same kind live in `_segment.py`, `_group.py` and `_post_rules.py`; they are not assign's and are not counted here. And at the one site that places a LONE name word, GIVEN_OR_FAMILY for the @@ -70,12 +72,12 @@ effective_script, is_suffix_lenient, resolve_script_set, ) from nameparser._pipeline._pieces import ( - is_suffix_piece, leading_titles, peel_walk, segment_suffix_reading, - tail_reading, trailing_titles, + is_suffix_piece, leading_titles, listed_lean, peel_walk, + segment_suffix_reading, tail_reading, trailing_titles, ) from nameparser._pipeline._state import ( - ParseState, PendingAmbiguity, Structure, WorkToken, - _AMBIGUOUS_CREDENTIAL_TAGS, _NEVER_FLIPPED, + AMBIGUOUS_ACRONYM_TAG, ParseState, PendingAmbiguity, Structure, + WorkToken, _AMBIGUOUS_CREDENTIAL_TAGS, _NEVER_FLIPPED, ) from nameparser._policy import Policy, Script from nameparser._types import AmbiguityKind, Role @@ -580,6 +582,16 @@ def assign(state: ParseState) -> ParseState: # 'PhD' is settled vocabulary and carries neither tag, and # even a second CLASS member there would not be read here # (test_assign.py asserts the count). + # + # The #531 emitter at the far end of this branch counts + # differently, and the difference is the slot rather than a + # second policy: it reports once per member of the trailing + # run it reads, so 'Doe, John MA JD' reports TWICE, matching + # the comma-less 'John Smith MA JD'. A member the writing + # keeps as a name stops that run, which is why + # 'Doe, John MA Ma' reports once and for 'Ma' alone -- 'MA' + # then has a name word behind it and is never asked + # (rules.md#S2). if state.pieces[1] and len(state.pieces[1][0]) == 1: i = state.pieces[1][0][0] if not tokens[i].tags.isdisjoint(_AMBIGUOUS_CREDENTIAL_TAGS): @@ -606,6 +618,15 @@ def assign(state: ParseState) -> ParseState: # place. titled_idx: tuple[int, ...] = () walkable: list[int] = [] + #: Where the trailing suffix run starts, for the #531 + #: report below: `trailing_floor`'s answer, read once on + #: the first member the loop meets and -1 until then (no + #: piece index can be negative, and the loop's own floor is + #: n + 1). Once is enough for the same reason one number is + #: enough inside that walk: the question is MONOTONE, the + #: loop below ascends, and a later member can only ask for + #: LESS of the walk than the first one did. + run_floor = -1 def previous_kept(m: int, titled: tuple[int, ...]) -> int: """The piece before `m` that the H5 chain did NOT @@ -633,6 +654,89 @@ def previous_kept(m: int, titled: tuple[int, ...]) -> int: m -= 1 return m + #: State of the #531 walk below, per `titled` value: how + #: far down the trailing suffix run has been walked, and + #: whether that walk has SETTLED (it stopped on a piece + #: that refuses, so no lower piece can end the given part + #: either and the refusal is never re-asked). + #: + #: The key carries `titled` for the same reason the + #: predicate takes it as a parameter: the two passes ask + #: about the same pieces with different ones spliced out. + #: It CANNOT go stale. Everything the recorded verdict + #: rests on is tags and text -- `is_suffix_piece` reads + #: ptags and token tags, `_reads_as_a_trailing_suffix` + #: reads text plus `is_suffix_piece`, `listed_lean` reads + #: tags, text and `state.one_case` -- and none of them + #: reads `.role`, verified by reading all three + #: (2026-09-19). The one thing this segment's code rewrites + #: between the two passes is the role, through `_set_roles`, + #: which is a `dataclasses.replace(role=...)` and leaves + #: text and tags identical. + floors: dict[tuple[int, ...], tuple[int, bool]] = {} + + def trailing_floor(m: int, titled: tuple[int, ...]) -> int: + """Where the trailing suffix run starts, walked as far + down as `m` needs it: `m >= trailing_floor(m, titled)` + is exactly "every kept piece behind `m` reads as a + suffix", which is what ENDING the given part means + (rules.md#S2, #531). + + ONE walk per `titled` value, shared by the predicate + below and by that rule's report at the foot of this + segment, and the reason both are LINEAR in the run's + length. The question is MONOTONE -- a piece ends the + given part whenever the piece behind it does -- so a + single descent answers for every member, each piece + read at most once. Asked member by member instead, the + reading is recursive (a member ends the given part iff + everything kept behind it reads as a suffix, and a + piece behind it is a member asking the same of its own + tail): a walk per member, which cost O(run**2) with the + per-member memo this replaced and 2**run without one. + Measured on `'Doe, John ' + 'MA '*k`, k doubling from + 200: 2.1/4.1/8.3/17.1ms here, against 8.6/31.4/120/463 + with the memo and 1.5/3.2/7.2/17.3 at cc78c960, where + no such walk existed at all (2026-09-19). + + `low` descends only to `m`, so what comes back is a + floor FOR `m` rather than the run's own first piece + whenever the run reaches past it; that is all either + caller asks, and stopping there is what keeps the + member's own frame count where it was. `final` carries + the other half: a piece that refuses settles the floor + for everything in front of it. + + Re-entrant by construction, and it has to be: the + descent asks the predicate below about a piece that is + itself often a member, which asks this back. `floors` + names the piece under test BEFORE that call, so the + re-entrant reading is "this piece ends the given part", + which is what the descent has just established of it. + """ + entry = floors.get(titled) + if entry is None: + # previous_kept() of one past the end, spelled out + # here rather than called: the frame budget again, + # this walk being asked of every family-comma name + # with a member in the given part, and the skip is + # two lines. Keep the two in step. + low = len(pieces) - 1 + while low in titled: + low -= 1 + entry = (low, False) + floors[titled] = entry + low, final = entry + while not final and low > m: + if reads_as_a_suffix(low, titled): + low -= 1 + while low in titled: + low -= 1 + else: + final = True + floors[titled] = (low, final) + return low + def reads_as_a_suffix(m: int, titled: tuple[int, ...]) -> bool: """Does this segment's walk read piece `m` as a suffix? @@ -652,6 +756,69 @@ def reads_as_a_suffix(m: int, titled: tuple[int, ...]) -> bool: """ if is_suffix_piece(pieces[m], ptags[m], tokens): return True + # rules.md#S2, the given part's trailing slot (#531). + # INLINE, and that is the frame budget talking rather + # than taste: the walkable pass asks this closure once + # per piece of every family-comma segment, so a helper + # call would cost a frame on every non-member piece and + # a generator expression would cost its own on 3.11. + # Membership is therefore a bare `in` on tags already + # in hand, after a `len` -- 'Doe, John Q.' and + # 'Smith, John V' measure +0 with this shape and +1 + # with a helper (2026-09-18). + # + # That "once per piece" is the NON-MEMBER cost, and + # only it. A member is asked a second time by + # trailing_floor()'s descent above, and twice is the + # whole of it: the descent takes each piece once and + # the walkable pass asks each piece once, which is what + # makes the run linear. What the slot costs, measured + # against cc78c960: 'Smith, John', 'Doe, John Q.', + # 'Smith, John V', 'Smith, MA' and 'Berg, Jan vd' are + # all +0 frames, while 'Doe, John MA' is +6 and 'Doe, + # John MA PhD' +23 -- the member's descent, its lean, + # and the report below (2026-09-19). + # + # The comma has already named the family and the first + # piece after it is the given name, so the words to + # spare S2's count asks about are there by + # construction and the count says nothing at this + # slot. What is left is the writing, which is the same + # evidence the comma-less spelling of the same name + # reads. #144's two-segment restriction below is NOT + # inherited: it exists because a trailing 'V' before a + # third comma part is likely a middle initial, and a + # class member is not initial-shaped while a + # credential list behind it makes the credential + # reading likelier rather than less. + piece = pieces[m] + if len(piece) == 1: + tok = tokens[piece[0]] + if AMBIGUOUS_ACRONYM_TAG in tok.tags: + # 'ending the given part' reaches past the + # credentials behind it and past a trailing + # title, which trailing_floor() skips the way + # previous_kept() does -- so 'Doe, John MA + # Prof.' and 'Doe, John Prof. MA' land on one + # answer without a second notion of trailing. + # A name word behind the member ends the + # reach, and the member is an ordinary middle + # name read in silence. + if m >= trailing_floor(m, titled): + lean = listed_lean(tok, state.one_case) + # A member that is ALSO particle + # vocabulary reads as the credential only + # on a POSITIVE credential lean: P6's + # attachment outranks this reading in + # every other spelling, and 'Doe, John do' + # leans nothing, so the positional reading + # would take it -- the wrong answer there, + # not merely a stray report + # (decisions.md#S2, 2026-09-18). + if lean == "credential" or ( + lean is None + and "particle" not in tok.tags): + return True prev = previous_kept(m, titled) # trailing piece of a two-part name is unambiguously # positioned: v1 accepts the lenient test there @@ -755,6 +922,75 @@ def reads_as_a_suffix(m: int, titled: tuple[int, ...]) -> bool: if titled_idx else m not in walkable) _set_roles(tokens, pieces[m], Role.SUFFIX if suffix_here else Role.MIDDLE) + # #531's report, and the FIFTH SUFFIX_OR_NAME site in + # this module. The gate reads EITHER tag, as the + # first-post-comma emitter's does: classify writes the + # shape tag whether or not the dotted switch admits + # the token, which is what lets a declined fork be + # reported without being taken. + # + # The report tracks the FORK CONSULTED, not the lean + # (#289's rule), so a member the writing kept as a + # name reports too -- 'Doe, John Ma' stays a middle + # name and says so. It reports only in the TRAILING + # RUN: with a name word behind it no fork was + # consulted, and AGENTS.md's "a kind is worth adding + # only if a reader would hesitate too" is why that + # must stay silent rather than why it happens to (the + # sentence is the 2.0-conventions section's, not + # rules.md#A1's, which this cited until 2026-09-19). + # + # The `do` carve-out is a REPORT carve-out on top of + # the reading one above: a particle-tagged member this + # walk did not itself take is P6's fork, and P6 + # reports it in its own kind, which is + # mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE read + # strictly. Without it 'Doe, John do' reported both + # kinds. + piece = pieces[m] + if (len(piece) == 1 + and not tokens[piece[0]].tags.isdisjoint( + _AMBIGUOUS_CREDENTIAL_TAGS) + and (suffix_here + or "particle" not in tokens[piece[0]].tags)): + # The SAME floor the predicate measures members + # against, so this report has no walk of its own to + # regress: it had one, and on a run the predicate's + # memo had already made quadratic the report was + # CUBIC, because its per-member walk re-scanned + # `walkable` -- a list -- at every step. Measured + # then on `'Doe, ' + 'John '*r + 'MA '*r`, r + # doubling from 100: 5.8/28/175/1229ms, 4.9x then + # 6.2x then 7.0x per doubling and heading for the + # 8x a cubic gives, against 1.8/3.7/8.0ms here + # (2026-09-19). Nothing guarded it: the scan was a + # C-level `in` over a list and emitted no frame, so + # the frame-ratio test in tests/v2/test_benchmark.py + # was structurally blind to it, and the clock-based + # shapes beside it repeat ONE unit where this cost + # needs a name holding two runs. What keeps it gone + # is the structure: one walk, read by both callers, + # so a second would have to be written on purpose. + # + # Read once for the whole loop, `run_floor` being + # where the FIRST member's walk stopped and the + # floor for every later member too: if the walk + # reached that member, nothing behind it refuses + # and no later member can be refused either; if it + # stopped short, it stopped at the LAST piece that + # refuses, which is exactly what a later member's + # own walk would have found (2026-09-19). + if run_floor < 0: + run_floor = trailing_floor(m, titled_idx) + if m >= run_floor: + i2 = piece[0] + ambiguities.append(PendingAmbiguity( + AmbiguityKind.SUFFIX_OR_NAME, + f"{tokens[i2].text!r} ending the given " + f"part is also an ordinary name word; " + f"read as " + f"{'a credential' if suffix_here else 'a name'}", + (i2,))) if reading is not None and sum( 1 for k, piece in enumerate(fam_pieces) if not is_suffix_piece(piece, fam_tags[k], tokens)) > 1: diff --git a/nameparser/_pipeline/_post_rules.py b/nameparser/_pipeline/_post_rules.py index f391ebbf..326681e3 100644 --- a/nameparser/_pipeline/_post_rules.py +++ b/nameparser/_pipeline/_post_rules.py @@ -26,8 +26,8 @@ from nameparser._lexicon import _run_addresses_by_given from nameparser._pipeline._assign import _name_positions from nameparser._pipeline._state import ( - ParseState, PendingAmbiguity, Structure, WorkToken, _NEVER_FLIPPED, - comma_bucket, + AMBIGUOUS_ACRONYM_TAG, ParseState, PendingAmbiguity, Structure, + WorkToken, _NEVER_FLIPPED, comma_bucket, ) from nameparser._pipeline._vocab import delimiter_cores from nameparser._policy import PatronymicRule @@ -702,7 +702,23 @@ def post_rules(state: ParseState) -> ParseState: for i in seg[end - 1])): end -= 1 k = end - while k and all("particle" in tokens[i].tags for i in seg[k - 1]): + # #531: a class member the given-part slot read as a + # credential is NOT part of the run. P6 keys on vocabulary + # rather than role by design, which is what gives its + # attachment precedence over S2 -- so assign's suffix role + # alone does not stand it down, verified by running #531's + # assign half with this condition absent ('Doe, John DO' read + # family 'DO Doe', the suffix role silently overridden). + # Narrowed to AMBIGUOUS_ACRONYM_TAG rather than to the + # suffix role: `vd` and `mc` are unambiguous suffix + # vocabulary, also particles, also suffix-roled, and the tag + # is what keeps them inside the run. + while k and all("particle" in tokens[i].tags + for i in seg[k - 1]) \ + and not (len(seg[k - 1]) == 1 + and tokens[seg[k - 1][0]].role is Role.SUFFIX + and AMBIGUOUS_ACRONYM_TAG + in tokens[seg[k - 1][0]].tags): k -= 1 # GIVEN alone, which is what P6 says ("provided at least one # given word remains"). Not `_NAME_ROLES`: P1's fold runs diff --git a/nameparser/_types.py b/nameparser/_types.py index 07b44d3b..ab23d12b 100644 --- a/nameparser/_types.py +++ b/nameparser/_types.py @@ -417,14 +417,59 @@ class AmbiguityKind(StrEnum): #: has to be one, and only that word reports. #: WHERE it is emitted is narrower than where the doubt exists, #: and this is the boundary rather than an omission to be read - #: past. The emitters cover the trailing slot of a name, the - #: post-comma slot of a family-comma listing, and the extra - #: segments beyond it. They do NOT cover the trailing slot of the - #: GIVEN part after a family comma: "Doe, John MA" reads middle - #: ``MA`` and "Doe, John X.Y.Z." middle ``X.Y.Z.``, both silently, - #: as every 2.x release has; 1.4.0 read "Doe, John MA" as a - #: suffix. The fork is real there and nothing reports it; - #: decisions.md#S2 records it as open. + #: past. The emitters cover the trailing slot of a name, the FIRST + #: PIECE after a family comma -- that piece and no further -- the + #: trailing slot of that listing's GIVEN part, and the extra + #: segments beyond it. + #: Since 2.4 the given part's trailing slot reports whichever way + #: it read the word: "Doe, John MA" reads suffix ``MA`` and says + #: so, "Doe, John Ma" keeps middle ``Ma`` and says so too. Not in + #: EVERY direction, though: where the member is also a particle + #: and the particle rule keeps it, that rule reports at its own + #: site and this kind stays out of the way -- "Doe, John do" gives + #: family ``do Doe`` and one ``PARTICLE_OR_GIVEN``, never two + #: reports of one word. + #: FOUR positions stay silent, and all four are boundaries + #: rather than omissions. A member with a NAME WORD behind it was + #: never a fork -- "Doe, John MA Smith" reads middle ``MA Smith``, + #: the ordinary reading, and nothing consulted the class. The + #: maiden walk claims everything behind its marker before this + #: slot exists, so "Doe, Jane nee Smith MA" gives maiden + #: ``Smith MA`` in silence, exactly as the comma-less "John Smith + #: nee Jones R.A.I." does. And -- pre-existing, and untouched by + #: 2.4 -- a member with no name word IN FRONT of it is not at this + #: slot either, because the slot is the end of a given part and + #: there is none: a title took that position. "Doe, Dr. MA" gives + #: suffix ``MA`` and "Doe, Mr. MA PhD" suffix ``MA PhD``, the + #: credential-run gate reading those segments whole; "Doe, Dr. Ma" + #: reaches the walk instead and makes ``Ma`` the given name + #: itself, which the walk starts above. The first-piece emitter + #: does not cover for it, reading the piece that stands + #: immediately after the comma and nothing behind that piece: + #: "Doe, MA Smith" reports its ``MA``, and "Doe, Dr. MA Smith" -- + #: the same member, one title in front of it -- reads given ``MA`` + #: in silence. And -- pre-existing and untouched by 2.4 as well -- + #: a PARTICLE beside the member can + #: take it out of this slot, from either side. Where a chain has + #: swallowed the member into one piece there is no lone member to + #: ask about, and the member may as well HEAD that piece as trail + #: it: "Doe, John van Ma" reads middle ``van Ma`` in silence, and + #: so does "Doe, John DO Ed", where the member is itself the + #: particle the chain runs on and the name word behind it joins + #: the piece -- though "Doe, John DO" alone reads the credential + #: and reports. Where a particle the suffix vocabulary + #: does not also claim stands BEHIND it, the given part ends at + #: that particle as this walk reads it, and the attachment that + #: moves the particle to the family runs a stage too late to + #: re-open the question -- so "Doe, John MA do" keeps middle + #: ``MA``, capitals and all, beside family ``do Doe``, reporting + #: only the attachment's own ``PARTICLE_OR_GIVEN``. Neither half + #: is a rule about particles as such, and the caps spellings show + #: it: "Doe, John van MA" reads family ``van Doe``, suffix ``MA`` + #: and reports both forks, and "Doe, John MA vd" reads suffix + #: ``MA`` past a ``vd`` the suffix vocabulary claims outright. + #: All four POSITIONS above are silent -- those last two names are + #: the boundary each one stops at, not instances of it. SUFFIX_OR_NAME = "suffix-or-name" #: An input the title peel eats down to one last word which is #: itself title vocabulary still has to name somebody, so that diff --git a/tests/v2/cases.py b/tests/v2/cases.py index 9f064d26..20dd42ed 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -845,7 +845,13 @@ def _check_cjk_shape_purity(self) -> None: "The declined post-nominal is what the report names " "(#405), so the kind is suffix-or-name and not " "particle-or-given -- 'vd' is no given name in either " - "reading"), + "reading. Also the control for #531's P6 condition, " + "which declines a piece that is one token, roled SUFFIX " + "and carries `vocab:suffix-ambiguous`: 'vd' is " + "UNAMBIGUOUS suffix vocabulary, so it carries no such " + "tag and stays inside P6's run -- byte-identical before " + "and after #531. Narrowing that condition to the " + "ambiguous tag is what buys it"), Case("tussenvoegsel_behind_a_post_nominal", "Berg, Jan van Jr.", {"given": "Jan", "family": "van Berg", "suffix": "Jr."}, classification="fix(#379)", @@ -2057,15 +2063,23 @@ def _check_cjk_shape_purity(self) -> None: "narrows what makes the deviation true"), Case("the_caps_comma_multi_word_run_declines_at_one_word", "Smith, LEED AP", - {"given": "LEED", "middle": "AP", "family": "Smith"}, + {"given": "LEED", "family": "Smith", "suffix": "AP"}, policy=Policy(unlisted_caps_suffixes=True), - ambiguities=("suffix-or-name",), - notes="the one-pre-comma-word twin of the row above: the run " - "is still a candidate (so the fork still reports, via " - "assign's family-comma emitter reading the first " - "post-comma piece's tag) but one name word before the " - "comma is never enough to flip the structure, item 5's " - "count doing its job on real data"), + classification="fix(#531)", + ambiguities=("suffix-or-name", "suffix-or-name"), + notes="the one-pre-comma-word twin of the row above, and the " + "only corpus mover #531 found outside the two known " + "names -- only behind a default-off switch, so no " + "default reading is at stake (Derek's Q3, accepted " + "2026-09-18). The STRUCTURE still does not flip: one " + "name word before the comma is never enough, item 5's " + "count doing its job on real data, so 'LEED' stays the " + "given name. What moved is the TRAILING slot behind it " + "-- before #531 'AP' was the middle name and the only " + "report was assign's family-comma emitter reading the " + "first post-comma piece's tag; now the given part's " + "trailing slot takes 'AP' as the credential and reports " + "its own decision. Two slots, two forks, two reports"), # #516 review round: LISTED members must keep #289's lean with # the switch on -- the caps branch must never ride SHAPE_ACRONYM_ # TAG beside a listed member's own membership tag, which is what @@ -2152,11 +2166,18 @@ def _check_cjk_shape_purity(self) -> None: "John Smith, Ed Ma", {"given": "Ed", "middle": "Ma", "family": "John Smith"}, policy=Policy(unlisted_caps_suffixes=True), - ambiguities=("suffix-or-name",), + classification="fix(#531)", + ambiguities=("suffix-or-name", "suffix-or-name"), notes="'Ed' and 'Ma' are both LISTED ambiguous members, " "Title-case (leans NAME, #289) -- the caps run test " "must not admit a run the listed class already reads " - "on its own"), + "on its own. #531 adds the SECOND report and moves no " + "field: this is a family-comma name whose given part " + "now ends in a class member, and 'Ma' is Title-cased, " + "so the slot consults the fork and declines it " + "exactly as 'Doe, John Ma' does. The report tracks " + "the fork CONSULTED (#530), which is why a declined " + "reading still says so"), # #516 review round (second finding): the caps branch read # `one_case_own` -- true only for a token INSIDE the maiden # clause's own-words span -- where it needed the bare NAME-level @@ -2216,15 +2237,24 @@ def _check_cjk_shape_purity(self) -> None: "whole run rather than per token"), Case("caps_run_needs_every_token_not_any", "John Smith, LEED BA", - {"given": "LEED", "middle": "BA", "family": "John Smith"}, + {"given": "LEED", "family": "John Smith", "suffix": "BA"}, policy=Policy(unlisted_caps_suffixes=True), - ambiguities=("suffix-or-name",), + classification="fix(#531)", + ambiguities=("suffix-or-name", "suffix-or-name"), notes="pins `all()` rather than `any()`: 'BA' is a LISTED " "ambiguous acronym, so the caps shape test excludes it " "and the run is no caps run -- the structure stays the " - "listing form. The report is assign's post-comma one, " - "fired on 'LEED' alone, which carries the shape tag " - "from classify whatever segment made of the run"), + "listing form. The first report is assign's post-comma " + "one, fired on 'LEED' alone, which carries the shape " + "tag from classify whatever segment made of the run. " + "#531 moves 'BA' itself: this is a family-comma name " + "whose given part now ends in a class member, and the " + "member is written in CAPITALS in a name written in " + "more than one case, so it leans credential (#289) and " + "reads suffix where it read middle -- its own second " + "report. What #516 pins here is untouched: the caps " + "run test still declines, and the structure is still " + "the listing form"), Case("the_comma_count_counts_names_not_words_behind_a_title", "Mr Smith, Ma", {"given": "Ma", "family": "Mr Smith"}, @@ -2455,6 +2485,689 @@ def _check_cjk_shape_purity(self) -> None: "anyway as the other half of #397's before-picture, " "beside its upper twin above", shape=1), + # ---- #531: the given part's trailing slot ---------------------- + # The slot: after a family comma the comma has already named the + # family and the first word after it is the given name, so a class + # member ENDING the given part has words to spare by construction + # and the count says nothing. The writing decides, exactly as it + # does for the comma-less spelling of the same name. + Case("the_given_parts_trailing_slot_reads_the_credential", + "Doe, John MA", + {"given": "John", "family": "Doe", "suffix": "MA"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="1.4.0 RESTORED: v1 read suffix 'MA' and 2.0 through " + "2.3 read middle 'MA' in silence. The comma fixed the " + "family and the first post-comma word is the given " + "name, so the words-to-spare count is satisfied by " + "construction and the positional reading at this slot " + "IS the credential. Same answer as the comma-less " + "'John Doe MA', which is the whole point " + "(decisions.md#S2)", + shape=2), + Case("the_given_parts_trailing_dotted_slot_reads_the_credential", + "Doe, John X.Y.Z.", + {"given": "John", "family": "Doe", "suffix": "X.Y.Z."}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="the BY-SHAPE half of the same slot, and the half that " + "does NOT restore 1.4.0: v1 read middle 'X.Y.Z.' here " + "and this becomes a suffix. `listed_lean` returns None " + "wherever the shape tag rides, so a by-shape member " + "never leans and falls to the positional reading, which " + "at this slot is the credential -- matching the " + "comma-less 'John Doe X.Y.Z.' and the 2.4 shape rule " + "rather than v1. One name, one new divergence, accepted " + "(decisions.md#S2)", + shape=2), + Case("the_given_parts_trailing_slot_in_all_caps", + "DOE, JOHN MA", + {"given": "JOHN", "family": "DOE", "suffix": "MA"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="one case, so the lean is inert and the positional " + "reading decides alone -- and at this slot it is the " + "credential. 1.4.0 parity", + shape=2), + Case("the_given_parts_trailing_slot_in_all_lower", + "doe, john ma", + {"given": "john", "family": "doe", "suffix": "ma"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="the other one-case spelling, same reason as its " + "all-caps twin: nothing leans, the position decides. " + "1.4.0 parity", + shape=2), + Case("the_trailing_slot_reads_every_listed_member", + "Doe, John BA", + {"given": "John", "family": "Doe", "suffix": "BA"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="'ba' is another listed member (one of the five " + "decisions.md#suffix-acronym-collisions marked " + "ambiguous rather than removing), so the slot is not a " + "rule about 'ma' -- it is the whole class", + shape=2), + Case("the_trailing_slot_leaves_the_middle_initial_alone", + "Doe, John Q. MA", + {"given": "John", "middle": "Q.", "family": "Doe", + "suffix": "MA"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="the member leaves the given part and 'Q.' stays the " + "middle initial it always was -- the slot reaches the " + "trailing word, not the run in front of it. 1.4.0 " + "parity on both fields", + shape=2), + Case("the_trailing_slot_joins_the_credential_run_behind_it", + "Doe, John MA PhD", + {"given": "John", "family": "Doe", "suffix": "MA PhD"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="'ending the given part' REACHES PAST the credentials " + "behind it, so 'MA' joins the run rather than being " + "walled off by it. Rendered with a SPACE and not a " + "comma: R1 derives suffix entries from the commas the " + "WRITER typed (#436/#437), so only 'Doe, John MA, PhD' " + "renders 'MA, PhD'. 1.4.0 wrote a comma in both. ONE " + "report: 'PhD' is settled vocabulary and carries " + "neither class tag", + shape=2), + Case("the_trailing_slot_reaches_past_a_generational_suffix", + "Doe, John MA Jr", + {"given": "John", "family": "Doe", "suffix": "MA Jr"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="the run behind the member does not have to be " + "credentials -- a generational suffix is a suffix " + "piece and the walk reaches past it the same way", + shape=2), + Case("the_trailing_slot_is_a_run_not_a_position", + "Doe, John PhD MA", + {"given": "John", "family": "Doe", "suffix": "PhD MA"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="the member stands LAST here and the settled " + "credential in front of it is what used to strand it: " + "2.0 through 2.3 read middle 'MA' with suffix 'PhD', " + "which is a name word behind a post-nominal. One " + "reading now, and it is 1.4.0's", + shape=2), + Case("the_trailing_slot_survives_a_third_comma_part", + "Doe, John MA, PhD", + {"given": "John", "family": "Doe", "suffix": "MA, PhD"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="#144's two-segment restriction is NOT inherited by " + "this branch. That restriction exists because a " + "trailing 'V' before a third comma part is likely a " + "middle initial; 'MA' is not initial-shaped, and a " + "credential list behind it makes the credential " + "reading MORE likely rather than less. The comma the " + "writer typed is what renders here, which is why this " + "row shows 'MA, PhD' where its spaced twin shows " + "'MA PhD' (#436/#437)"), + Case("two_members_in_the_trailing_run_report_twice", + "Doe, John MA JD", + {"given": "John", "family": "Doe", "suffix": "MA JD"}, + classification="fix(#531)", + ambiguities=("suffix-or-name", "suffix-or-name"), + notes="one decision, one report; two members, two reports -- " + "matching 'John Doe MA JD', which reports twice today. " + "No token is ever reported twice: these are two " + "tokens", + shape=2), + # ---- the declined direction: the reading does NOT move and the + # report is new. Derek's Q4, answered 'report both directions': + # #530's rule is that the report tracks the FORK CONSULTED, not + # the lean. + Case("the_trailing_slot_declines_a_title_cased_member", + "Doe, John Ma", + {"given": "John", "middle": "Ma", "family": "Doe"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="Title case in a mixed-case name is written the way a " + "NAME is written, so the member stays a middle name -- " + "and the fork was consulted, so it reports. The first " + "of the rows where #531 adds a report without moving a " + "field; 'Doe, Mary Jo Ma', 'Doe, John Ed' and 'Doe, " + "John MA Ma' below are the others, each declining on " + "the same signal. It is the same rule the comma-less " + "'John Doe Ma' has followed since #289", + shape=2), + Case("the_trailing_slot_declines_behind_two_given_words", + "Doe, Mary Jo Ma", + {"given": "Mary", "middle": "Jo Ma", "family": "Doe"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="the lean does not care how many given words stand in " + "front -- Title case declines here exactly as it does " + "with one. 1.4.0 read middle 'Jo', suffix 'Ma'; the " + "case signal is what this release chose over that", + shape=2), + Case("the_trailing_slot_declines_a_title_cased_name_word", + "Doe, John Ed", + {"given": "John", "middle": "Ed", "family": "Doe"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="'Ed' is a given name far more often than it is a " + "doctorate, and Title case is how that is written. The " + "report is how a caller finds the other reading", + shape=2), + Case("the_trailing_slot_declines_then_takes_the_next_member", + "Doe, John Ma JD", + {"given": "John", "middle": "Ma", "family": "Doe", + "suffix": "JD"}, + classification="fix(#531)", + ambiguities=("suffix-or-name", "suffix-or-name"), + notes="two members, two decisions, two reports -- and they " + "go opposite ways. 'JD' is all-caps and takes the " + "credential; 'Ma' is Title-cased and stays a name, " + "which also ENDS the run, so nothing in front of it is " + "reached"), + Case("a_declined_member_ends_the_trailing_run", + "Doe, John MA Ma", + {"given": "John", "middle": "MA Ma", "family": "Doe"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="the trailing member declines on its Title case, and " + "the walk stops at the declined pick rather than " + "continuing past it -- so the all-caps 'MA' in front " + "is never asked and never reports. ONE report, for " + "'Ma'. Identical in shape to rules.md#S2's accepted " + "'Jack Wei Ma' clause", + shape=2), + Case("a_taken_member_behind_a_declined_one_still_reports", + "Doe, John Ma MA", + {"given": "John", "middle": "Ma", "family": "Doe", + "suffix": "MA"}, + classification="fix(#531)", + ambiguities=("suffix-or-name", "suffix-or-name"), + notes="the mirror of the row above, and the pair is the " + "argument: order in the string decides which member " + "the run reaches. Here the trailing 'MA' is taken, the " + "run then reaches 'Ma', which declines and ends it"), + Case("a_third_member_behind_the_declined_one_stays_silent", + "Doe, John MA Ma MA", + {"given": "John", "middle": "MA Ma", "family": "Doe", + "suffix": "MA"}, + classification="fix(#531)", + ambiguities=("suffix-or-name", "suffix-or-name"), + notes="the run walked to its floor and then measured against " + "it: the trailing 'MA' is taken, 'Ma' declines on its " + "Title case and STOPS the run there, and the leading " + "'MA' -- behind a piece this walk refused -- is an " + "ordinary middle name, silent. TWO reports, for 'Ma' " + "and the trailing 'MA'; the third member reaches no " + "fork. The row that would catch a floor left over from " + "an earlier member, since the three members ask the " + "same walk three times and only the first two are " + "above the floor. Today middle 'MA Ma MA', silent"), + # ---- H5: a trailing title is transparent to this reading ------ + Case("a_trailing_title_is_transparent_to_the_slot", + "Doe, John MA Prof.", + {"title": "Prof.", "given": "John", "family": "Doe", + "suffix": "MA"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="the walk starts at previous_kept(), which is the H5 " + "chain's own notion of where the name ends -- so " + "'past a trailing title' costs no second definition " + "of 'trailing'. 1.4.0 read middle 'Prof.', suffix " + "'MA'; the title role is 2.x's own H5 and not at " + "stake here"), + Case("a_title_in_front_of_the_member_becomes_a_title", + "Doe, John Prof. MA", + {"title": "Prof.", "given": "John", "family": "Doe", + "suffix": "MA"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="MEASURED, and a SECOND field moves: today this is " + "middle 'Prof. MA'. Once 'MA' leaves `walkable` the " + "H5 trailing-title chain reaches 'Prof.' and takes " + "it. That is H5's stated transparency working, and it " + "lands this row on the same answer as its neighbour " + "above rather than against it"), + Case("a_leading_title_does_not_block_the_slot", + "Doe, Dr. John MA", + {"title": "Dr.", "given": "John", "family": "Doe", + "suffix": "MA"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="the leading title run is peeled before the walk " + "starts, so the slot sees exactly what it sees " + "without it. 1.4.0 parity", + shape=2), + Case("an_initial_given_name_does_not_block_the_slot", + "Doe, J. MA", + {"given": "J.", "family": "Doe", "suffix": "MA"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="the first post-comma piece is the given name " + "whatever its shape, so a one-letter given leaves the " + "member at the trailing slot as any other would. " + "1.4.0 parity", + shape=2), + Case("the_trailing_slot_reads_two_given_words_in_one_case", + "DOE, MARY JO MA", + {"given": "MARY", "middle": "JO", "family": "DOE", + "suffix": "MA"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="an ACCEPTED COST stated plainly: one case carries no " + "contrast, so the positional reading takes the member " + "and a record writing a middle name in capitals loses " + "it. 1.4.0 read exactly this, and the comma-less form " + "has carried the same cost since 2.0", + shape=2), + # ---- the negative controls: the reading does NOT move and + # NOTHING is reported ----------------------------------------- + Case("a_name_word_behind_the_member_ends_the_reach", + "Doe, John MA Smith", + {"given": "John", "middle": "MA Smith", "family": "Doe"}, + classification="fix(comma-family)", + notes="UNCHANGED and SILENT, and the silence is the point: " + "a name word behind the member ends the trailing run, " + "so the member is an ordinary middle name and no fork " + "was consulted. AGENTS.md's 'a kind is worth adding " + "only if a reader would hesitate too' is why this must " + "stay silent rather than why it happens to -- that " + "sentence is the 2.0-conventions section's, and this " + "note cited rules.md#A1 for it until 2026-09-19. NOT " + "1.4.0 " + "parity, and not #531's doing either: v1 read middle " + "'Smith', suffix 'MA', and 2.0.0 already read what " + "this row reads (both measured on the released wheels, " + "2026-09-18). What moved was 2.0 reading the " + "post-comma part's suffix vocabulary by POSITION, so " + "the divergence files under the family-comma group " + "rather than under the case signal -- measured " + "case-independent, 'DOE, JOHN MA SMITH' and " + "'doe, john ma smith' reading the same way", + shape=2), + Case("a_multi_token_piece_never_reaches_the_slot", + "Doe, John MA y", + {"given": "John", "middle": "MA y", "family": "Doe"}, + classification="fix(comma-family)", + notes="the slot reads a PIECE, not a token: 'y' is a " + "conjunction, so grouping joins 'MA y' into one piece " + "and both len() tests -- the branch's and the report " + "gate's -- decline it. The row that PINS them, and the " + "one the particle rows above cannot be: delete the " + "branch's test and the piece reads as a credential, " + "suffix 'MA y'; delete the report gate's and 'MA' is " + "reported where nothing was decided (both measured " + "2026-09-19). 'Doe, John A.B. e' is the by-shape " + "spelling of the same piece and moves with it, so it " + "gets no row of its own. UNCHANGED by #531 -- 2.0.0 " + "and 2.3.0 read this middle too, where 1.4.0 read " + "middle 'y', suffix 'MA' -- and the divergence is " + "'Doe, John MA Smith's, the post-comma part read by " + "POSITION since 2.0", + shape=2), + Case("a_maiden_clause_takes_the_member_with_it", + "Doe, Jane nee Smith MA", + {"given": "Jane", "family": "Doe", "maiden": "Smith MA"}, + classification="fix(#274)", + notes="the silence no walk decides: the maiden marker claims " + "everything behind it, so the member is inside the " + "maiden clause and this slot never exists for it. The " + "comma-less 'John Smith nee Jones R.A.I.' is silent " + "for the same reason. 1.4.0 had no maiden routing and " + "read middle 'nee Smith', suffix 'MA'; 2.0.0 and 2.3.0 " + "read this maiden (measured 2026-09-19)"), + Case("a_leading_title_takes_the_slot_the_member_would_have_had", + "Doe, Dr. MA Smith", + {"title": "Dr.", "given": "MA", "middle": "Smith", + "family": "Doe"}, + notes="beside 'Doe, MA Smith' below, and the pair is what " + "says how far the FIRST-PIECE emitter reaches: it reads " + "the piece standing immediately after the comma, which " + "is the title here, so the member behind it is read as " + "the given name in silence. Not this slot either -- a " + "title took the position the given part would have had. " + "Parity at 1.4.0, 2.0.0 and 2.3.0 alike (measured " + "2026-09-19)", + shape=2), + Case("the_post_comma_emitter_reports_the_same_member_untitled", + "Doe, MA Smith", + {"given": "MA", "middle": "Smith", "family": "Doe"}, + ambiguities=("suffix-or-name",), + notes="the same member, the same reading, one title fewer -- " + "and now the first piece after the comma IS the member, " + "so #289's emitter reports it. The report is the whole " + "difference from the row above: 1.4.0 read first 'MA', " + "middle 'Smith' too, so the fields are parity", + shape=2), + Case("a_title_led_segment_consumes_a_whole_run_in_silence", + "Doe, Mr. MA PhD", + {"title": "Mr.", "family": "Doe", "suffix": "MA PhD"}, + classification="fix(#289)", + notes="'Doe, Dr. MA' with a credential run behind the " + "member, which is what the row adds: the no-name gate " + "reads the segment whole, so the member joins the run " + "rather than taking the given slot, and neither " + "emitter is in that path. 1.4.0, 2.0.0 and 2.3.0 all " + "read first 'MA', suffix 'PhD' -- the case lean moved " + "it, which is 'Doe, Dr. MA's classification " + "(measured 2026-09-19)", + shape=2), + Case("the_initial_veto_before_a_third_comma_part_is_untouched", + "Doe, John V, PhD", + {"given": "John", "middle": "V", "family": "Doe", + "suffix": "PhD"}, + notes="#144's two-segment restriction still governs its own " + "predicate: 'V' is a roman numeral reached by the " + "LENIENT trailing test, not by the ambiguous class, so " + "the new branch never sees it and the middle initial " + "survives. The control that pins the restriction's " + "scope after #531 narrowed nothing about it"), + Case("the_no_name_gate_path_still_reports_exactly_once", + "Doe, MA", + {"family": "Doe", "suffix": "MA"}, + classification="fix(#289)", + ambiguities=("suffix-or-name",), + notes="the reading is the_lean_reaches_the_post_comma_slot's " + "('Smith, MA') word for word, so it carries that row's " + "classification: 1.4.0 AND 2.0.0 both read first 'MA', " + "last 'Doe', and #289's lean is what made the " + "credential (measured on both wheels 2026-09-18). " + "UNCHANGED by #531, and structurally unreachable by the new " + "emitter rather than luckily missed: segment 1 holds " + "no name word, `segment_suffix_reading` returns " + "non-None, and assign sets every role from that " + "reading without entering the placement loop at all. " + "So the slot the old emitter owns and the slot the new " + "one owns cannot both fire -- which is why no token is " + "ever reported twice", + shape=2), + Case("the_no_name_gate_path_with_a_run_still_reports_once", + "Doe, MA PhD", + {"family": "Doe", "suffix": "MA PhD"}, + classification="fix(#289)", + ambiguities=("suffix-or-name",), + notes="the same gate path with a credential run behind the " + "member, and the same classification as its bare twin " + "above: 1.4.0 and 2.0.0 alike read first 'MA', suffix " + "'PhD'. One report, for the one class member, exactly " + "as before #531", + shape=2), + Case("a_title_led_segment_consumes_the_member_in_silence", + "Doe, Dr. MA", + {"title": "Dr.", "family": "Doe", "suffix": "MA"}, + classification="fix(#289)", + notes="the THIRD position this kind stays silent at, and a " + "boundary rather than an omission: a title took the " + "slot the given name would have had, so there is no " + "given part for the member to end. The segment holds " + "no name word, the credential-run gate reads it whole, " + "and #531's emitter is in the placement loop the gate " + "path never enters. The post-comma emitter cannot " + "reach it either -- that one reads the FIRST piece " + "after the comma, which is 'Dr.'. So the reading moves " + "and nothing reports. MEASURED on the wheels: 1.4.0, " + "2.0.0 and 2.3.0 all read first 'MA', last 'Doe'; " + "#289's case lean is what makes it a credential, which " + "is this row's twin 'Doe, MA' above, and 'DOE, DR. MA' " + "keeps given 'MA' where one case carries no lean", + shape=2), + # ---- the `do` pairing: capitals decide, the particle rule keeps + # every other spelling (Derek, 2026-09-18) --------------------- + Case("capitals_decide_for_the_particle_member", + "Doe, John DO", + {"given": "John", "family": "Doe", "suffix": "DO"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="'do' is the one class member that is also particle " + "vocabulary, so this slot and P6's attachment want the " + "same word. A POSITIVE credential lean -- an all-caps " + "member in a name written in more than one case -- is " + "what takes it, and P6 stands down. 1.4.0 read suffix " + "'DO'. Reports ONCE: where this reading wins, P6 has " + "nothing to report", + shape=2), + Case("the_particle_rule_keeps_the_lower_case_member", + "Nascimento, Edson Arantes do", + {"given": "Edson", "middle": "Arantes", + "family": "do Nascimento"}, + classification="fix(#380)", + ambiguities=("particle-or-given",), + notes="UNCHANGED, and the record the pairing is built to " + "protect. Mixed case with an all-lower member leans " + "nothing, so the reading carve-out hands it to P6 and " + "the report carve-out keeps P6's own kind as the only " + "one. This is the half of the pair the capitals buy. " + "NOT 1.4.0 parity: v1 read suffix 'do', and P6's " + "attachment has outranked that since 2.3 -- which is " + "'Berg, Jan vd's classification, this word being the " + "same collision", + shape=2), + Case("one_case_cannot_tell_the_osteopath_from_the_record", + "SMITH, JOHN DO", + {"given": "JOHN", "family": "DO SMITH"}, + classification="fix(#380)", + ambiguities=("particle-or-given",), + notes="ACCEPTED COST, and the pairing IS the argument: in " + "one case the rule cannot tell this from " + "'NASCIMENTO, EDSON ARANTES DO' and reads both as the " + "particle -- right about the Brazilian record, wrong " + "about the osteopath, whose 'DO' 1.4.0 read as a " + "suffix. Both report `particle-or-given`, which is how " + "a caller finds the second. Accepted rather than " + "repaired: the one-case reading is the commoner of the " + "two collisions and the Nascimento record is the name " + "a wrong answer would damage", + shape=2), + Case("a_member_inside_a_particle_run_never_reaches_the_slot", + "Doe, John van DO", + {"given": "John", "family": "van DO Doe"}, + classification="fix(#380)", + ambiguities=("particle-or-given",), + notes="the capitals lean cannot reach this 'DO', and the " + "reason is EARLIER than #531: grouping merges 'van DO' " + "into ONE particle piece before the trailing slot " + "exists. What declines it is the TAG test -- the " + "piece's first token is 'van', which carries no class " + "tag -- and the len() test in front of that decides " + "nothing here, since deleting it leaves this row where " + "it stands ('Doe, John MA y' is the row that pins it). " + "Not a carve-out and not this release's doing -- " + "verified byte-identical at cc78c960 and on the 2.3.0 " + "wheel (2026-09-19), where 'Doe, John DO' moved and " + "this did not. The report is P6's, named for 'van', " + "the word of the run that is also a given name. 1.4.0 " + "read middle 'van DO'; the attachment is #380's, which " + "is the classification 'Nascimento, Edson Arantes do' " + "carries above", + shape=2), + Case("a_member_heading_a_particle_chain_never_reaches_it_either", + "Doe, John DO Ed", + {"given": "John", "middle": "DO Ed", "family": "Doe"}, + classification="fix(comma-family)", + notes="the member HEADS the joined piece here where 'van DO' " + "has it trailing: 'do' is particle vocabulary, so the " + "chain takes the name word behind it and 'DO Ed' is " + "one two-token piece. Both words silent -- though " + "'Doe, John DO' alone reads the credential and reports " + "-- and nothing attaches, the run not being wholly " + "particles. The SECOND killer for the len() tests, and " + "the one where the piece's first token IS the class " + "member: without them the piece reads as a credential, " + "suffix 'DO Ed'. 1.4.0 read suffix 'DO, Ed'; 2.0.0 and " + "2.3.0 read this (measured 2026-09-19)", + shape=2), + Case("a_declining_member_inside_a_particle_run_is_silent_too", + "Doe, John van Ma", + {"given": "John", "middle": "van Ma", "family": "Doe"}, + classification="fix(comma-family)", + notes="'Doe, John van DO's Title-cased twin, and it differs " + "in what P6 does rather than in what the slot does: " + "the joined piece is two tokens either way, but 'Ma' " + "is not particle vocabulary, so the trailing run is " + "not wholly particles and nothing attaches -- middle " + "'van Ma', family 'Doe', silent on both counts. 1.4.0 " + "read middle 'van', suffix 'Ma'; 2.0.0 and 2.3.0 read " + "this", + shape=2), + Case("the_particle_carve_out_silences_the_member_in_front", + "Doe, John MA do", + {"given": "John", "middle": "MA", "family": "do Doe"}, + classification="fix(#380)", + ambiguities=("particle-or-given",), + notes="the carve-out cascades: lower-case 'do' leans " + "nothing, so this walk does not read it as a suffix -- " + "and a piece the walk refuses ENDS the run, so the " + "all-caps 'MA' in front of it is never asked and never " + "reports, capitals and all. P6 then attaches 'do' a " + "stage later, too late to re-open the question. One " + "report, P6's. UNCHANGED by #531: byte-identical at " + "cc78c960 and on the 2.3.0 wheel (2026-09-19)", + shape=2), + Case("one_case_keeps_the_particle_on_the_real_record", + "NASCIMENTO, EDSON ARANTES DO", + {"given": "EDSON", "middle": "ARANTES", + "family": "DO NASCIMENTO"}, + classification="fix(#380)", + ambiguities=("particle-or-given",), + notes="the fourth corner of the pairing, and the reason the " + "one-case cost is accepted: this is a real Portuguese " + "record and the particle reading is right about it. " + "1.4.0 read middle 'ARANTES', suffix 'DO'", + shape=2), + Case("the_particle_member_declines_in_title_case_too", + "Doe, John Do", + {"given": "John", "family": "Do Doe"}, + classification="fix(#380)", + ambiguities=("particle-or-given",), + notes="Title case leans NAME, which is not a positive " + "credential lean, so the carve-out hands it to P6 " + "like every non-capital spelling. One report, P6's. " + "1.4.0 read suffix 'Do'", + shape=2), + Case("p6_still_claims_the_other_unambiguous_suffix_particle", + "Berg, Jan mc", + {"given": "Jan", "family": "mc Berg"}, + classification="fix(#380)", + ambiguities=("suffix-or-name",), + notes="'mc' is the second word in both vocabularies whose " + "suffix half is unambiguous, and it is the control " + "that stops #531's P6 condition being written against " + "one example -- 'Berg, Jan vd' above is the first, and " + "both are byte-identical before and after. Carries " + "that row's classification for the same reason: 1.4.0 " + "and 2.0.0 both read suffix 'mc' and P6's attachment " + "is what took it (measured on both wheels 2026-09-18)", + shape=2), + # ---- the particle cascade Derek accepted (Q1) ---------------- + Case("the_slot_hands_a_trailing_particle_to_p6", + "Doe, John van MA", + {"given": "John", "family": "van Doe", "suffix": "MA"}, + classification="fix(#531)", + ambiguities=("particle-or-given", "suffix-or-name"), + notes="A FAMILY MOVE, found by measurement and accepted by " + "Derek 2026-09-18: today this is middle 'van MA', " + "silent. Once 'MA' becomes a suffix, P6's own walk " + "looks past that post-nominal, finds 'van' as a " + "trailing all-particle run and attaches it. That is " + "exactly 'Berg, Jan van Jr.'s reading, arriving " + "through a shape it could not reach before. TWO " + "reports, one per fork, and neither is this slot " + "reporting twice"), + Case("the_slot_reaches_past_an_unambiguous_suffix_particle", + "Doe, John MA vd", + {"given": "John", "family": "vd Doe", "suffix": "MA"}, + classification="fix(#531)", + ambiguities=("suffix-or-name", "suffix-or-name"), + notes="the row above with the particle BEHIND the member " + "instead of in front, and the pair says the cascade is " + "not about where the particle stands: 'vd' is claimed " + "outright by the suffix vocabulary, so this walk reads " + "past it and takes 'MA', and P6 then attaches 'vd' " + "over that reading -- reporting in the kind naming the " + "reading it OVERRODE, which is why both reports here " + "are `suffix-or-name` where 'Doe, John van MA' gives " + "one of each. Today middle 'MA', family 'vd Doe', one " + "report ('Berg, Jan vd' is that reading with nothing " + "in front of the particle)"), + # ---- policy rows. Every one is _CORE_ONLY (see + # tests/v2/test_facade_cases.py) -------------------------------- + Case("the_dotted_slot_reports_with_the_switch_off", + "Doe, John X.Y.Z.", + {"given": "John", "middle": "X.Y.Z.", "family": "Doe"}, + policy=Policy(unlisted_dotted_suffixes=False), + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="the declined-but-reported shape at this slot: " + "classify writes `shape:acronym` whether or not the " + "switch admits the token, which `peel_trailing` " + "already relies on, so the fork is consulted and " + "reported while the reading stays a middle name. Same " + "treatment 'Smith, A.B.' gets at the first post-comma " + "slot"), + Case("the_caps_switch_reaches_the_trailing_slot", + "Doe, John XYZ", + {"given": "John", "family": "Doe", "suffix": "XYZ"}, + policy=Policy(unlisted_caps_suffixes=True), + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="the by-shape class reaches this slot through the " + "same tag the listed class does. `listed_lean` returns " + "None wherever `shape:acronym` rides, so a by-shape " + "member never has a lean and takes the positional " + "reading -- which at this slot is the credential"), + Case("the_trailing_slot_ignores_the_strict_comma_knob", + "Doe, John MA", + {"given": "John", "family": "Doe", "suffix": "MA"}, + policy=Policy(lenient_comma_suffixes=False), + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="`lenient_comma_suffixes` governs #144's initial-veto " + "predicate, which this branch does not go through -- " + "so the strict knob reads this slot exactly as the " + "default does. The control that says which predicate " + "owns the word"), + Case("the_trailing_slot_reads_the_same_under_family_first", + "Doe, John MA", + {"given": "John", "family": "Doe", "suffix": "MA"}, + policy=Policy(name_order=FAMILY_FIRST), + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="a family comma names the family, so the declared " + "order arbitrates nothing here. Measured identical " + "under all three orders, which is also why " + "`_ORDER_EXEMPTION_EFFECT` gains no row"), + Case("the_trailing_slot_reads_the_same_under_ff_given_last", + "Doe, John MA", + {"given": "John", "family": "Doe", "suffix": "MA"}, + policy=Policy(name_order=FAMILY_FIRST_GIVEN_LAST), + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="the third order, for the same reason as the second"), + # ---- caseless scripts. tolerated=True, NOT a shape tag: + # cases.py's __post_init__ hard-errors on a composed CJK comma + # form carrying a shape. + Case("a_caseless_script_takes_the_trailing_credential", + "田中, 太郎 MA", + {"given": "太郎", "family": "田中", "suffix": "MA"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="'caseless is inert' holds for the LEAN, not for the " + "OUTCOME: `is_one_case` answers True for a caseless " + "script, so nothing leans, so the positional reading " + "applies -- and at this slot the positional reading IS " + "the credential. 1.4.0 parity, measured. A composed " + "CJK comma form, so tolerated rather than contract " + "(rules.md#W3's 2026-09-01 demotion)", + tolerated=True), + Case("a_hangul_family_takes_the_trailing_credential_too", + "김, 민준 MA", + {"given": "민준", "family": "김", "suffix": "MA"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="the Korean twin of the row above, and the pair is " + "what stops the reading being a fact about Han. " + "Tolerated for the same reason", + tolerated=True), Case("family_comma_lenient_trailing", "Smith, John V", {"given": "John", "family": "Smith", "suffix": "V"}, notes="v1 #144: the trailing piece of a two-part comma name " diff --git a/tests/v2/pipeline/test_assign.py b/tests/v2/pipeline/test_assign.py index bb0fa7af..8d89e8ad 100644 --- a/tests/v2/pipeline/test_assign.py +++ b/tests/v2/pipeline/test_assign.py @@ -720,3 +720,115 @@ def test_the_comma_report_says_which_way_it_read_the_word() -> None: assert detail["Smith, A.B."] == [ "'A.B.' after the comma is also an ordinary name word; read as " "the given name"] + + +def test_the_trailing_given_slot_takes_a_bare_class_member() -> None: + out = _assigned("Doe, John MA", lexicon=Lexicon.default()) + assert _by_role(out, Role.SUFFIX) == "MA" + assert _by_role(out, Role.MIDDLE) == "" + + +def test_the_trailing_given_slot_walks_past_a_title() -> None: + """previous_kept() is what makes 'past a trailing title' the same + walk H5 already uses, so the two spellings agree rather than + needing two notions of 'trailing'.""" + for text in ("Doe, John MA Prof.", "Doe, John Prof. MA"): + out = _assigned(text, lexicon=Lexicon.default()) + assert _by_role(out, Role.SUFFIX) == "MA", text + assert _by_role(out, Role.TITLE) == "Prof.", text + + +def test_the_trailing_given_slot_steps_over_a_title_inside_the_run() -> None: + """A title BETWEEN the member and a suffix: the floor's descent + starts on 'Jr', steps over the titled 'Prof.' and lands on the + member. Both spellings above put the title at an end of the run, + where the descent starts past it and never takes that step.""" + out = _assigned("Doe, John MA Prof. Jr", lexicon=Lexicon.default()) + assert _by_role(out, Role.SUFFIX) == "MA Jr" + assert _by_role(out, Role.TITLE) == "Prof." + assert _by_role(out, Role.MIDDLE) == "" + # the declined spelling takes the same step and stops on the member + out = _assigned("Doe, John Ma Prof. Jr", lexicon=Lexicon.default()) + assert _by_role(out, Role.SUFFIX) == "Jr" + assert _by_role(out, Role.MIDDLE) == "Ma" + + +def test_the_trailing_given_slot_reads_the_lean_three_ways() -> None: + """credential / name / no-lean, the three answers listed_lean + gives, each landing where the rule says.""" + assert _by_role(_assigned("Doe, John MA", lexicon=Lexicon.default()), + Role.SUFFIX) == "MA" # lean credential + assert _by_role(_assigned("Doe, John Ma", lexicon=Lexicon.default()), + Role.MIDDLE) == "Ma" # lean name + assert _by_role(_assigned("doe, john ma", lexicon=Lexicon.default()), + Role.SUFFIX) == "ma" # one case, no lean + + +def test_the_trailing_given_slot_falls_through_for_a_by_shape_member( +) -> None: + """listed_lean returns None wherever the shape tag rides, so a + by-shape member never leans and takes the positional reading -- + which at this slot is the credential.""" + out = _assigned("Doe, John X.Y.Z.", lexicon=Lexicon.default()) + assert _by_role(out, Role.SUFFIX) == "X.Y.Z." + + +def test_the_trailing_given_slot_ignores_the_two_segment_floor() -> None: + """#144's len(segments) == 2 restriction belongs to the lenient + trailing predicate and is NOT inherited here: 'MA' is not + initial-shaped, and a credential list behind it makes the + credential reading likelier rather than less.""" + out = _assigned("Doe, John MA, PhD", lexicon=Lexicon.default()) + # _by_role space-joins by role; the written comma is preserved by + # post_rules' entry pass (#436/#437, rules.md#R1), a later stage + # this assign-only helper does not run -- the full-pipeline case + # row `the_trailing_slot_survives_a_third_comma_part` pins the + # comma-rendered "MA, PhD". What this unit test checks is the role + # assignment alone: both members land in SUFFIX regardless of + # #144's two-segment floor. + assert _by_role(out, Role.SUFFIX) == "MA PhD" + # and the restriction still governs its own predicate + out = _assigned("Doe, John V, PhD", lexicon=Lexicon.default()) + assert _by_role(out, Role.MIDDLE) == "V" + + +def test_a_name_word_behind_the_member_ends_the_run() -> None: + out = _assigned("Doe, John MA Smith", lexicon=Lexicon.default()) + assert _by_role(out, Role.MIDDLE) == "MA Smith" + assert not [a for a in out.ambiguities + if a.kind is AmbiguityKind.SUFFIX_OR_NAME] + + +def test_two_members_in_the_trailing_run_report_once_each() -> None: + out = _assigned("Doe, John MA JD", lexicon=Lexicon.default()) + assert len([a for a in out.ambiguities + if a.kind is AmbiguityKind.SUFFIX_OR_NAME]) == 2 + + +def test_the_no_name_gate_path_still_reports_exactly_once() -> None: + """The sibling of the existing count assertion. Where the first + post-comma piece IS the trailing run, segment 1 holds no name + word, segment_suffix_reading returns non-None, and assign never + enters the placement loop -- so the new emitter is unreachable on + exactly the path the old one owns.""" + for text in ("Doe, MA", "Doe, MA PhD", "Doe, MA JD"): + out = _assigned(text, lexicon=Lexicon.default()) + assert len([a for a in out.ambiguities + if a.kind is AmbiguityKind.SUFFIX_OR_NAME]) == 1, text + + +def test_the_trailing_given_slot_detail_is_verbatim() -> None: + """Both branches, verbatim -- only the string distinguishes them, + the kind being the same either way.""" + detail = { + text: [a.detail for a in + _assigned(text, lexicon=Lexicon.default()).ambiguities + if a.kind is AmbiguityKind.SUFFIX_OR_NAME] + for text in ("Doe, John MA", "Doe, John Ma") + } + assert detail["Doe, John MA"] == [ + "'MA' ending the given part is also an ordinary name word; " + "read as a credential"] + assert detail["Doe, John Ma"] == [ + "'Ma' ending the given part is also an ordinary name word; " + "read as a name"] diff --git a/tests/v2/pipeline/test_post_rules.py b/tests/v2/pipeline/test_post_rules.py index 73bf4dab..3f5fcab0 100644 --- a/tests/v2/pipeline/test_post_rules.py +++ b/tests/v2/pipeline/test_post_rules.py @@ -3,6 +3,7 @@ import pytest +from nameparser import parse from nameparser._lexicon import Lexicon from nameparser._pipeline import run from nameparser._pipeline._post_rules import suffix_entries @@ -1121,3 +1122,39 @@ def test_suffix_entries_keys_on_the_suffix_role() -> None: neither a continuation.""" assert _forced_entry_tags("A B", Role.MIDDLE) == [ ("A", False), ("B", False)] + + +def test_p6_declines_a_credential_lean_class_member() -> None: + """P6 keys on VOCABULARY, not role, so assign's suffix role would + otherwise be silently overridden -- verified by running the + change's assign half alone, which left 'Doe, John DO' family + 'DO Doe'.""" + name = parse("Doe, John DO") + assert name.suffix == "DO" + assert name.family == "Doe" + assert [a.kind.value for a in name.ambiguities] == ["suffix-or-name"] + + +def test_p6_still_claims_an_unambiguous_suffix_particle() -> None: + """The condition is narrowed to AMBIGUOUS_ACRONYM_TAG, which is + what keeps 'vd' and 'mc' -- unambiguous suffix vocabulary, also + particles, also suffix-roled -- inside the run.""" + for text, family in (("Berg, Jan vd", "vd Berg"), + ("Berg, Jan mc", "mc Berg")): + name = parse(text) + assert name.family == family, text + assert name.suffix == "", text + + +def test_p6_keeps_every_non_capital_spelling_of_the_particle_member( +) -> None: + """Capitals decide; the particle rule keeps the rest, and each + reports P6's own kind ONCE (decisions.md#S2, 2026-09-18).""" + for text, family in (("Doe, John do", "do Doe"), + ("Doe, John Do", "Do Doe"), + ("DOE, JOHN DO", "DO DOE"), + ("doe, john do", "do doe")): + name = parse(text) + assert name.family == family, text + assert [a.kind.value for a in name.ambiguities] == \ + ["particle-or-given"], text diff --git a/tests/v2/test_benchmark.py b/tests/v2/test_benchmark.py index 02d6b388..c1c0a452 100644 --- a/tests/v2/test_benchmark.py +++ b/tests/v2/test_benchmark.py @@ -342,3 +342,108 @@ def test_policy_gated_cost_grows_no_worse_than_linearly( reaches: Callable[[Parser], bool]) -> None: assert reaches(parser), "shape no longer reaches the gated stage" _assert_grows_linearly(unit, parser.parse) + + +# The shapes above repeat a unit 800 times and measure the CLOCK. That +# pairing cannot guard a mutual recursion: #531's trailing-slot walk +# re-entered the predicate that owns it, so a run of k ambiguous +# credentials cost 2**k, and a run long enough to separate 2**k from +# k**2 on the clock does not finish -- 24 members measured 5.8s and +# 5.9s on two runs, 32 would be days. So this one counts FRAMES over a +# run of 8 against a run of 16, where the exponential is still cheap +# enough to profile (112ms at 16 UNDER the sys.setprofile hook this +# measures with, which is most of that figure: 23ms unprofiled) and +# already 168x. +# +# Frames are the right instrument here for the #475 reason and one +# more: the recursion IS frame entries, one per re-entry, so the count +# is the defect itself rather than a proxy for it. Measured on the +# unfixed tree before the fix landed: 2,347 frames at 8 and 394,671 at +# 16. On this tree (py3.11): 865 at 8, 1,497 at 16, 5,289 at 64. +# +# TWO ratios, ordered, because one pair cannot see both defects. At 2x +# the input a quadratic is NOT the textbook 4x -- the per-member linear +# work dominates at these sizes, and the per-member memo #531 first +# shipped, a genuine quadratic, measured 2.08x here against this tree's +# 1.73x. So the 8-vs-16 pair guards the EXPONENTIAL and nothing else, +# and a second pair at 4x the input separates quadratic from linear: +# 16-vs-64 measures 3.53x here, 3.33x at cc78c960 (the tree before the +# slot existed at all) and 7.42x with that memo. The ORDER is what +# keeps the larger pair usable: an exponential does not return from a +# run of 64, so the cheap pair is asserted first and the tree that +# would hang has already failed. +_RUN_SMALL = 8 +_RUN_LARGE = 16 +_RUN_HUGE = 64 +#: 8 -> 16, the exponential's bound: 1.73x measured here against the +#: 168x of the recursion this was written for, so 6.0 leaves 3.5x of +#: headroom over the measurement and nothing short of a re-entrant walk +#: can reach it. It does NOT see a quadratic (2.08x, measured above), +#: which is what the second bound is for. +_RUN_MAX_RATIO = 6.0 +#: 16 -> 64, the quadratic's bound: 3.53x measured here against the +#: memo version's 7.42x, so 5.0 sits ~1.4x over the measurement and +#: ~1.5x under the regression it is aimed at. Frame counts do not move +#: under load, so both margins are for a future shape change rather +#: than for runner noise. +_RUN_HUGE_MAX_RATIO = 5.0 + + +def _frames_for(text: str) -> int: + """Python frame entries for ONE parse of `text`. + + One parse, not a mean: this measures growth between two inputs, and + the count is deterministic for a given (tree, interpreter) -- see + `_calls_per_parse`, which takes a mean only because it reports an + absolute figure against a 2% band. + """ + parse("warm up the caches") + calls = 0 + + def counter(frame: object, event: str, arg: object) -> None: + nonlocal calls + if event == "call": + calls += 1 + + sys.setprofile(counter) + try: + parse(text) + finally: + sys.setprofile(None) + return calls + + +def test_a_trailing_credential_run_does_not_cost_exponentially() -> None: + if sys.getprofile() is not None: + pytest.skip("a profile hook is already installed; this test owns it") + small_text = "Doe, John " + "MA " * _RUN_SMALL + large_text = "Doe, John " + "MA " * _RUN_LARGE + huge_text = "Doe, John " + "MA " * _RUN_HUGE + # REACHABILITY, for the reason _POLICY_SHAPES carries one: the walk + # under measurement runs only where every member of the run reads + # as a credential. Route these to MIDDLE instead and the guard + # measures a walk that no longer happens, at a comfortable ratio, + # forever. Asked of the longest run too: the run length is what + # this varies, so "still a credential run" is a claim at each size. + assert parse(small_text).suffix == " ".join(["MA"] * _RUN_SMALL) + assert parse(huge_text).suffix == " ".join(["MA"] * _RUN_HUGE) + small = _frames_for(small_text) + large = _frames_for(large_text) + ratio = large / small + assert ratio < _RUN_MAX_RATIO, ( + f"a run of {_RUN_SMALL} credentials costs {small} frames and a run " + f"of {_RUN_LARGE} costs {large} -- {ratio:.1f}x for 2x the input, " + f"where this tree measures 1.7x and the exponential this guards " + f"measured 168x. The trailing-slot walk in _assign.py has " + f"re-entered the predicate that owns it (#531)") + # Only now the long run: it is the pair that can see a QUADRATIC, + # and it is also the one an exponential never returns from, which + # the assertion above has already caught. + huge = _frames_for(huge_text) + huge_ratio = huge / large + assert huge_ratio < _RUN_HUGE_MAX_RATIO, ( + f"a run of {_RUN_LARGE} credentials costs {large} frames and a run " + f"of {_RUN_HUGE} costs {huge} -- {huge_ratio:.1f}x for 4x the " + f"input, where this tree measures 3.5x and the per-member memo " + f"#531 first shipped measured 7.4x. Something in _assign.py's " + f"trailing slot is asking a walk per member again (#531)") diff --git a/tests/v2/test_facade_cases.py b/tests/v2/test_facade_cases.py index 30487b15..c8804c75 100644 --- a/tests/v2/test_facade_cases.py +++ b/tests/v2/test_facade_cases.py @@ -167,6 +167,20 @@ # answers first). "the_caps_shape_never_reaches_a_tail_segment", "the_chain_reports_the_by_shape_half_too", + # #531: the given part's trailing slot. Five rows, and all five are + # core-only for a field-by-field reason rather than a blanket one + # -- `unlisted_dotted_suffixes`, `unlisted_caps_suffixes`, + # `lenient_comma_suffixes` and `name_order` are each in + # _UNTRANSLATED above, the first three having no v1 Constants + # manager at all and the fourth having no v1 spelling. The + # DEFAULT-policy rows of the same change are ordinary rows and run + # here, which is what pins the facade parity that matters: + # HumanName("Doe, John MA") gives first John, last Doe, suffix MA. + "the_dotted_slot_reports_with_the_switch_off", + "the_caps_switch_reaches_the_trailing_slot", + "the_trailing_slot_ignores_the_strict_comma_knob", + "the_trailing_slot_reads_the_same_under_family_first", + "the_trailing_slot_reads_the_same_under_ff_given_last", }) diff --git a/tests/v2/test_ledger_guards.py b/tests/v2/test_ledger_guards.py index 35e73896..7c97f5d4 100644 --- a/tests/v2/test_ledger_guards.py +++ b/tests/v2/test_ledger_guards.py @@ -749,8 +749,79 @@ def test_case_shape_ids_exist_in_the_inventory() -> None: #: claims 'abd Allah'), and those are recorded in the rules' comments #: instead. _MUST_NOT_MATCH: dict[str, tuple[str, ...]] = { - "fix(#380)": ("vd Berg, Jan", "Jan vd Berg", "Smith vd", - "Berg, Jan mc"), + # Keyed on the full issue text, not "fix(#380)": since #531 that + # substring matches three rules -- the vd one below, and the `mc` + # and `mc or do` ones that carry the same decision over the other + # two collision words -- and 'Berg, Jan mc' is the vd rule's + # boundary and the others' claim. The same rekeying fix(#360)'s + # pair already records, for the same reason. + "fix(#380) a trailing vd after a family comma is the tussenvoegsel, not a post-nominal": + ("vd Berg, Jan", "Jan vd Berg", "Smith vd", "Berg, Jan mc"), + # #531's five trailing-particle names at 1.4.0. 'Doe, John DO' is + # the spelling the capitals take (decisions.md#S2), so this rule + # must not reach it however the other four are spelled; 'Berg, Jan + # vd' is the vd rule's own name; and the two Mc* probes are the + # widening the comment refuses -- a regex over the trailing word + # rather than the literal list. + "fix(#380) a trailing mc or do after a family comma": + ("Doe, John DO", "Berg, Jan vd", "McDonald, Ronald", + "Mcintyre Smith Jr."), + # The same boundary at 2.0.0/2.1.0, where only 'mc' diffs. + "fix(#380) a trailing mc after a family comma": + ("Doe, John DO", "Berg, Jan vd", "McDonald, Ronald", + "Mcintyre Smith Jr."), + # #531's trailing slot. The probes are the three boundaries the + # rule's own comment argues: a member the WRITING declines + # ('Doe, John Ma', 'DOE, JOHN MA' being one-case and so a mover, + # not a probe), a member with a NAME WORD behind it, which ends + # the run ('Doe, John MA Smith'), the third-comma-part shape #144's + # restriction still owns ('Doe, John V, PhD'), the `do` spelling + # P6 keeps ('Doe, John do'), and the comma-less spelling, which is + # #289's and has read the credential since 2.0 ('John Doe MA'). + "fix(#531) a credential ending the given part": + ("Doe, John Ma", "Doe, John MA Smith", "Doe, John V, PhD", + "Doe, John do", "John Doe MA"), + # The declining half. Its probes are the movers: a rule that + # reached one would claim a ROLE regression as a report-only + # diff, which is the direction `fields = ["_ambiguities"]` alone + # cannot catch. + "fix(#531) a member the writing declines": + ("Doe, John MA", "DOE, JOHN MA", "Doe, John X.Y.Z.", + "Doe, John MA Smith"), + # The `do` carve-out, whose probes ARE the pairing + # decisions.md#S2 records: every other spelling attaches to the + # family and reports as P6's fork, and each is measured + # byte-identical across #531. + "fix(#531) capitals take the do collision": + ("Doe, John do", "Doe, John Do", "DOE, JOHN DO", "doe, john do", + "SMITH, JOHN DO", "Nascimento, Edson Arantes do", + "Doe, John van DO"), + # The caseless pair. Their probes are the CJK comma names whose + # member sits in the FIRST post-comma piece, which is #289's slot: + # a rule reaching one would stand ready to explain a regression in + # the half of the comma reading #531 did not touch. + "fix(#531) the trailing slot's positional reading reaches a caseless script": + ("田中 太郎, MA", "毛泽东, MA", "마틴 킹, MA", "田中, 太郎"), + # #531's dotted half at 1.4.0. The probes are the dotted shapes + # the VOCABULARY answers for, which fix(#516)'s rule holds out for + # the same reason, plus the acronym spelling of the same slot, + # which reaches 1.4.0 parity and diffs nowhere. + "fix(#531) a dotted credential ending the given part": + ("John Smith X.Y.Z.", "Doe, John Msc.Ed.", "Smith, A.B.C.", + "Doe, John MA"), + # The interior-credential control at 1.4.0. Its probes are the + # trailing-slot names on either side of it: a rule reaching one + # would absorb #531's own diff under a 2.0-era label. + "fix(comma-family) an interior credential acronym": + ("Doe, John MA", "Doe, John Ma", "John Smith MA", + "Doe, John MA Smith Jr."), + # The 1.4.0 rule #531 re-pointed. 'Doe, John MA' is the name it + # LOST -- the tree now agrees with v1 there, so a rule reaching it + # would explain a diff that no longer exists and stand ready for + # its regression; the other three are the neighbouring readings. + "fix(given-part-trailing-slot)": + ("Doe, John MA", "Doe, John MA Smith", "John Smith Ma", + "Doe, John X.Y.Z."), "fix(#399)": ("Jane van der Berg née", "Jane van der Berg née y Jones", "van der Berg, abdul née Jones", "Jane Smith née Jones"), # Keyed on the full issue text, not "fix(#360)": that substring now @@ -1971,7 +2042,9 @@ class _LatinCopy(NamedTuple): r"Smith Jr\., MA", r"Smith Jr\., Ma", "Smith, MA", "abdul Smith Berg Ma", "abdul Smith Jr Ma", "abdul Smith Ma", "john smith, ma"}), - frozenset({"Davis Royce, Ed", "Freiherr von Berg MA", + frozenset({"Davis Royce, Ed", r"Doe, Dr\. MA", "Doe, MA", + "Doe, MA PhD", r"Doe, Mr\. MA PhD", + "Freiherr von Berg MA", "JOHN SMITH, MA", "Jack MA", r"Jack MA\.", "Jack Wei Ma", r"John Prof\. MA", "John Smith Ma", "John Smith, Ed", "John Smith, MA", "John Smith, Ma", "John de Ma", @@ -1984,14 +2057,57 @@ class _LatinCopy(NamedTuple): r"John Smith Q\.W\.E\.R\.T\.", r"John Smith X\.Y\.Z\.", r"John Smith, A\.B\.", r"Smith, E\.S\.Q\.", r"john smith x\.y\.z\."}), - frozenset({r"J\.A\. K\.D\.", r"Jack X\.Y\.Z\.", + frozenset({"Doe, MA Smith", r"J\.A\. K\.D\.", r"Jack X\.Y\.Z\.", r"John Smith J\.u\.n\.i\.o\.r\.", r"John Smith R\.A\.I\.", "Royce, Ed", r"Smith Jr\., A\.B\.", r"Smith, A\.B\.", "Smith, Ma"}), - frozenset({r"J\.A\. K\.D\.", r"Jack X\.Y\.Z\.", + frozenset({"Doe, MA Smith", r"J\.A\. K\.D\.", r"Jack X\.Y\.Z\.", r"John Smith J\.u\.n\.i\.o\.r\.", r"John Smith R\.A\.I\.", "Royce, Ed", r"Smith Jr\., A\.B\.", r"Smith, A\.B\.", "Smith, Ma", "Steven Hardman, MD, DO, DDS"}), + # #531's movers, one corpus name per alternative -- lists of + # names, not copies of any wordlist. Three sets: the thirteen the + # trailing slot moves, the four the WRITING declines (one set, and + # it is shared by two rules -- the 2.x report-only rule and the + # 1.4.0 fix(given-part-trailing-slot) rule, which hold the same + # four names for opposite halves of the same reading), and the + # five trailing-particle names the fix(#380) decision covers. + # + # What selects the first two is a SLOT the vocabulary participates + # in only at one end: a member of the ambiguous credential class + # ENDING the given part of a family-comma listing. A member + # copying SUFFIX_ACRONYMS_AMBIGUOUS would reach 'Doe, John MA + # Smith', where a name word behind the member ends the run, and + # the whole `do` family, which rules.md#P6 keeps -- every one of + # them measured not to move. _MUST_NOT_MATCH carries the probes. + frozenset({"DOE, JOHN MA", "DOE, MARY JO MA", r"Doe, Dr\. John MA", + r"Doe, J\. MA", "Doe, John BA", "Doe, John MA", + "Doe, John MA JD", "Doe, John MA Jr", + "Doe, John MA PhD", "Doe, John PhD MA", + r"Doe, John Q\. MA", r"Doe, John X\.Y\.Z\.", + "doe, john ma"}), + frozenset({"Doe, John Ed", "Doe, John MA Ma", "Doe, John Ma", + "Doe, Mary Jo Ma"}), + # The third set. What selects these five is a WORD that is both + # particle and credential vocabulary standing last after a family + # comma -- `mc` and `do` -- and a member copying PARTICLES would + # reach every Dutch and Portuguese comma listing in the corpora, + # which is exactly the reach fix(#379)'s shape regex has and this + # rule deliberately does not. + frozenset({"Berg, Jan mc", "Doe, John Do", + "NASCIMENTO, EDSON ARANTES DO", + "Nascimento, Edson Arantes do", "SMITH, JOHN DO"}), + # A fourth set, from #531's fix round: the three 1.4.0 names whose + # member grouping JOINED into a multi-token piece before the slot + # could ask about it. A list of names, not a copy of any wordlist, + # and the vocabularies they touch could not select them anyway -- + # the piece is what the rule turns on, and a piece is not in the + # string. A member copying CONJUNCTIONS or PARTICLES would reach + # every comma listing carrying one, and a member copying + # SUFFIX_ACRONYMS_AMBIGUOUS would reach 'Doe, John MA', which the + # slot takes. _MUST_NOT_MATCH carries both probes. + frozenset({"Doe, John DO Ed", "Doe, John MA y", + "Doe, John van Ma"}), # #449's movers, one corpus name per alternative -- a list of # names, not a copy of any wordlist, so there is no vocabulary for # it to drift from. What selects these names is a SHAPE the @@ -2688,14 +2804,39 @@ def _claim(rule: dict) -> _Claim: # '毛泽东, MA', '田中 太郎, MA', '마틴 킹, MA' -- all of them # names this rule already described. Growth into new # corpus, not a widened regex. + # 2026-09-18, #531: 135 -> 137. The two CJK comma forms that + # round's case rows added, '田中, 太郎 MA' and '김, 민준 MA', + # both tolerated rows and so both in the radar corpus. Growth + # into new corpus again; no role joined the list. "fix(#271/#272/#298) native-script CJK: family-first order, hangul segmentation, the kana license and the dots": - _Claim(135, ('family', 'given', 'middle'), "fd79e8e1e2ef", None), + _Claim(137, ('family', 'given', 'middle'), "94b869596b3e", None), "fix(#274) maiden markers consumed": _Claim(33, ('family', 'maiden', 'middle'), "6f8bf7136b09", None), "fix(cjk-maiden-marker) maiden marker consumed, compounding with the CJK order flip": _Claim(5, ('family', 'given', 'maiden', 'middle'), "bc0e10dd7ec8", None), + # 2026-09-18, #531: 13 -> 19. Six corpus names whose trailing + # word is particle vocabulary this regex already lists -- + # 'Doe, John DO', 'Doe, John Do', 'SMITH, JOHN DO', + # 'Nascimento, Edson Arantes do', 'NASCIMENTO, EDSON ARANTES + # DO' and 'Berg, Jan mc'. Five of the six read exactly what + # this rule describes; the sixth, 'Doe, John DO', is the one + # spelling where the capitals take the credential instead, + # and it files here because the ROLES it moves are this + # rule's. Growth into new corpus, not a widened regex. + # 2026-09-19, #531 review round: 19 -> 20. One new corpus + # name, 'Doe, John van DO' -- the control row for the member + # grouping merges INTO a particle piece, whose trailing word + # is particle vocabulary this regex already lists. Growth into + # new corpus again; no role joined the list. + # 2026-09-19, #531 fix round: 20 -> 21. One new corpus name, + # 'Doe, John MA do', which this rule's shape regex reaches on + # its trailing tussenvoegsel. REACH, not explanation: the name + # diffs {family, middle, suffix} and this rule declares two of + # the three, so it is ineligible and the fix(#380) rule the + # round added explains it. Verified to be that name and no + # other. "fix(#379) a tussenvoegsel after a family comma attaches to the family": - _Claim(13, ('family', 'middle'), "973617235cda", None), + _Claim(21, ('family', 'middle'), "b017f45d04f5", None), "fix(#380) a trailing vd after a family comma is the tussenvoegsel, not a post-nominal": _Claim(2, ('family', 'suffix'), "ec0d45289dc1", None), # 279 -> 280 with #371, and the growth is corpus, not behavior: @@ -2731,8 +2872,27 @@ def _claim(rule: dict) -> _Claim: # 'John Smith, Ma' and 'Smith Jr., Ma' -- rules.md#C1's own # witnesses that the NAME-word count is asked before the # written case is. + # 2026-09-18, #531: 304 -> 310 was that round; 310 -> 334 is + # this one. Twenty-four comma-bearing corpus names arrived + # with #531's case rows and rules.md example lines, every one + # inside this rule's Latin comma range -- the whole 26 new + # names less the two CJK forms. The count grew with the + # CORPUS and not with the rule; no role joined the list. + # 2026-09-19, #531 review round: 334 -> 336. The two control + # rows the round added, 'Doe, Dr. MA' and 'Doe, John van DO', + # both comma-bearing and both inside this rule's Latin comma + # range. Verified to be those two names and no others. + # 2026-09-19, #531 fix round: 336 -> 343. Seven more shape-2 + # rows, and it is the WHOLE of that round's new corpus text -- + # every one of 'Doe, Dr. MA Smith', 'Doe, John DO Ed', + # 'Doe, John MA do', 'Doe, John MA y', 'Doe, John van Ma', + # 'Doe, MA Smith' and 'Doe, Mr. MA PhD' is comma-bearing and + # Latin. Matched name by name against the regex rather than + # inferred from the arithmetic. Reach again, not explanation: + # four of the seven diff in fields this rule cannot admit and + # are explained further down the file. "fix(comma-family) lone post-comma piece routes to suffix/title, not first": - _Claim(310, ('given', 'suffix', 'title'), "2803b1932de1", None), + _Claim(343, ('given', 'suffix', 'title'), "75e89d1dfd04", None), "fix(comma-family) a comma followed only by titles keeps the given/family split": _Claim(2, ('family', 'given'), "5bd9c6d96c38", None), "fix(comma-family) a comma followed only by titles keeps the given/family split, the C1 example": @@ -2757,8 +2917,11 @@ def _claim(rule: dict) -> _Claim: # 'Smith, Ma' -- the pair #289's lean reads in opposite # directions, both of them lone post-comma credentials # by this rule's own description. + # 2026-09-18, #531: 20 -> 21. One new corpus name, 'Doe, MA' -- + # the family-word twin of the 'Smith, MA' above, and a lone + # post-comma credential by this rule's own description. "fix(#296) a lone post-comma credential is a suffix": - _Claim(20, ('family', 'given', 'suffix', 'title'), "c94afd0c7617", None), + _Claim(21, ('family', 'given', 'suffix', 'title'), "3a950b6bb63d", None), "fix(#325) a split credential followed by another suffix after a one-word family comma reads as suffixes": _Claim(6, ('given', 'suffix', 'title'), "7911e0158337", None), "fix(#325) a credential run across a second comma reads as suffixes": @@ -2771,8 +2934,14 @@ def _claim(rule: dict) -> _Claim: # the lone-post-comma rule above and for the same reason. # 2026-09-18, second round: 302 -> 304, the same two new comma # names as the rule above. + # 2026-09-18, #531: 310 -> 334, the same twenty-four new comma + # names as the rule above and for the same reason. + # 2026-09-19, #531 review round: 334 -> 336, the same two new + # comma names as the rule above. + # 2026-09-19, #531 fix round: 336 -> 343, the same seven new + # comma names as the rule above and for the same reason. "fix(comma-precomma-family) pre-comma run reads as family, not given": - _Claim(310, ('family', 'given'), "2803b1932de1", None), + _Claim(343, ('family', 'given'), "75e89d1dfd04", None), "fix(#397) NOT WANTED: a trailing Catalan/Polish linking 'i' is read as a generation marker and the family is lost": _Claim(1, ('family', 'suffix'), "498602f3cfd0", None), "fix(suffix-delimiter-rendering) no-space delimiter core token kept whole": @@ -2790,11 +2959,16 @@ def _claim(rule: dict) -> _Claim: # 2026-09-18: 26 -> 29. The three CJK comma forms #289's own # case rows added ('毛泽东, MA', '田中 太郎, MA', # '마틴 킹, MA'). Growth into new corpus. + # 2026-09-18, #531: 32 -> 34, the two CJK comma forms that + # round added ('田中, 太郎 MA' and '김, 민준 MA'). Both + # regexes here ask only for a comma beside a CJK codepoint, + # so every CJK comma name in the corpora reaches them. "fix(cjk-comma-honorific-peel) glued honorific peels off a post-comma given name": - _Claim(32, ('given', 'suffix'), "a11674bdc8f2", None), + _Claim(34, ('given', 'suffix'), "ec7e1119a738", None), # 2026-09-18: 26 -> 29, the same three new CJK comma forms. + # 2026-09-18, #531: 32 -> 34, the same two as the rule above. "fix(cjk-comma-compound) comma routing compounds with the CJK order flip": - _Claim(32, ('family', 'given', 'suffix', 'title'), "a11674bdc8f2", None), + _Claim(34, ('family', 'given', 'suffix', 'title'), "ec7e1119a738", None), # 37 -> 35 with the 2026-09-05 narrowing, which is a rule # NARROWING and not corpus movement: the three negative # lookbehinds stop the regex matching a listed honorific @@ -3087,7 +3261,41 @@ def _claim(rule: dict) -> _Claim: # name whose given part ends in an ambiguous acronym -- # including the ones v1 read the same way. "fix(given-part-trailing-slot) a credential acronym ending the given part of a family-comma listing reads as a middle name": - _Claim(1, ('middle', 'suffix'), "c164207a6725", ('DEFAULT',)), + # 2026-09-19, #531: 1 -> 4, and the name it HELD is not among + # the four. 'Doe, John MA' reached 1.4.0 parity and left; the + # four that replaced it -- 'Doe, John Ma', 'Doe, John Ed', + # 'Doe, Mary Jo Ma', 'Doe, John MA Ma' -- entered the corpora + # with #531's case rows carrying the SAME reading this rule + # describes, declined by the writing rather than by the slot's + # silence. Verified name by name against the gate's + # UNEXPLAINED block at this baseline. A rule re-pointed, not + # widened: the count grew and the digest moved together. + _Claim(4, ('middle', 'suffix'), "c6d26d145ee1", ('DEFAULT',)), + # 2026-09-19, #531. Three rules the round added at this + # baseline, each literal-anchored and each claiming exactly + # the names its ledger comment names -- verified name by name + # against the gate's UNEXPLAINED block here. #531's dotted + # half (one name, the one v1 parity BREAK the change accepts); + # the interior-credential control, whose member is not at the + # trailing slot at all; and the fix(#380) decision read over + # the two other collision words, five names this PR measured + # byte-identical at cc78c960. + "fix(#531) a dotted credential ending the given part leaves 1.4's middle name": + _Claim(1, ('middle', 'suffix'), "5800f141483d", ('DEFAULT',)), + "fix(comma-family) an interior credential acronym stays a middle-name word": + _Claim(1, ('middle', 'suffix'), "4831097f5067", ('DEFAULT',)), + # 2026-09-19, #531 fix round: two rules land, each literal- + # anchored to names that arrived in the same commit, so both + # claims are born at exactly what their regex spells -- three + # names and one. Neither reading moved in that commit: all + # four parse byte-identically at cc78c960 (measured). + "fix(comma-family) a credential acronym grouping joined into one piece stays middle-name text": + _Claim(3, ('middle', 'suffix'), "4095bb985318", ('DEFAULT',)), + "fix(#380) the do attachment leaves the credential in front of it in the middle name": + _Claim(1, ('family', 'middle', 'suffix'), "8080ca73a18a", + ('DEFAULT',)), + "fix(#380) a trailing mc or do after a family comma is the tussenvoegsel, not a post-nominal": + _Claim(5, ('family', 'suffix'), "13fad71c96d0", ('DEFAULT',)), }, "expected_since_2.0.0.toml": { # #436/#437's Latin alternation, first in every ledger. @@ -3189,15 +3397,24 @@ def _claim(rule: dict) -> _Claim: _Claim(1, ('middle', 'suffix'), "e9f282da0d0f", None), "fix(#429) a wholly-credential segment after a one-word family renders as one entry": _Claim(1, ('suffix', 'title'), "9e0b9e8d5cbe", None), + # 2026-09-18, #531: 13 -> 19, the same six particle-trailing + # comma names as the 1.4 twin, whose entry carries the roster. + # 2026-09-19, #531 review round: 19 -> 20, the same one new + # name as that twin ('Doe, John van DO'). "fix(#379) a tussenvoegsel after a family comma attaches to the family": - _Claim(13, ('_ambiguities', 'family', 'middle'), "973617235cda", None), + # 2026-09-19, #531 fix round: 20 -> 21, the same one new + # corpus name as the 1.4.0 copy ('Doe, John MA do'), reached + # on the trailing tussenvoegsel and explained by neither. + _Claim(21, ('_ambiguities', 'family', 'middle'), "b017f45d04f5", None), # 2026-09-18: 126 -> 131. Five corpus names arrived with # #289/#516's own case rows -- 'J.씨', 'John Smith 田.中.', # '毛泽东, MA', '田中 太郎, MA', '마틴 킹, MA' -- all of them # names this rule already described. Growth into new # corpus, not a widened regex. + # 2026-09-18, #531: 135 -> 137, the same two CJK comma forms + # as the 1.4 twin. "fix(#271/#272/#298) native-script CJK: family-first order, hangul segmentation, the kana license and the dots": - _Claim(135, ('_ambiguities', 'family', 'given', 'middle'), "fd79e8e1e2ef", None), + _Claim(137, ('_ambiguities', 'family', 'given', 'middle'), "94b869596b3e", None), # 37 -> 35 with the same 2026-09-05 narrowing as the 1.4 twin, # whose entry carries the reason. Here the one name that # changed hands, '김민준 박사님', goes to the spaced rule @@ -3281,8 +3498,10 @@ def _claim(rule: dict) -> _Claim: # 'Smith, Ma' -- the pair #289's lean reads in opposite # directions, both of them lone post-comma credentials # by this rule's own description. + # 2026-09-18, #531: 20 -> 21, the same one new name + # ('Doe, MA') as the 1.4 twin. "fix(#296) a lone post-comma credential is a suffix": - _Claim(20, ('suffix', 'title'), "c94afd0c7617", None), + _Claim(21, ('suffix', 'title'), "3a950b6bb63d", None), "fix(#325) a split credential followed by another suffix after a one-word family comma reads as suffixes": _Claim(6, ('given', 'suffix', 'title'), "7911e0158337", None), "fix(#325) a credential run across a second comma reads as suffixes": @@ -3425,7 +3644,17 @@ def _claim(rule: dict) -> _Claim: # one-particle spelling of 'John van der Berg Ma'. Its diff # is the restored chain report at the 2.x baselines and the # role move at 1.4.0 and 2.2/2.3; no role joined the list. - _Claim(20, ('_ambiguities', 'family', 'given', 'middle', 'suffix'), "fe5f68146005", ('DEFAULT',)), + # 2026-09-19, #531: 20 -> 23. Three corpus names, 'Doe, MA', + # 'Doe, MA PhD' and 'Doe, Dr. MA', the lone post-comma + # credentials #531's control rows brought in. Verified to be + # those three and no others; no role joined the list, and the + # 1.4.0 copy does not carry them (its ledger comment says + # which rule already owns their diff there). + # 2026-09-19, #531 fix round: 23 -> 24. One corpus name, + # 'Doe, Mr. MA PhD' -- 'Doe, Dr. MA' with a credential run + # behind the member. Verified to be that name and no other; + # no role joined the list. + _Claim(24, ('_ambiguities', 'family', 'given', 'middle', 'suffix'), "9af04fd5b07d", ('DEFAULT',)), # #516's alternation. Literal-anchored to the by-shape movers, # `orders` DEFAULT. Same reasoning as the rule above: the # class is a shape the vocabulary does not spell, so the @@ -3437,12 +3666,40 @@ def _claim(rule: dict) -> _Claim: # the gate -- which is the one thing this row can say about a # rule whose whole subject is a report. "fix(#289/#516) the ambiguous credential class reports at slots that were silent": - _Claim(8, ('_ambiguities',), "4f4844f54ba1", ('DEFAULT',)), + # 2026-09-19, #531 fix round: one corpus name joins, + # 'Doe, MA Smith' -- the post-comma given slot reporting where + # no role moves at any baseline. Verified to be that name and + # no other. + _Claim(9, ('_ambiguities',), "879e77ab5baa", ('DEFAULT',)), # The 2026-09-18 review round's honorific-peel rule: three # composed CJK forms, radar tier, classified because the arc # intended the move (the ledger comment carries the argument). "fix(#289/#516) the glued CJK honorific peel reads the case lean": _Claim(3, ('_ambiguities', 'family', 'given', 'suffix', 'title'), "4fb6dbdbccd4", ('DEFAULT',)), + # 2026-09-19, #531. Five rules, every one + # literal-anchored and claiming exactly the names its ledger + # comment names -- verified name by name against the gate's + # UNEXPLAINED and UNCLASSIFIED (radar) blocks at this + # baseline. The caseless pair is radar-tier and classified all + # the same, on fix(#289/#516)'s stated ground: a radar diff an + # arc intended is one a release note may be written from. + # `Doe, John DO` moves {middle, suffix} here and + # {family, suffix} from 2.2.0 on, which is P6's own shipping + # date showing through. + "fix(#531) a credential ending the given part of a family-comma listing reads as a credential": + _Claim(13, ('_ambiguities', 'middle', 'suffix'), "c577ebf116f2", ('DEFAULT',)), + "fix(#531) a member the writing declines keeps its name reading and reports the fork": + _Claim(4, ('_ambiguities',), "c6d26d145ee1", ('DEFAULT',)), + "fix(#531) capitals take the do collision from the family-comma particle attachment": + _Claim(1, ('_ambiguities', 'middle', 'suffix'), "8ad64f404621", ('DEFAULT',)), + "fix(#531) the trailing slot's positional reading reaches a caseless script": + _Claim(2, ('_ambiguities', 'middle', 'suffix'), "8419a6f53c3e", ('DEFAULT',)), + # The `mc` collision, at the two baselines where it diffs: + # `mc` joined the never-given particles in 2.2 and P6's + # family-comma attachment shipped in 2.3, so 2.2.0 and 2.3.0 + # already read family 'mc Berg' and carry no rule for it. + "fix(#380) a trailing mc after a family comma is the tussenvoegsel, not a post-nominal": + _Claim(1, ('_ambiguities', 'family', 'suffix'), "105229644206", ('DEFAULT',)), }, # The 2.3 cycle's first rule, and a facade-only render fix: every # role is identical, so `_initials` alone. Reach and digest as in @@ -3613,7 +3870,17 @@ def _claim(rule: dict) -> _Claim: # one-particle spelling of 'John van der Berg Ma'. Its diff # is the restored chain report at the 2.x baselines and the # role move at 1.4.0 and 2.2/2.3; no role joined the list. - _Claim(20, ('_ambiguities', 'family', 'given', 'middle', 'suffix'), "fe5f68146005", ('DEFAULT',)), + # 2026-09-19, #531: 20 -> 23. Three corpus names, 'Doe, MA', + # 'Doe, MA PhD' and 'Doe, Dr. MA', the lone post-comma + # credentials #531's control rows brought in. Verified to be + # those three and no others; no role joined the list, and the + # 1.4.0 copy does not carry them (its ledger comment says + # which rule already owns their diff there). + # 2026-09-19, #531 fix round: 23 -> 24. One corpus name, + # 'Doe, Mr. MA PhD' -- 'Doe, Dr. MA' with a credential run + # behind the member. Verified to be that name and no other; + # no role joined the list. + _Claim(24, ('_ambiguities', 'family', 'given', 'middle', 'suffix'), "9af04fd5b07d", ('DEFAULT',)), # #516's alternation. Literal-anchored to the by-shape movers, # `orders` DEFAULT. Same reasoning as the rule above: the # class is a shape the vocabulary does not spell, so the @@ -3625,12 +3892,34 @@ def _claim(rule: dict) -> _Claim: # the gate -- which is the one thing this row can say about a # rule whose whole subject is a report. "fix(#289/#516) the ambiguous credential class reports at slots that were silent": - _Claim(9, ('_ambiguities',), "f89a7428e317", ('DEFAULT',)), + # 2026-09-19, #531 fix round: one corpus name joins, + # 'Doe, MA Smith' -- the post-comma given slot reporting where + # no role moves at any baseline. Verified to be that name and + # no other. + _Claim(10, ('_ambiguities',), "5d427b25c1c7", ('DEFAULT',)), # The 2026-09-18 review round's honorific-peel rule: three # composed CJK forms, radar tier, classified because the arc # intended the move (the ledger comment carries the argument). "fix(#289/#516) the glued CJK honorific peel reads the case lean": _Claim(3, ('_ambiguities', 'family', 'given', 'suffix', 'title'), "4fb6dbdbccd4", ('DEFAULT',)), + # 2026-09-19, #531. Four rules, every one + # literal-anchored and claiming exactly the names its ledger + # comment names -- verified name by name against the gate's + # UNEXPLAINED and UNCLASSIFIED (radar) blocks at this + # baseline. The caseless pair is radar-tier and classified all + # the same, on fix(#289/#516)'s stated ground: a radar diff an + # arc intended is one a release note may be written from. + # `Doe, John DO` moves {family, suffix} here, P6 having + # shipped in 2.3; at 2.0.0 and 2.1.0 the same name moves + # {middle, suffix} instead. + "fix(#531) a credential ending the given part of a family-comma listing reads as a credential": + _Claim(13, ('_ambiguities', 'middle', 'suffix'), "c577ebf116f2", ('DEFAULT',)), + "fix(#531) a member the writing declines keeps its name reading and reports the fork": + _Claim(4, ('_ambiguities',), "c6d26d145ee1", ('DEFAULT',)), + "fix(#531) capitals take the do collision from the family-comma particle attachment": + _Claim(1, ('_ambiguities', 'family', 'suffix'), "8ad64f404621", ('DEFAULT',)), + "fix(#531) the trailing slot's positional reading reaches a caseless script": + _Claim(2, ('_ambiguities', 'middle', 'suffix'), "8419a6f53c3e", ('DEFAULT',)), }, "expected_since_2.1.0.toml": { # #436/#437's Latin alternation, first in every ledger. @@ -3740,8 +4029,15 @@ def _claim(rule: dict) -> _Claim: _Claim(1, ('middle', 'suffix'), "e9f282da0d0f", None), "fix(#429) a wholly-credential segment after a one-word family renders as one entry": _Claim(1, ('suffix', 'title'), "9e0b9e8d5cbe", None), + # 2026-09-18, #531: 13 -> 19, the same six particle-trailing + # comma names as the 1.4 twin, whose entry carries the roster. + # 2026-09-19, #531 review round: 19 -> 20, the same one new + # name as that twin ('Doe, John van DO'). "fix(#379) a tussenvoegsel after a family comma attaches to the family": - _Claim(13, ('_ambiguities', 'family', 'middle'), "973617235cda", None), + # 2026-09-19, #531 fix round: 20 -> 21, the same one new + # corpus name as the 1.4.0 copy ('Doe, John MA do'), reached + # on the trailing tussenvoegsel and explained by neither. + _Claim(21, ('_ambiguities', 'family', 'middle'), "b017f45d04f5", None), "fix(#424) an unlisted abbreviation is as transparent as a listed title to the leading particle": _Claim(1, ('_ambiguities', 'family', 'given'), "ca7b37af6cf8", None), "fix(#367) a title no longer displaces a leading particle out of the leading position": @@ -3806,8 +4102,10 @@ def _claim(rule: dict) -> _Claim: # 'Smith, Ma' -- the pair #289's lean reads in opposite # directions, both of them lone post-comma credentials # by this rule's own description. + # 2026-09-18, #531: 20 -> 21, the same one new name + # ('Doe, MA') as the 1.4 twin. "fix(#296) a lone post-comma credential is a suffix": - _Claim(20, ('suffix', 'title'), "c94afd0c7617", None), + _Claim(21, ('suffix', 'title'), "3a950b6bb63d", None), "fix(#325) a split credential followed by another suffix after a one-word family comma reads as suffixes": _Claim(6, ('given', 'suffix', 'title'), "7911e0158337", None), "fix(#325) a credential run across a second comma reads as suffixes": @@ -3939,7 +4237,17 @@ def _claim(rule: dict) -> _Claim: # one-particle spelling of 'John van der Berg Ma'. Its diff # is the restored chain report at the 2.x baselines and the # role move at 1.4.0 and 2.2/2.3; no role joined the list. - _Claim(20, ('_ambiguities', 'family', 'given', 'middle', 'suffix'), "fe5f68146005", ('DEFAULT',)), + # 2026-09-19, #531: 20 -> 23. Three corpus names, 'Doe, MA', + # 'Doe, MA PhD' and 'Doe, Dr. MA', the lone post-comma + # credentials #531's control rows brought in. Verified to be + # those three and no others; no role joined the list, and the + # 1.4.0 copy does not carry them (its ledger comment says + # which rule already owns their diff there). + # 2026-09-19, #531 fix round: 23 -> 24. One corpus name, + # 'Doe, Mr. MA PhD' -- 'Doe, Dr. MA' with a credential run + # behind the member. Verified to be that name and no other; + # no role joined the list. + _Claim(24, ('_ambiguities', 'family', 'given', 'middle', 'suffix'), "9af04fd5b07d", ('DEFAULT',)), # #516's alternation. Literal-anchored to the by-shape movers, # `orders` DEFAULT. Same reasoning as the rule above: the # class is a shape the vocabulary does not spell, so the @@ -3951,12 +4259,40 @@ def _claim(rule: dict) -> _Claim: # the gate -- which is the one thing this row can say about a # rule whose whole subject is a report. "fix(#289/#516) the ambiguous credential class reports at slots that were silent": - _Claim(8, ('_ambiguities',), "4f4844f54ba1", ('DEFAULT',)), + # 2026-09-19, #531 fix round: one corpus name joins, + # 'Doe, MA Smith' -- the post-comma given slot reporting where + # no role moves at any baseline. Verified to be that name and + # no other. + _Claim(9, ('_ambiguities',), "879e77ab5baa", ('DEFAULT',)), # The 2026-09-18 review round's honorific-peel rule: three # composed CJK forms, radar tier, classified because the arc # intended the move (the ledger comment carries the argument). "fix(#289/#516) the glued CJK honorific peel reads the case lean": _Claim(3, ('_ambiguities', 'family', 'given', 'suffix', 'title'), "4fb6dbdbccd4", ('DEFAULT',)), + # 2026-09-19, #531. Five rules, every one + # literal-anchored and claiming exactly the names its ledger + # comment names -- verified name by name against the gate's + # UNEXPLAINED and UNCLASSIFIED (radar) blocks at this + # baseline. The caseless pair is radar-tier and classified all + # the same, on fix(#289/#516)'s stated ground: a radar diff an + # arc intended is one a release note may be written from. + # `Doe, John DO` moves {middle, suffix} here and + # {family, suffix} from 2.2.0 on, which is P6's own shipping + # date showing through. + "fix(#531) a credential ending the given part of a family-comma listing reads as a credential": + _Claim(13, ('_ambiguities', 'middle', 'suffix'), "c577ebf116f2", ('DEFAULT',)), + "fix(#531) a member the writing declines keeps its name reading and reports the fork": + _Claim(4, ('_ambiguities',), "c6d26d145ee1", ('DEFAULT',)), + "fix(#531) capitals take the do collision from the family-comma particle attachment": + _Claim(1, ('_ambiguities', 'middle', 'suffix'), "8ad64f404621", ('DEFAULT',)), + "fix(#531) the trailing slot's positional reading reaches a caseless script": + _Claim(2, ('_ambiguities', 'middle', 'suffix'), "8419a6f53c3e", ('DEFAULT',)), + # The `mc` collision, at the two baselines where it diffs: + # `mc` joined the never-given particles in 2.2 and P6's + # family-comma attachment shipped in 2.3, so 2.2.0 and 2.3.0 + # already read family 'mc Berg' and carry no rule for it. + "fix(#380) a trailing mc after a family comma is the tussenvoegsel, not a post-nominal": + _Claim(1, ('_ambiguities', 'family', 'suffix'), "105229644206", ('DEFAULT',)), }, "expected_since_2.3.0.toml": { # #383/#479's three rules, the first this ledger carries. The @@ -3996,7 +4332,17 @@ def _claim(rule: dict) -> _Claim: # one-particle spelling of 'John van der Berg Ma'. Its diff # is the restored chain report at the 2.x baselines and the # role move at 1.4.0 and 2.2/2.3; no role joined the list. - _Claim(20, ('_ambiguities', 'family', 'given', 'middle', 'suffix'), "fe5f68146005", ('DEFAULT',)), + # 2026-09-19, #531: 20 -> 23. Three corpus names, 'Doe, MA', + # 'Doe, MA PhD' and 'Doe, Dr. MA', the lone post-comma + # credentials #531's control rows brought in. Verified to be + # those three and no others; no role joined the list, and the + # 1.4.0 copy does not carry them (its ledger comment says + # which rule already owns their diff there). + # 2026-09-19, #531 fix round: 23 -> 24. One corpus name, + # 'Doe, Mr. MA PhD' -- 'Doe, Dr. MA' with a credential run + # behind the member. Verified to be that name and no other; + # no role joined the list. + _Claim(24, ('_ambiguities', 'family', 'given', 'middle', 'suffix'), "9af04fd5b07d", ('DEFAULT',)), # #516's alternation. Literal-anchored to the by-shape movers, # `orders` DEFAULT. Same reasoning as the rule above: the # class is a shape the vocabulary does not spell, so the @@ -4008,12 +4354,34 @@ def _claim(rule: dict) -> _Claim: # the gate -- which is the one thing this row can say about a # rule whose whole subject is a report. "fix(#289/#516) the ambiguous credential class reports at slots that were silent": - _Claim(9, ('_ambiguities',), "f89a7428e317", ('DEFAULT',)), + # 2026-09-19, #531 fix round: one corpus name joins, + # 'Doe, MA Smith' -- the post-comma given slot reporting where + # no role moves at any baseline. Verified to be that name and + # no other. + _Claim(10, ('_ambiguities',), "5d427b25c1c7", ('DEFAULT',)), # The 2026-09-18 review round's honorific-peel rule: three # composed CJK forms, radar tier, classified because the arc # intended the move (the ledger comment carries the argument). "fix(#289/#516) the glued CJK honorific peel reads the case lean": _Claim(3, ('_ambiguities', 'family', 'given', 'suffix', 'title'), "4fb6dbdbccd4", ('DEFAULT',)), + # 2026-09-19, #531. Four rules, every one + # literal-anchored and claiming exactly the names its ledger + # comment names -- verified name by name against the gate's + # UNEXPLAINED and UNCLASSIFIED (radar) blocks at this + # baseline. The caseless pair is radar-tier and classified all + # the same, on fix(#289/#516)'s stated ground: a radar diff an + # arc intended is one a release note may be written from. + # `Doe, John DO` moves {family, suffix} here, P6 having + # shipped in 2.3; at 2.0.0 and 2.1.0 the same name moves + # {middle, suffix} instead. + "fix(#531) a credential ending the given part of a family-comma listing reads as a credential": + _Claim(13, ('_ambiguities', 'middle', 'suffix'), "c577ebf116f2", ('DEFAULT',)), + "fix(#531) a member the writing declines keeps its name reading and reports the fork": + _Claim(4, ('_ambiguities',), "c6d26d145ee1", ('DEFAULT',)), + "fix(#531) capitals take the do collision from the family-comma particle attachment": + _Claim(1, ('_ambiguities', 'family', 'suffix'), "8ad64f404621", ('DEFAULT',)), + "fix(#531) the trailing slot's positional reading reaches a caseless script": + _Claim(2, ('_ambiguities', 'middle', 'suffix'), "8419a6f53c3e", ('DEFAULT',)), }, } diff --git a/tests/v2/test_properties.py b/tests/v2/test_properties.py index a321835c..f37b0a50 100644 --- a/tests/v2/test_properties.py +++ b/tests/v2/test_properties.py @@ -170,6 +170,79 @@ def test_a_leading_ambiguous_particle_is_reported_once_and_only_once( f"{len(failures)} shape(s) disagree:\n" + "\n".join(failures[:15])) +#: The three `do` rows P6's attachment owns by design. A CLOSED list +#: with the rule that owns each one: a fourth exception fails, which is +#: the whole point of carrying them by name rather than by count. +_COMMA_AGREEMENT_EXCEPTIONS = { + "John Doe do": "rules.md#P6 attaches the lower-case particle member", + "John Q. Doe do": "rules.md#P6 attaches the lower-case particle member", + "Mary Jo Doe do": "rules.md#P6 attaches the lower-case particle member", +} + + +def test_a_comma_form_and_its_comma_less_twin_agree_on_the_class( +) -> None: + """#531's invariant, and the shape of the defect it closed. + + A word ending the given part of a family-comma listing and the + same word ending the comma-less spelling of the same name must be + put in the SAME CLASS -- credential (role SUFFIX) or name (any of + the other six roles). CLASS, not FIELD, and the difference + matters: 'John Doe Ma' reads family 'Ma' with middle 'Doe' while + 'Doe, John Ma' reads middle 'Ma', different fields and both + 'name', and this test must pass on that pair. + + Measured on this tree before #531: 48 of the 78 pairs disagreed, + every one of them in the same direction -- the comma form + declining a credential the comma-less form took. After: 3, and + they are the allowlist above. + """ + words = [] + for base in ("ba", "do", "ed", "jd", "ma", "x.y.z.", "q.w.e.r.t."): + words += [base, base.upper(), base.capitalize()] + words += ["phd", "PhD", "Jr", "xyz", "XYZ"] + shapes = (("John Doe {w}", "Doe, John {w}"), + ("John Q. Doe {w}", "Doe, John Q. {w}"), + ("Mary Jo Doe {w}", "Doe, Mary Jo {w}")) + parser = Parser() + + def side(name: object, word: str) -> str: + for role in Role: + value = getattr(name, role.value) + if value == word or word in value.split(): + return "credential" if role is Role.SUFFIX else "name" + return "absent" + + failures = [] + for word in words: + for plain, comma in shapes: + a = side(parser.parse(plain.format(w=word)), word) + b = side(parser.parse(comma.format(w=word)), word) + if a == b: + continue + key = plain.format(w=word) + if key in _COMMA_AGREEMENT_EXCEPTIONS: + continue + failures.append( + f"{key!r} reads {a} but {comma.format(w=word)!r} " + f"reads {b}") + assert not failures, ( + f"{len(failures)} pair(s) disagree:\n" + "\n".join(failures[:15])) + + +def test_the_comma_agreement_exceptions_are_all_still_exceptions( +) -> None: + """The allowlist's own negative control: a row that stopped being + an exception is an allowlist entry silently covering nothing, and + the sweep above cannot notice that about itself.""" + parser = Parser() + for plain in _COMMA_AGREEMENT_EXCEPTIONS: + comma_form = "Doe, " + plain.replace("John Doe ", "John ").replace( + "John Q. Doe ", "John Q. ").replace("Mary Jo Doe ", "Mary Jo ") + assert parser.parse(plain).suffix, plain + assert not parser.parse(comma_form).suffix, comma_form + + @pytest.mark.parametrize("text", _FORK_CORPUS) def test_a_fork_is_never_reported_twice_on_a_real_name(text: str) -> None: state = run(ParseState(original=text, lexicon=Lexicon.default(), diff --git a/tools/differential/corpus_cjk_tolerated.jsonl b/tools/differential/corpus_cjk_tolerated.jsonl index cb31023e..b72e078a 100644 --- a/tools/differential/corpus_cjk_tolerated.jsonl +++ b/tools/differential/corpus_cjk_tolerated.jsonl @@ -14,6 +14,7 @@ "毛泽东, MA" "王先生, V." "田中 太郎, MA" +"田中, 太郎 MA" "田中, 太郎さん" "田中." "田中. 太郎" @@ -28,6 +29,7 @@ "田中さん, 様." "田中さん." "田中さん., V." +"김, 민준 MA" "김, 민준씨" "김, 민준씨 (Jimmy)" "김. 민준" diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index 5a4757f4..d3c7b438 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -33,9 +33,18 @@ "Berg, abdul vd" "Davis Royce, Ed" "Del Toro" +"Doe, John DO" +"Doe, John DO Ed" +"Doe, John Do" "Doe, John MA" +"Doe, John MA JD" +"Doe, John MA Ma" +"Doe, John MA Smith" +"Doe, John Ma" "Doe, John Msc.Ed." "Doe, John X.Y.Z." +"Doe, John van DO" +"Doe, John van Ma" "Dr Jr" "Dr King Jr" "Dr." @@ -201,6 +210,7 @@ "Rinpoche" "Royce, Ed" "SHIRLEY MACLAINE" +"SMITH, JOHN DO" "STEVEN HARDMAN, MD, DO, DDS" "Salam, abd Allah" "Sean O'Connor" diff --git a/tools/differential/corpus_shapes.jsonl b/tools/differential/corpus_shapes.jsonl index 8ed94df5..1b330df7 100644 --- a/tools/differential/corpus_shapes.jsonl +++ b/tools/differential/corpus_shapes.jsonl @@ -63,11 +63,45 @@ {"name": "хосе и мария сантос", "shape": 1} {"name": "محمد و علي", "shape": 1} {"name": "Beethoven, Ludwig van", "shape": 2} +{"name": "Berg, Jan mc", "shape": 2} +{"name": "DOE, JOHN MA", "shape": 2} +{"name": "DOE, MARY JO MA", "shape": 2} +{"name": "Doe, Dr. John MA", "shape": 2} +{"name": "Doe, Dr. MA", "shape": 2} +{"name": "Doe, Dr. MA Smith", "shape": 2} +{"name": "Doe, J. MA", "shape": 2} {"name": "Doe, John A.", "shape": 2} +{"name": "Doe, John BA", "shape": 2} +{"name": "Doe, John DO", "shape": 2} +{"name": "Doe, John DO Ed", "shape": 2} +{"name": "Doe, John Do", "shape": 2} +{"name": "Doe, John Ed", "shape": 2} +{"name": "Doe, John MA", "shape": 2} +{"name": "Doe, John MA JD", "shape": 2} +{"name": "Doe, John MA Jr", "shape": 2} +{"name": "Doe, John MA Ma", "shape": 2} +{"name": "Doe, John MA PhD", "shape": 2} +{"name": "Doe, John MA Smith", "shape": 2} +{"name": "Doe, John MA do", "shape": 2} +{"name": "Doe, John MA y", "shape": 2} +{"name": "Doe, John Ma", "shape": 2} {"name": "Doe, John Msc.Ed.", "shape": 2} +{"name": "Doe, John PhD MA", "shape": 2} +{"name": "Doe, John Q. MA", "shape": 2} +{"name": "Doe, John X.Y.Z.", "shape": 2} +{"name": "Doe, John van DO", "shape": 2} +{"name": "Doe, John van Ma", "shape": 2} +{"name": "Doe, MA", "shape": 2} +{"name": "Doe, MA PhD", "shape": 2} +{"name": "Doe, MA Smith", "shape": 2} +{"name": "Doe, Mary Jo Ma", "shape": 2} +{"name": "Doe, Mr. MA PhD", "shape": 2} {"name": "John Smith, 1.4", "shape": 2} {"name": "Kennedy, John (Jack)", "shape": 2} +{"name": "NASCIMENTO, EDSON ARANTES DO", "shape": 2} +{"name": "Nascimento, Edson Arantes do", "shape": 2} {"name": "Royce, Ed", "shape": 2} +{"name": "SMITH, JOHN DO", "shape": 2} {"name": "Salem, Abdul Rahman Ahmed", "shape": 2} {"name": "Smith Jr., A.B.", "shape": 2} {"name": "Smith Jr., John", "shape": 2} @@ -83,6 +117,7 @@ {"name": "Smith, MA", "shape": 2} {"name": "Smith, Ma", "shape": 2} {"name": "de la Vega, Juan", "shape": 2} +{"name": "doe, john ma", "shape": 2} {"name": "Davis Royce, Ed", "shape": 3} {"name": "Dr. John P. Doe-Ray, CLU, CFP, LUTC", "shape": 3} {"name": "JOHN SMITH, MA", "shape": 3} diff --git a/tools/differential/expected_since_1.4.0.toml b/tools/differential/expected_since_1.4.0.toml index f22c6a8a..187f096c 100644 --- a/tools/differential/expected_since_1.4.0.toml +++ b/tools/differential/expected_since_1.4.0.toml @@ -3565,9 +3565,9 @@ issue = "fix(#289) a written case contrast decides a bare ambiguous acronym" # each member of a pair goes. # # Literal-anchored to the measured movers. The four 2.x ledgers share -# ONE set; this copy is that set less one name, for the reason the -# paragraph below gives. "One set for all five" was written before -# that carve-out existed and is corrected in all five copies +# ONE set; this copy is that set less FOUR names, for the reasons the +# paragraphs below give. "One set for all five" was written before +# the first carve-out existed and is corrected in all five copies # (2026-09-18). The set is literal because the class is # wide (every mixed-case name whose trailing piece is a listed # ambiguous acronym, plus every comma form with two name words before @@ -3585,6 +3585,18 @@ issue = "fix(#289) a written case contrast decides a bare ambiguous acronym" # rather than the acronym's half of it, and stands first -- so the # name is narrowed out rather than left to be decided by file order, # which the harness refuses to let go undeclared. +# +# THREE MORE are held out here and only here, added to the 2.x copies +# on 2026-09-19: 'Doe, MA', 'Doe, MA PhD' and 'Doe, Dr. MA', the lone +# post-comma credentials #531's control rows brought into the corpora. +# They diff at this baseline too -- v1 read first 'MA', last 'Doe' -- +# but that diff is already claimed here by `fix(comma-family) lone +# post-comma piece routes to suffix/title, not first`, which stands +# near the top of this file and whose {given, suffix, title} admits +# their {given, suffix}. Adding them to this rule would give one diff +# two rules and leave file order to arbitrate, which is the shape the +# harness refuses to let go undeclared; so the 2.x copies carry them +# and this one does not. name_regex = "^(?:Davis Royce, Ed|Freiherr von Berg MA|JOHN SMITH, MA|Jack MA|Jack MA\\.|Jack Wei Ma|John Smith Ma|John Smith, Ed|John Smith, MA|John Smith, Ma|John de Ma|John van der Berg Ma|Smith Jr\\., MA|Smith Jr\\., Ma|Smith, MA|abdul Smith Berg Ma|abdul Smith Jr Ma|abdul Smith Ma|john smith, ma)$" fields = ["family", "given", "middle", "suffix"] orders = ["DEFAULT"] @@ -3634,38 +3646,197 @@ orders = ["DEFAULT"] [[change]] issue = "fix(given-part-trailing-slot) a credential acronym ending the given part of a family-comma listing reads as a middle name" -# THIS PR DID NOT MOVE THIS READING, and the rule exists only because -# the name entered the corpus here. 'Doe, John MA' reads suffix 'MA' -# on the 1.4.0 wheel and middle 'MA' at 2.0.0, 2.1.0, 2.2.0, 2.3.0 and -# on this tree alike -- a 2.0-era reading, so the diff is against v1 -# only and no 2.x ledger carries it. The name arrived as a rules.md#S2 -# boundary example line (the verification round of 2026-09-18), which -# is why a reading five years old is being classified today. -# -# The mechanism is a SLOT rather than a vocabulary or a case lean: -# assign's family-comma emitter reads the first piece after the comma, -# so a class member standing at the END of the given part is never -# asked about, and the positional reading takes it as a middle name in -# silence. rules.md#S2 states which slots report and names this one as -# the slot that does not; rules.md#S3 carries the same boundary for -# the dotted spelling. -# -# The dotted sibling is NOT here and must not be folded in: 'Doe, John -# X.Y.Z.' reads middle 'X.Y.Z.' at 1.4.0 too, so it produces no diff -# at this baseline at all. One name, one reading, one spelling -- the -# acronym half is the only half with a v1 disagreement. -# -# Whether to widen the emitter to this slot is open, recorded in -# decisions.md#S2's 2026-09-18 comma-reach entry, which now carries -# the 1.4.0 measurement as a PARITY question beside the noise one. If -# that widening ever lands, the roles here stop moving and this rule -# retires; a `suffix-or-name` report would be added on top and is not -# a field this rule claims. +# 2026-09-19, #531: THE WIDENING THIS RULE'S OWN COMMENT PREDICTED HAS +# LANDED, AND THE RULE NEITHER RETIRED NOR NARROWED -- IT CHANGED +# NAMES. What the rule SAYS is still true, of a different set, and for +# a different reason, so read the two paragraphs below as replacing +# the retirement paragraph rather than amending it. +# +# 'Doe, John MA' is GONE from the regex: the emitter now reaches the +# trailing slot of the given part, so the tree reads suffix 'MA' +# exactly as the 1.4.0 wheel does and the name has no diff at this +# baseline at all (the report it gains is a v2 surface that cannot +# enter a diff below baseline 2.0, which is why the 2.x ledgers carry +# it and this one does not). +# +# What is here instead is the four names where the reading is STILL +# the middle name v1 read as a post-nominal -- and the mechanism is no +# longer the slot's silence. It is the WRITING: a member written in a +# cased form that is neither capitals nor wholly lower, in a name +# written in more than one case, is written the way a name is written +# and stays one (rules.md#S2, decisions.md#S2). 'Doe, John Ma', +# 'Doe, John Ed' and 'Doe, Mary Jo Ma' are Title-cased members in +# mixed-case names, and 'Doe, John MA Ma' is a two-member run whose +# LAST member is Title-cased, so the walk declines there and the whole +# run stays middle text. v1 took all four as suffixes, and the four +# keep the 2.0-era reading against it. Each of them now REPORTS the +# fork it declined, which is #531's doing and is a v2 surface, so it +# is invisible here and carried by the 2.x ledgers' own rule. +# +# The four names entered the corpora with #531's case rows +# (2026-09-18); the reading they carry is a 2.0-era one this PR did +# not move, exactly as 'Doe, John MA's was when this rule was written. +# +# The dotted sibling is still NOT here and must not be folded in: it +# now LEAVES 1.4.0's middle for the credential reading, which is the +# opposite direction, and has a rule of its own at the bottom of this +# file. # # Literal-anchored for the same reason the fix(#289) rule above gives: # the class is a slot, and a regex for the slot would claim every # family-comma name whose given part ends in an ambiguous acronym, -# including the ones v1 agreed with. -name_regex = "^Doe, John MA$" +# including 'Doe, John MA' itself, which v1 now agrees with. +# _MUST_NOT_MATCH carries that name as a probe, beside +# 'Doe, John MA Smith', whose member is not at the trailing slot at +# all and whose diff has the comma-family rule below. +name_regex = "^(?:Doe, John Ed|Doe, John MA Ma|Doe, John Ma|Doe, Mary Jo Ma)$" +fields = ["middle", "suffix"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#531) a dotted credential ending the given part leaves 1.4's middle name" +# The dotted half of #531, and the one place the change accepts a v1 +# parity BREAK rather than restoring one. 'Doe, John X.Y.Z.' reads +# middle 'X.Y.Z.' on the 1.4.0 wheel (measured) and middle at 2.0.0 +# through 2.3.0 too; the tree reads suffix 'X.Y.Z.'. +# +# Deliberate, and the reason is agreement rather than history: a +# multi-chunk dotted token no vocabulary claims joins the ambiguous +# credential class BY SHAPE and is read by POSITION (rules.md#S3, +# fix(#516)'s rule above), the comma-less 'John Doe X.Y.Z.' reads it +# as the credential, and at this slot the comma has already named the +# family so the words-to-spare count is satisfied by construction. +# The acronym half of #531 restores v1; this half departs from it, and +# decisions.md#S2 records the pair as one decision with two +# directions. +# +# `listed_lean` returns None wherever the by-shape tag rides, so no +# case evidence is consulted here at all -- which is why this name +# needs a rule of its own rather than sitting with the acronyms above +# or with the four the writing declines. +# +# Literal, one name. A regex for the dotted SHAPE would claim every +# dotted token the vocabulary already answers for, which is the reach +# fix(#516)'s own comment refuses; _MUST_NOT_MATCH carries the probes +# it names. +name_regex = "^Doe, John X\\.Y\\.Z\\.$" fields = ["middle", "suffix"] orders = ["DEFAULT"] + +[[change]] +issue = "fix(comma-family) an interior credential acronym stays a middle-name word" +# THIS PR DID NOT MOVE THIS READING, and the rule exists only because +# the name entered the corpus here -- the shape the rule above is +# bounded by, arriving as #531's recorded negative control +# (2026-09-18). 'Doe, John MA Smith' reads middle 'Smith', suffix 'MA' +# on the 1.4.0 wheel and middle 'MA Smith' at 2.0.0 and every baseline +# since, this tree included. Measured case-independent: 'DOE, JOHN MA +# SMITH' and 'doe, john ma smith' read the same way. +# +# It is NOT the trailing slot and must not be folded into either rule +# above: a NAME WORD stands behind the member, which ends the trailing +# run, so no fork is consulted, nothing reports, and the member is an +# ordinary middle-name word. What v1 did instead was read the +# post-comma part's suffix vocabulary by POSITION and lift 'MA' out of +# the middle of the given part -- a family-comma routing difference +# 2.0 settled, which is why this files under the comma-family group +# that tests/v2/cases.py gives the row. +# +# Literal, one name: the shape is "a family-comma name whose given +# part carries an ambiguous acronym with a name word behind it", and a +# regex for it would claim the trailing-slot names two rules above. +name_regex = "^Doe, John MA Smith$" +fields = ["middle", "suffix"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(comma-family) a credential acronym grouping joined into one piece stays middle-name text" +# THIS PR DID NOT MOVE THESE READINGS EITHER, and the three names +# arrived for the same reason the one above did: #531's fix round +# needed the rows that PIN the trailing slot's piece test, so they +# entered the corpora (2026-09-19). All three read the middle name at +# 2.0.0 and at every baseline since, this tree included, and v1 read +# the trailing word as a post-nominal. +# +# The mechanism is a STAGE EARLIER than the slot, and it is the piece +# rather than the word: grouping merges each of these members into a +# multi-token piece before assign asks anything -- 'MA y' on the +# conjunction, 'van Ma' on the particle chain in front, 'DO Ed' on the +# chain the member itself heads -- and the slot reads a piece, so a +# piece of two tokens is not the lone member the branch asks about. +# What v1 did instead was read the post-comma part's suffix +# vocabulary by POSITION, which is the same family-comma routing +# difference 'Doe, John MA Smith' carries above; these differ from it +# only in what ends the reach. +# +# Literal, three names. The shape is "a family-comma name whose given +# part ends in a joined piece carrying a class member", and a regex +# for it would need the join, which is not in the string: 'Doe, John +# MA Smith' and 'Doe, John MA' are spelled alike and neither is here. +# _MUST_NOT_MATCH carries both as probes. +name_regex = "^(?:Doe, John DO Ed|Doe, John MA y|Doe, John van Ma)$" +fields = ["middle", "suffix"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#380) the do attachment leaves the credential in front of it in the middle name" +# THE SAME DECISION AS THE fix(#380) RULE BELOW, on a name that moves +# one field more than any of its five. 'Doe, John MA do' entered the +# corpora with #531's fix round (2026-09-19) and is NOT this PR's +# doing -- byte-identical at cc78c960 and on the 2.3.0 wheel. +# +# v1 read the whole tail as post-nominals, suffix 'MA, do'. Here the +# lower-case 'do' leans nothing, so the trailing slot declines it to +# P6 exactly as 'Doe, John Do' does -- and a piece that walk refuses +# ENDS the run, so the all-caps 'MA' in front of it is never asked +# and stays middle-name text, capitals and all. Two readings on one +# string: {family, suffix} is the attachment the rule below names, +# and `middle` is the silence behind it, which is why the name needs +# a rule of its own rather than joining that literal set -- a +# three-field diff is no subset of its {family, suffix}. +# +# Literal, one name. tests/v2/cases.py's +# the_particle_carve_out_silences_the_member_in_front is the row. +name_regex = "^Doe, John MA do$" +fields = ["family", "middle", "suffix"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#380) a trailing mc or do after a family comma is the tussenvoegsel, not a post-nominal" +# THE SAME DECISION AS THE fix(#380) RULE ABOVE, over the two other +# words that carry the collision it names. Five names entered the +# corpora with #531's case rows and its `do` pairing (2026-09-18), and +# every one of them reads here what 'Berg, Jan vd' reads there: the +# trailing word attaches to the family the comma already named +# (rules.md#P6) where v1 read it as a post-nominal, so {family, +# suffix} moves and nothing else does. +# +# 'Berg, Jan mc' is the second word of decisions.md#vocabulary- +# collisions' live pair, and decisions.md#suffix-acronym-collisions +# is where 'mc' was left in the acronym set as no borne name at all. +# The other four are the `do` rows: 'Doe, John Do', 'SMITH, JOHN DO', +# 'NASCIMENTO, EDSON ARANTES DO' and 'Nascimento, Edson Arantes do'. +# +# THIS PR DID NOT MOVE ANY OF THE FIVE -- verified name by name +# against the tree at cc78c960, where all five parse byte-identically +# (2026-09-19). What #531 DID do is take the sixth spelling OUT of +# this set: 'Doe, John DO' is an all-caps member in a mixed-case name, +# the one writing that carries a positive credential lean, so the +# credential reading wins, P6 stands down, and the name reaches 1.4.0 +# parity and drops out of this ledger entirely. That is +# decisions.md#S2's `do` pairing, and 'Doe, John DO' is a +# _MUST_NOT_MATCH probe here because of it -- the one-case +# 'SMITH, JOHN DO' beside it in the regex is the accepted cost the +# same entry records. +# +# Literal-anchored, unlike the vd rule's shape regex: `do` is +# ordinary given-name and particle text in a way `vd` is not, so a +# regex over the trailing word would claim every Portuguese and +# Vietnamese comma listing in the corpora. The vd rule's own +# _MUST_NOT_MATCH key was rekeyed to its full issue text when this +# rule landed, for the reason fix(#360)'s pair already records: the +# bare tag now matches two rules, and each one's boundary is the +# other's claim. +name_regex = "^(?:Berg, Jan mc|Doe, John Do|NASCIMENTO, EDSON ARANTES DO|Nascimento, Edson Arantes do|SMITH, JOHN DO)$" +fields = ["family", "suffix"] +orders = ["DEFAULT"] diff --git a/tools/differential/expected_since_2.0.0.toml b/tools/differential/expected_since_2.0.0.toml index 3a70abdd..c2ed9e4f 100644 --- a/tools/differential/expected_since_2.0.0.toml +++ b/tools/differential/expected_since_2.0.0.toml @@ -2377,14 +2377,40 @@ issue = "fix(#289) a written case contrast decides a bare ambiguous acronym" # Both sit on this rule because the rule is what states which way # each member of a pair goes. # +# 2026-09-19, #531: three names join, all of them lone post-comma +# credentials this rule's own second sentence already describes. +# 'Doe, MA' and 'Doe, MA PhD' are 'Smith, MA's family-word twins and +# 'Doe, Dr. MA' is the same slot behind a title; each reads family +# 'Doe' with the credential run in `suffix` -- 'MA', and 'MA PhD' +# on the one that has a run -- where the baseline read given 'MA'. +# They +# entered the corpora with #531's control rows and are NOT #531's +# doing -- verified byte-identical at cc78c960, the #530 merge +# (2026-09-19). No role joined the field list: the three move +# {given, suffix} and, on two of them, `_ambiguities`. +# +# 2026-09-19, #531 review round: 'Doe, Mr. MA PhD' joins them. It is +# 'Doe, Dr. MA' with a credential run behind the member -- the gate +# reads the segment whole, so the member joins the run rather than +# taking the given slot -- and it moves {given, suffix}, both already +# in the field list. It entered the corpora with that round's control +# rows and is NOT its doing: byte-identical at cc78c960 (measured +# 2026-09-19). +# # Literal-anchored to the measured movers, and ONE SET FOR THE FOUR # 2.x LEDGERS so the rule says the same thing at each of them. The -# 1.4.0 copy differs by exactly one member and holds 'John Prof. MA' +# 1.4.0 copy differs by FOUR members now. It holds 'John Prof. MA' # out: 1.4.0 read a trailing period-marked title as a title itself, # so that name's diff there belongs to fix(#316)'s rule and not to # this one. "One set for all five" was written before that carve-out # existed and was false in these four copies from the day it did -# (2026-09-18). The set is literal because the class is +# (2026-09-18). And it holds the three names above out for a +# different reason: at 1.4.0 their diff is already claimed by +# `fix(comma-family) lone post-comma piece routes to suffix/title, +# not first`, which stands far earlier in that file, so adding them +# there would reach a name without explaining it and would put two +# rules on one diff with nothing to arbitrate between them +# (2026-09-19). The set is literal because the class is # wide (every mixed-case name whose trailing piece is a listed # ambiguous acronym, plus every comma form with two name words before # the comma) and a regex for the class would claim the one-case @@ -2392,7 +2418,7 @@ issue = "fix(#289) a written case contrast decides a bare ambiguous acronym" # given baseline is simply one the reading already agreed with there. # `fields` is per-baseline, the union the run at THAT baseline # measures (#452). -name_regex = "^(?:Davis Royce, Ed|Freiherr von Berg MA|JOHN SMITH, MA|Jack MA|Jack MA\\.|Jack Wei Ma|John Prof\\. MA|John Smith Ma|John Smith, Ed|John Smith, MA|John Smith, Ma|John de Ma|John van der Berg Ma|Smith Jr\\., MA|Smith Jr\\., Ma|Smith, MA|abdul Smith Berg Ma|abdul Smith Jr Ma|abdul Smith Ma|john smith, ma)$" +name_regex = "^(?:Davis Royce, Ed|Doe, Dr\\. MA|Doe, MA|Doe, MA PhD|Doe, Mr\\. MA PhD|Freiherr von Berg MA|JOHN SMITH, MA|Jack MA|Jack MA\\.|Jack Wei Ma|John Prof\\. MA|John Smith Ma|John Smith, Ed|John Smith, MA|John Smith, Ma|John de Ma|John van der Berg Ma|Smith Jr\\., MA|Smith Jr\\., Ma|Smith, MA|abdul Smith Berg Ma|abdul Smith Jr Ma|abdul Smith Ma|john smith, ma)$" fields = ["family", "given", "middle", "suffix", "_ambiguities"] orders = ["DEFAULT"] @@ -2456,6 +2482,14 @@ issue = "fix(#289/#516) the ambiguous credential class reports at slots that wer # run it is, because 'DO' leans credential in a mixed-case name. That # is the one place this arc quiets a report rather than adding one. # +# 2026-09-19, #531 review round: 'Doe, MA Smith' joins, and it is the +# post-comma given slot with a NAME WORD behind the member: no role +# moves at any baseline -- the member is the given name here as it +# always was -- and what is new is only the report that emitter makes. +# Its silent twin 'Doe, Dr. MA Smith' puts a title in that first piece +# instead, which the emitter reads rather than the member, and agrees +# with every baseline on every field, so it is in no ledger at all. +# # `_ambiguities` alone, which keeps `_initials` out of these rows as # always (#484), and the rule cannot absorb a role diff on any of # these names for the same reason. Literal-anchored, one set for all @@ -2469,7 +2503,7 @@ issue = "fix(#289/#516) the ambiguous credential class reports at slots that wer # narrowed out rather than left to file order. The comma-structure # flag it loses is a 2.2-era report in any case, and the 2.2.0 and # 2.3.0 ledgers are where the loss is recorded. -name_regex = "^(?:J\\.A\\. K\\.D\\.|Jack X\\.Y\\.Z\\.|John Smith J\\.u\\.n\\.i\\.o\\.r\\.|John Smith R\\.A\\.I\\.|Royce, Ed|Smith Jr\\., A\\.B\\.|Smith, A\\.B\\.|Smith, Ma)$" +name_regex = "^(?:Doe, MA Smith|J\\.A\\. K\\.D\\.|Jack X\\.Y\\.Z\\.|John Smith J\\.u\\.n\\.i\\.o\\.r\\.|John Smith R\\.A\\.I\\.|Royce, Ed|Smith Jr\\., A\\.B\\.|Smith, A\\.B\\.|Smith, Ma)$" fields = ["_ambiguities"] orders = ["DEFAULT"] @@ -2498,3 +2532,177 @@ issue = "fix(#289/#516) the glued CJK honorific peel reads the case lean" name_regex = "^(?:Dr\\. 김민준씨, MA|Jo김민준씨, DO|Kim김민준씨, MA)$" fields = ["title", "given", "family", "suffix", "_ambiguities"] orders = ["DEFAULT"] + +# --------------------------------------------------------------- +# #531: THE GIVEN PART'S TRAILING SLOT. Four rules -- the thirteen +# the credential reading takes, the four the writing declines and that +# gain a REPORT alone, the one `do` spelling the capitals take from +# P6, and the two caseless comma forms on the radar tier. The decision is +# decisions.md#S2's 2026-09-18 pair; rules.md#S2, #S3, #C1 and #P6 +# carry the statements. +# --------------------------------------------------------------- + +[[change]] +issue = "fix(#531) a credential ending the given part of a family-comma listing reads as a credential" +# After a family comma, a word of the ambiguous credential class +# ending the given part is now read by the same rule the comma-less +# spelling uses for a word ending the name. The comma has already +# named the family and the first word after it is the given name, so +# the words to spare are there by construction and the count says +# nothing: the word reads as the credential unless its WRITING says +# otherwise. rules.md#S2 states it, rules.md#S3 carries the by-shape +# spelling, and decisions.md#S2 records the decision. +# +# Thirteen names, and they are the class read from every side the +# corpora reach it: the bare member ('Doe, John MA', 'Doe, John BA'); +# its one-case spellings, where the lean is inert and the POSITIONAL +# reading is not ('DOE, JOHN MA', 'doe, john ma', 'DOE, MARY JO MA'); +# the slot behind a title ('Doe, Dr. John MA') and behind an initial +# ('Doe, J. MA'); a run of two members, where the credential the +# member joins was already a suffix so `suffix` GROWS rather than +# filling ('Doe, John MA JD', 'Doe, John MA PhD', 'Doe, John PhD MA', +# 'Doe, John MA Jr'); the slot reached past a middle initial, which +# keeps its own middle ('Doe, John Q. MA', 'DOE, MARY JO MA'); and the +# by-shape member, whose lean is always None so the positional reading +# takes it outright ('Doe, John X.Y.Z.'). +# +# Do not read "thirteen role movers" off that list at every baseline: +# `fields` is the union the run at THIS baseline measures (#452), and +# a name here whose roles the baseline already agreed with contributes +# only its report. 'Doe, John BA' is that name at 2.1.0 and 2.2.0, and +# 'Doe, John MA JD' is the other end of the range, gaining TWO reports +# for its two members. +# +# `_ambiguities` is in the fields because every one of the thirteen +# gains the existing SUFFIX_OR_NAME -- no new kind, and the report +# tracks the FORK CONSULTED rather than which way it went, which is +# the rule fix(#289/#516) above already states for the other slots. +# #452's over-declaration check recomputes the union on every run. +# +# Literal-anchored for the reason the two rules above give, and it is +# sharper here: the class is a SLOT, so a regex for it would claim +# every family-comma name whose given part ends in an ambiguous +# acronym -- including the four the WRITING declines, which keep their +# name reading and have the rule below, and 'Doe, John MA Smith', +# where a name word behind the member ends the run so nothing moves +# and nothing reports. _MUST_NOT_MATCH carries both, with the `do` +# spellings P6 keeps and the third-comma-part name #144's restriction +# still owns. +name_regex = "^(?:DOE, JOHN MA|DOE, MARY JO MA|Doe, Dr\\. John MA|Doe, J\\. MA|Doe, John BA|Doe, John MA|Doe, John MA JD|Doe, John MA Jr|Doe, John MA PhD|Doe, John PhD MA|Doe, John Q\\. MA|Doe, John X\\.Y\\.Z\\.|doe, john ma)$" +fields = ["_ambiguities", "middle", "suffix"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#531) a member the writing declines keeps its name reading and reports the fork" +# The other direction of the same fork, and the whole of what these +# four names do: no role moves at any 2.x baseline and the decision +# starts being REPORTED. 'Doe, John Ma', 'Doe, John Ed' and +# 'Doe, Mary Jo Ma' carry a Title-cased member in a mixed-case name, +# which is written the way a name is written, so the member stays a +# middle name; 'Doe, John MA Ma' is a two-member run whose last member +# is Title-cased, so the walk declines there and the run stays middle +# text whole. +# +# Reporting a DECLINED fork is #530's stated rule -- the report tracks +# the fork consulted, not the lean -- and rules.md#A1's "a kind is +# worth adding only if a reader would hesitate too" is the standing +# objection it answers. It is also the only part of #531 that adds a +# report without moving a field, which is why it is a rule of its own +# rather than a widening of the one above: `fields` is +# `_ambiguities` alone, so this rule cannot absorb a role diff on any +# of the four. +# +# At 1.4.0 these same four DO move roles -- v1 read all four members +# as post-nominals -- and that divergence is carried there by +# fix(given-part-trailing-slot), whose comment records that #531 +# changed which names it holds rather than retiring it. +# +# Literal-anchored: the class is the slot's declining half, and a +# regex for it would claim the thirteen movers above. +name_regex = "^(?:Doe, John Ed|Doe, John MA Ma|Doe, John Ma|Doe, Mary Jo Ma)$" +fields = ["_ambiguities"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#531) capitals take the do collision from the family-comma particle attachment" +# 'Doe, John DO', alone, because the word is alone in the class: `do` +# is the one ambiguous credential that is also particle vocabulary, so +# this slot and P6's attachment (rules.md#P6) want the same word. +# Derek's decision, recorded at decisions.md#S2: CAPITALS DECIDE, AND +# THE PARTICLE RULE KEEPS EVERY OTHER SPELLING. An all-caps member in +# a name written in more than one case carries a positive credential +# lean, so the credential reading wins and P6 stands down; every other +# spelling attaches exactly as it did, with P6's own +# `particle-or-given` and no second report. +# +# THE PAIRING IS THE ARGUMENT and the accepted cost is half of it. In +# ONE CASE the rule cannot tell 'NASCIMENTO, EDSON ARANTES DO' from +# 'SMITH, JOHN DO' and reads both as the particle -- right about the +# Brazilian record, wrong about the osteopath. Those two, and +# 'Doe, John Do', 'DOE, JOHN DO', 'doe, john do' and +# 'Nascimento, Edson Arantes do', are _MUST_NOT_MATCH probes: this +# rule has no business claiming a single one of them, and each is a +# name #531 measured to be byte-identical before and after. +# +# `fields` is per-baseline and the two halves of this file's set +# differ by which role the particle came OUT of, which is P6's own +# history: {middle, suffix} at 2.0.0 and 2.1.0, where the +# baseline read middle 'DO' because P6's family-comma attachment had +# not shipped, and {family, suffix} at 2.2.0 and 2.3.0, where it had +# and the baseline reads family 'DO Doe'. +name_regex = "^Doe, John DO$" +fields = ["_ambiguities", "middle", "suffix"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#531) the trailing slot's positional reading reaches a caseless script" +# '田中, 太郎 MA' and '김, 민준 MA': middle 'MA' -> suffix 'MA', one +# report each. A caseless script answers True to the one-case test, so +# the lean is None -- and at this slot a None lean is not inert, +# because the POSITIONAL reading the lean would have overridden is +# itself the credential. "Caseless is inert" holds for the LEAN and +# not for the outcome, which is the sentence tests/v2/cases.py's two +# rows carry. +# +# Both are COMPOSED forms -- a family comma between a CJK family and a +# CJK given name, which no East Asian writing system produces -- so +# both are `tolerated` case rows on the radar tier +# (decisions.md#cjk-comma-demotion), and their diffs print under +# UNCLASSIFIED (radar) rather than blocking. They are classified all +# the same, on fix(#289/#516)'s stated ground: a radar diff an arc +# INTENDED is one a release note may be written from, and an +# unclassified one is not. +# +# Literal-anchored, and the probes are the CJK comma names that do NOT +# move at this slot: '田中 太郎, MA', '毛泽东, MA' and '마틴 킹, MA' put +# their member in the FIRST post-comma piece, which is #289's slot and +# not this one. +name_regex = "^(?:田中, 太郎 MA|김, 민준 MA)$" +fields = ["_ambiguities", "middle", "suffix"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#380) a trailing mc after a family comma is the tussenvoegsel, not a post-nominal" +# THE SAME DECISION AS THE fix(#380) RULE ABOVE, over the other word +# that carries the collision it names. 'Berg, Jan mc' entered the +# corpora with #531's case rows (2026-09-18) as the negative control +# for the `do` carve-out -- 'mc' is unambiguous suffix vocabulary AND +# a never-given particle, so P6's run claims it whatever role assign +# gave it, and #531 left it byte-identical (verified against the tree +# at cc78c960, 2026-09-19). +# +# The diff is against these two baselines only: `mc` joined the +# never-given particles in 2.2 (decisions.md#suffix-acronym- +# collisions) and P6's family-comma attachment shipped in 2.3, so +# 2.2.0 and 2.3.0 already read family 'mc Berg' and carry no rule for +# it. `_ambiguities` is in the fields because the name also gains +# #530's report at these baselines, which is #289/#516's doing and +# not this rule's subject; the rule claims the union the run measures +# (#452) and the roles are P6's. +# +# Literal, one name. A regex over the trailing word would claim every +# Mc* surname in the corpora, which is the widening the vd rule's own +# _MUST_NOT_MATCH probes already refuse. +name_regex = "^Berg, Jan mc$" +fields = ["_ambiguities", "family", "suffix"] +orders = ["DEFAULT"] diff --git a/tools/differential/expected_since_2.1.0.toml b/tools/differential/expected_since_2.1.0.toml index 3ac9d0cf..2f6962de 100644 --- a/tools/differential/expected_since_2.1.0.toml +++ b/tools/differential/expected_since_2.1.0.toml @@ -2264,14 +2264,40 @@ issue = "fix(#289) a written case contrast decides a bare ambiguous acronym" # Both sit on this rule because the rule is what states which way # each member of a pair goes. # +# 2026-09-19, #531: three names join, all of them lone post-comma +# credentials this rule's own second sentence already describes. +# 'Doe, MA' and 'Doe, MA PhD' are 'Smith, MA's family-word twins and +# 'Doe, Dr. MA' is the same slot behind a title; each reads family +# 'Doe' with the credential run in `suffix` -- 'MA', and 'MA PhD' +# on the one that has a run -- where the baseline read given 'MA'. +# They +# entered the corpora with #531's control rows and are NOT #531's +# doing -- verified byte-identical at cc78c960, the #530 merge +# (2026-09-19). No role joined the field list: the three move +# {given, suffix} and, on two of them, `_ambiguities`. +# +# 2026-09-19, #531 review round: 'Doe, Mr. MA PhD' joins them. It is +# 'Doe, Dr. MA' with a credential run behind the member -- the gate +# reads the segment whole, so the member joins the run rather than +# taking the given slot -- and it moves {given, suffix}, both already +# in the field list. It entered the corpora with that round's control +# rows and is NOT its doing: byte-identical at cc78c960 (measured +# 2026-09-19). +# # Literal-anchored to the measured movers, and ONE SET FOR THE FOUR # 2.x LEDGERS so the rule says the same thing at each of them. The -# 1.4.0 copy differs by exactly one member and holds 'John Prof. MA' +# 1.4.0 copy differs by FOUR members now. It holds 'John Prof. MA' # out: 1.4.0 read a trailing period-marked title as a title itself, # so that name's diff there belongs to fix(#316)'s rule and not to # this one. "One set for all five" was written before that carve-out # existed and was false in these four copies from the day it did -# (2026-09-18). The set is literal because the class is +# (2026-09-18). And it holds the three names above out for a +# different reason: at 1.4.0 their diff is already claimed by +# `fix(comma-family) lone post-comma piece routes to suffix/title, +# not first`, which stands far earlier in that file, so adding them +# there would reach a name without explaining it and would put two +# rules on one diff with nothing to arbitrate between them +# (2026-09-19). The set is literal because the class is # wide (every mixed-case name whose trailing piece is a listed # ambiguous acronym, plus every comma form with two name words before # the comma) and a regex for the class would claim the one-case @@ -2279,7 +2305,7 @@ issue = "fix(#289) a written case contrast decides a bare ambiguous acronym" # given baseline is simply one the reading already agreed with there. # `fields` is per-baseline, the union the run at THAT baseline # measures (#452). -name_regex = "^(?:Davis Royce, Ed|Freiherr von Berg MA|JOHN SMITH, MA|Jack MA|Jack MA\\.|Jack Wei Ma|John Prof\\. MA|John Smith Ma|John Smith, Ed|John Smith, MA|John Smith, Ma|John de Ma|John van der Berg Ma|Smith Jr\\., MA|Smith Jr\\., Ma|Smith, MA|abdul Smith Berg Ma|abdul Smith Jr Ma|abdul Smith Ma|john smith, ma)$" +name_regex = "^(?:Davis Royce, Ed|Doe, Dr\\. MA|Doe, MA|Doe, MA PhD|Doe, Mr\\. MA PhD|Freiherr von Berg MA|JOHN SMITH, MA|Jack MA|Jack MA\\.|Jack Wei Ma|John Prof\\. MA|John Smith Ma|John Smith, Ed|John Smith, MA|John Smith, Ma|John de Ma|John van der Berg Ma|Smith Jr\\., MA|Smith Jr\\., Ma|Smith, MA|abdul Smith Berg Ma|abdul Smith Jr Ma|abdul Smith Ma|john smith, ma)$" fields = ["family", "given", "middle", "suffix", "_ambiguities"] orders = ["DEFAULT"] @@ -2343,6 +2369,14 @@ issue = "fix(#289/#516) the ambiguous credential class reports at slots that wer # run it is, because 'DO' leans credential in a mixed-case name. That # is the one place this arc quiets a report rather than adding one. # +# 2026-09-19, #531 review round: 'Doe, MA Smith' joins, and it is the +# post-comma given slot with a NAME WORD behind the member: no role +# moves at any baseline -- the member is the given name here as it +# always was -- and what is new is only the report that emitter makes. +# Its silent twin 'Doe, Dr. MA Smith' puts a title in that first piece +# instead, which the emitter reads rather than the member, and agrees +# with every baseline on every field, so it is in no ledger at all. +# # `_ambiguities` alone, which keeps `_initials` out of these rows as # always (#484), and the rule cannot absorb a role diff on any of # these names for the same reason. Literal-anchored, one set for all @@ -2356,7 +2390,7 @@ issue = "fix(#289/#516) the ambiguous credential class reports at slots that wer # narrowed out rather than left to file order. The comma-structure # flag it loses is a 2.2-era report in any case, and the 2.2.0 and # 2.3.0 ledgers are where the loss is recorded. -name_regex = "^(?:J\\.A\\. K\\.D\\.|Jack X\\.Y\\.Z\\.|John Smith J\\.u\\.n\\.i\\.o\\.r\\.|John Smith R\\.A\\.I\\.|Royce, Ed|Smith Jr\\., A\\.B\\.|Smith, A\\.B\\.|Smith, Ma)$" +name_regex = "^(?:Doe, MA Smith|J\\.A\\. K\\.D\\.|Jack X\\.Y\\.Z\\.|John Smith J\\.u\\.n\\.i\\.o\\.r\\.|John Smith R\\.A\\.I\\.|Royce, Ed|Smith Jr\\., A\\.B\\.|Smith, A\\.B\\.|Smith, Ma)$" fields = ["_ambiguities"] orders = ["DEFAULT"] @@ -2385,3 +2419,177 @@ issue = "fix(#289/#516) the glued CJK honorific peel reads the case lean" name_regex = "^(?:Dr\\. 김민준씨, MA|Jo김민준씨, DO|Kim김민준씨, MA)$" fields = ["title", "given", "family", "suffix", "_ambiguities"] orders = ["DEFAULT"] + +# --------------------------------------------------------------- +# #531: THE GIVEN PART'S TRAILING SLOT. Four rules -- the thirteen +# the credential reading takes, the four the writing declines and that +# gain a REPORT alone, the one `do` spelling the capitals take from +# P6, and the two caseless comma forms on the radar tier. The decision is +# decisions.md#S2's 2026-09-18 pair; rules.md#S2, #S3, #C1 and #P6 +# carry the statements. +# --------------------------------------------------------------- + +[[change]] +issue = "fix(#531) a credential ending the given part of a family-comma listing reads as a credential" +# After a family comma, a word of the ambiguous credential class +# ending the given part is now read by the same rule the comma-less +# spelling uses for a word ending the name. The comma has already +# named the family and the first word after it is the given name, so +# the words to spare are there by construction and the count says +# nothing: the word reads as the credential unless its WRITING says +# otherwise. rules.md#S2 states it, rules.md#S3 carries the by-shape +# spelling, and decisions.md#S2 records the decision. +# +# Thirteen names, and they are the class read from every side the +# corpora reach it: the bare member ('Doe, John MA', 'Doe, John BA'); +# its one-case spellings, where the lean is inert and the POSITIONAL +# reading is not ('DOE, JOHN MA', 'doe, john ma', 'DOE, MARY JO MA'); +# the slot behind a title ('Doe, Dr. John MA') and behind an initial +# ('Doe, J. MA'); a run of two members, where the credential the +# member joins was already a suffix so `suffix` GROWS rather than +# filling ('Doe, John MA JD', 'Doe, John MA PhD', 'Doe, John PhD MA', +# 'Doe, John MA Jr'); the slot reached past a middle initial, which +# keeps its own middle ('Doe, John Q. MA', 'DOE, MARY JO MA'); and the +# by-shape member, whose lean is always None so the positional reading +# takes it outright ('Doe, John X.Y.Z.'). +# +# Do not read "thirteen role movers" off that list at every baseline: +# `fields` is the union the run at THIS baseline measures (#452), and +# a name here whose roles the baseline already agreed with contributes +# only its report. 'Doe, John BA' is that name at 2.1.0 and 2.2.0, and +# 'Doe, John MA JD' is the other end of the range, gaining TWO reports +# for its two members. +# +# `_ambiguities` is in the fields because every one of the thirteen +# gains the existing SUFFIX_OR_NAME -- no new kind, and the report +# tracks the FORK CONSULTED rather than which way it went, which is +# the rule fix(#289/#516) above already states for the other slots. +# #452's over-declaration check recomputes the union on every run. +# +# Literal-anchored for the reason the two rules above give, and it is +# sharper here: the class is a SLOT, so a regex for it would claim +# every family-comma name whose given part ends in an ambiguous +# acronym -- including the four the WRITING declines, which keep their +# name reading and have the rule below, and 'Doe, John MA Smith', +# where a name word behind the member ends the run so nothing moves +# and nothing reports. _MUST_NOT_MATCH carries both, with the `do` +# spellings P6 keeps and the third-comma-part name #144's restriction +# still owns. +name_regex = "^(?:DOE, JOHN MA|DOE, MARY JO MA|Doe, Dr\\. John MA|Doe, J\\. MA|Doe, John BA|Doe, John MA|Doe, John MA JD|Doe, John MA Jr|Doe, John MA PhD|Doe, John PhD MA|Doe, John Q\\. MA|Doe, John X\\.Y\\.Z\\.|doe, john ma)$" +fields = ["_ambiguities", "middle", "suffix"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#531) a member the writing declines keeps its name reading and reports the fork" +# The other direction of the same fork, and the whole of what these +# four names do: no role moves at any 2.x baseline and the decision +# starts being REPORTED. 'Doe, John Ma', 'Doe, John Ed' and +# 'Doe, Mary Jo Ma' carry a Title-cased member in a mixed-case name, +# which is written the way a name is written, so the member stays a +# middle name; 'Doe, John MA Ma' is a two-member run whose last member +# is Title-cased, so the walk declines there and the run stays middle +# text whole. +# +# Reporting a DECLINED fork is #530's stated rule -- the report tracks +# the fork consulted, not the lean -- and rules.md#A1's "a kind is +# worth adding only if a reader would hesitate too" is the standing +# objection it answers. It is also the only part of #531 that adds a +# report without moving a field, which is why it is a rule of its own +# rather than a widening of the one above: `fields` is +# `_ambiguities` alone, so this rule cannot absorb a role diff on any +# of the four. +# +# At 1.4.0 these same four DO move roles -- v1 read all four members +# as post-nominals -- and that divergence is carried there by +# fix(given-part-trailing-slot), whose comment records that #531 +# changed which names it holds rather than retiring it. +# +# Literal-anchored: the class is the slot's declining half, and a +# regex for it would claim the thirteen movers above. +name_regex = "^(?:Doe, John Ed|Doe, John MA Ma|Doe, John Ma|Doe, Mary Jo Ma)$" +fields = ["_ambiguities"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#531) capitals take the do collision from the family-comma particle attachment" +# 'Doe, John DO', alone, because the word is alone in the class: `do` +# is the one ambiguous credential that is also particle vocabulary, so +# this slot and P6's attachment (rules.md#P6) want the same word. +# Derek's decision, recorded at decisions.md#S2: CAPITALS DECIDE, AND +# THE PARTICLE RULE KEEPS EVERY OTHER SPELLING. An all-caps member in +# a name written in more than one case carries a positive credential +# lean, so the credential reading wins and P6 stands down; every other +# spelling attaches exactly as it did, with P6's own +# `particle-or-given` and no second report. +# +# THE PAIRING IS THE ARGUMENT and the accepted cost is half of it. In +# ONE CASE the rule cannot tell 'NASCIMENTO, EDSON ARANTES DO' from +# 'SMITH, JOHN DO' and reads both as the particle -- right about the +# Brazilian record, wrong about the osteopath. Those two, and +# 'Doe, John Do', 'DOE, JOHN DO', 'doe, john do' and +# 'Nascimento, Edson Arantes do', are _MUST_NOT_MATCH probes: this +# rule has no business claiming a single one of them, and each is a +# name #531 measured to be byte-identical before and after. +# +# `fields` is per-baseline and the two halves of this file's set +# differ by which role the particle came OUT of, which is P6's own +# history: {middle, suffix} at 2.0.0 and 2.1.0, where the +# baseline read middle 'DO' because P6's family-comma attachment had +# not shipped, and {family, suffix} at 2.2.0 and 2.3.0, where it had +# and the baseline reads family 'DO Doe'. +name_regex = "^Doe, John DO$" +fields = ["_ambiguities", "middle", "suffix"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#531) the trailing slot's positional reading reaches a caseless script" +# '田中, 太郎 MA' and '김, 민준 MA': middle 'MA' -> suffix 'MA', one +# report each. A caseless script answers True to the one-case test, so +# the lean is None -- and at this slot a None lean is not inert, +# because the POSITIONAL reading the lean would have overridden is +# itself the credential. "Caseless is inert" holds for the LEAN and +# not for the outcome, which is the sentence tests/v2/cases.py's two +# rows carry. +# +# Both are COMPOSED forms -- a family comma between a CJK family and a +# CJK given name, which no East Asian writing system produces -- so +# both are `tolerated` case rows on the radar tier +# (decisions.md#cjk-comma-demotion), and their diffs print under +# UNCLASSIFIED (radar) rather than blocking. They are classified all +# the same, on fix(#289/#516)'s stated ground: a radar diff an arc +# INTENDED is one a release note may be written from, and an +# unclassified one is not. +# +# Literal-anchored, and the probes are the CJK comma names that do NOT +# move at this slot: '田中 太郎, MA', '毛泽东, MA' and '마틴 킹, MA' put +# their member in the FIRST post-comma piece, which is #289's slot and +# not this one. +name_regex = "^(?:田中, 太郎 MA|김, 민준 MA)$" +fields = ["_ambiguities", "middle", "suffix"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#380) a trailing mc after a family comma is the tussenvoegsel, not a post-nominal" +# THE SAME DECISION AS THE fix(#380) RULE ABOVE, over the other word +# that carries the collision it names. 'Berg, Jan mc' entered the +# corpora with #531's case rows (2026-09-18) as the negative control +# for the `do` carve-out -- 'mc' is unambiguous suffix vocabulary AND +# a never-given particle, so P6's run claims it whatever role assign +# gave it, and #531 left it byte-identical (verified against the tree +# at cc78c960, 2026-09-19). +# +# The diff is against these two baselines only: `mc` joined the +# never-given particles in 2.2 (decisions.md#suffix-acronym- +# collisions) and P6's family-comma attachment shipped in 2.3, so +# 2.2.0 and 2.3.0 already read family 'mc Berg' and carry no rule for +# it. `_ambiguities` is in the fields because the name also gains +# #530's report at these baselines, which is #289/#516's doing and +# not this rule's subject; the rule claims the union the run measures +# (#452) and the roles are P6's. +# +# Literal, one name. A regex over the trailing word would claim every +# Mc* surname in the corpora, which is the widening the vd rule's own +# _MUST_NOT_MATCH probes already refuse. +name_regex = "^Berg, Jan mc$" +fields = ["_ambiguities", "family", "suffix"] +orders = ["DEFAULT"] diff --git a/tools/differential/expected_since_2.2.0.toml b/tools/differential/expected_since_2.2.0.toml index fc14719c..783eada8 100644 --- a/tools/differential/expected_since_2.2.0.toml +++ b/tools/differential/expected_since_2.2.0.toml @@ -886,14 +886,40 @@ issue = "fix(#289) a written case contrast decides a bare ambiguous acronym" # Both sit on this rule because the rule is what states which way # each member of a pair goes. # +# 2026-09-19, #531: three names join, all of them lone post-comma +# credentials this rule's own second sentence already describes. +# 'Doe, MA' and 'Doe, MA PhD' are 'Smith, MA's family-word twins and +# 'Doe, Dr. MA' is the same slot behind a title; each reads family +# 'Doe' with the credential run in `suffix` -- 'MA', and 'MA PhD' +# on the one that has a run -- where the baseline read given 'MA'. +# They +# entered the corpora with #531's control rows and are NOT #531's +# doing -- verified byte-identical at cc78c960, the #530 merge +# (2026-09-19). No role joined the field list: the three move +# {given, suffix} and, on two of them, `_ambiguities`. +# +# 2026-09-19, #531 review round: 'Doe, Mr. MA PhD' joins them. It is +# 'Doe, Dr. MA' with a credential run behind the member -- the gate +# reads the segment whole, so the member joins the run rather than +# taking the given slot -- and it moves {given, suffix}, both already +# in the field list. It entered the corpora with that round's control +# rows and is NOT its doing: byte-identical at cc78c960 (measured +# 2026-09-19). +# # Literal-anchored to the measured movers, and ONE SET FOR THE FOUR # 2.x LEDGERS so the rule says the same thing at each of them. The -# 1.4.0 copy differs by exactly one member and holds 'John Prof. MA' +# 1.4.0 copy differs by FOUR members now. It holds 'John Prof. MA' # out: 1.4.0 read a trailing period-marked title as a title itself, # so that name's diff there belongs to fix(#316)'s rule and not to # this one. "One set for all five" was written before that carve-out # existed and was false in these four copies from the day it did -# (2026-09-18). The set is literal because the class is +# (2026-09-18). And it holds the three names above out for a +# different reason: at 1.4.0 their diff is already claimed by +# `fix(comma-family) lone post-comma piece routes to suffix/title, +# not first`, which stands far earlier in that file, so adding them +# there would reach a name without explaining it and would put two +# rules on one diff with nothing to arbitrate between them +# (2026-09-19). The set is literal because the class is # wide (every mixed-case name whose trailing piece is a listed # ambiguous acronym, plus every comma form with two name words before # the comma) and a regex for the class would claim the one-case @@ -901,7 +927,7 @@ issue = "fix(#289) a written case contrast decides a bare ambiguous acronym" # given baseline is simply one the reading already agreed with there. # `fields` is per-baseline, the union the run at THAT baseline # measures (#452). -name_regex = "^(?:Davis Royce, Ed|Freiherr von Berg MA|JOHN SMITH, MA|Jack MA|Jack MA\\.|Jack Wei Ma|John Prof\\. MA|John Smith Ma|John Smith, Ed|John Smith, MA|John Smith, Ma|John de Ma|John van der Berg Ma|Smith Jr\\., MA|Smith Jr\\., Ma|Smith, MA|abdul Smith Berg Ma|abdul Smith Jr Ma|abdul Smith Ma|john smith, ma)$" +name_regex = "^(?:Davis Royce, Ed|Doe, Dr\\. MA|Doe, MA|Doe, MA PhD|Doe, Mr\\. MA PhD|Freiherr von Berg MA|JOHN SMITH, MA|Jack MA|Jack MA\\.|Jack Wei Ma|John Prof\\. MA|John Smith Ma|John Smith, Ed|John Smith, MA|John Smith, Ma|John de Ma|John van der Berg Ma|Smith Jr\\., MA|Smith Jr\\., Ma|Smith, MA|abdul Smith Berg Ma|abdul Smith Jr Ma|abdul Smith Ma|john smith, ma)$" fields = ["family", "given", "middle", "suffix", "_ambiguities"] orders = ["DEFAULT"] @@ -965,12 +991,20 @@ issue = "fix(#289/#516) the ambiguous credential class reports at slots that wer # run it is, because 'DO' leans credential in a mixed-case name. That # is the one place this arc quiets a report rather than adding one. # +# 2026-09-19, #531 review round: 'Doe, MA Smith' joins, and it is the +# post-comma given slot with a NAME WORD behind the member: no role +# moves at any baseline -- the member is the given name here as it +# always was -- and what is new is only the report that emitter makes. +# Its silent twin 'Doe, Dr. MA Smith' puts a title in that first piece +# instead, which the emitter reads rather than the member, and agrees +# with every baseline on every field, so it is in no ledger at all. +# # `_ambiguities` alone, which keeps `_initials` out of these rows as # always (#484), and the rule cannot absorb a role diff on any of # these names for the same reason. Literal-anchored, one set for all # the 2.x ledgers; the 1.4.0 ledger has no twin, `_ambiguities` being # a v2 surface that cannot enter a diff below baseline 2.0. -name_regex = "^(?:J\\.A\\. K\\.D\\.|Jack X\\.Y\\.Z\\.|John Smith J\\.u\\.n\\.i\\.o\\.r\\.|John Smith R\\.A\\.I\\.|Royce, Ed|Smith Jr\\., A\\.B\\.|Smith, A\\.B\\.|Smith, Ma|Steven Hardman, MD, DO, DDS)$" +name_regex = "^(?:Doe, MA Smith|J\\.A\\. K\\.D\\.|Jack X\\.Y\\.Z\\.|John Smith J\\.u\\.n\\.i\\.o\\.r\\.|John Smith R\\.A\\.I\\.|Royce, Ed|Smith Jr\\., A\\.B\\.|Smith, A\\.B\\.|Smith, Ma|Steven Hardman, MD, DO, DDS)$" fields = ["_ambiguities"] orders = ["DEFAULT"] @@ -999,3 +1033,151 @@ issue = "fix(#289/#516) the glued CJK honorific peel reads the case lean" name_regex = "^(?:Dr\\. 김민준씨, MA|Jo김민준씨, DO|Kim김민준씨, MA)$" fields = ["title", "given", "family", "suffix", "_ambiguities"] orders = ["DEFAULT"] + +# --------------------------------------------------------------- +# #531: THE GIVEN PART'S TRAILING SLOT. Four rules -- the thirteen +# the credential reading takes, the four the writing declines and that +# gain a REPORT alone, the one `do` spelling the capitals take from +# P6, and the two caseless comma forms on the radar tier. The decision is +# decisions.md#S2's 2026-09-18 pair; rules.md#S2, #S3, #C1 and #P6 +# carry the statements. +# --------------------------------------------------------------- + +[[change]] +issue = "fix(#531) a credential ending the given part of a family-comma listing reads as a credential" +# After a family comma, a word of the ambiguous credential class +# ending the given part is now read by the same rule the comma-less +# spelling uses for a word ending the name. The comma has already +# named the family and the first word after it is the given name, so +# the words to spare are there by construction and the count says +# nothing: the word reads as the credential unless its WRITING says +# otherwise. rules.md#S2 states it, rules.md#S3 carries the by-shape +# spelling, and decisions.md#S2 records the decision. +# +# Thirteen names, and they are the class read from every side the +# corpora reach it: the bare member ('Doe, John MA', 'Doe, John BA'); +# its one-case spellings, where the lean is inert and the POSITIONAL +# reading is not ('DOE, JOHN MA', 'doe, john ma', 'DOE, MARY JO MA'); +# the slot behind a title ('Doe, Dr. John MA') and behind an initial +# ('Doe, J. MA'); a run of two members, where the credential the +# member joins was already a suffix so `suffix` GROWS rather than +# filling ('Doe, John MA JD', 'Doe, John MA PhD', 'Doe, John PhD MA', +# 'Doe, John MA Jr'); the slot reached past a middle initial, which +# keeps its own middle ('Doe, John Q. MA', 'DOE, MARY JO MA'); and the +# by-shape member, whose lean is always None so the positional reading +# takes it outright ('Doe, John X.Y.Z.'). +# +# Do not read "thirteen role movers" off that list at every baseline: +# `fields` is the union the run at THIS baseline measures (#452), and +# a name here whose roles the baseline already agreed with contributes +# only its report. 'Doe, John BA' is that name at 2.1.0 and 2.2.0, and +# 'Doe, John MA JD' is the other end of the range, gaining TWO reports +# for its two members. +# +# `_ambiguities` is in the fields because every one of the thirteen +# gains the existing SUFFIX_OR_NAME -- no new kind, and the report +# tracks the FORK CONSULTED rather than which way it went, which is +# the rule fix(#289/#516) above already states for the other slots. +# #452's over-declaration check recomputes the union on every run. +# +# Literal-anchored for the reason the two rules above give, and it is +# sharper here: the class is a SLOT, so a regex for it would claim +# every family-comma name whose given part ends in an ambiguous +# acronym -- including the four the WRITING declines, which keep their +# name reading and have the rule below, and 'Doe, John MA Smith', +# where a name word behind the member ends the run so nothing moves +# and nothing reports. _MUST_NOT_MATCH carries both, with the `do` +# spellings P6 keeps and the third-comma-part name #144's restriction +# still owns. +name_regex = "^(?:DOE, JOHN MA|DOE, MARY JO MA|Doe, Dr\\. John MA|Doe, J\\. MA|Doe, John BA|Doe, John MA|Doe, John MA JD|Doe, John MA Jr|Doe, John MA PhD|Doe, John PhD MA|Doe, John Q\\. MA|Doe, John X\\.Y\\.Z\\.|doe, john ma)$" +fields = ["_ambiguities", "middle", "suffix"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#531) a member the writing declines keeps its name reading and reports the fork" +# The other direction of the same fork, and the whole of what these +# four names do: no role moves at any 2.x baseline and the decision +# starts being REPORTED. 'Doe, John Ma', 'Doe, John Ed' and +# 'Doe, Mary Jo Ma' carry a Title-cased member in a mixed-case name, +# which is written the way a name is written, so the member stays a +# middle name; 'Doe, John MA Ma' is a two-member run whose last member +# is Title-cased, so the walk declines there and the run stays middle +# text whole. +# +# Reporting a DECLINED fork is #530's stated rule -- the report tracks +# the fork consulted, not the lean -- and rules.md#A1's "a kind is +# worth adding only if a reader would hesitate too" is the standing +# objection it answers. It is also the only part of #531 that adds a +# report without moving a field, which is why it is a rule of its own +# rather than a widening of the one above: `fields` is +# `_ambiguities` alone, so this rule cannot absorb a role diff on any +# of the four. +# +# At 1.4.0 these same four DO move roles -- v1 read all four members +# as post-nominals -- and that divergence is carried there by +# fix(given-part-trailing-slot), whose comment records that #531 +# changed which names it holds rather than retiring it. +# +# Literal-anchored: the class is the slot's declining half, and a +# regex for it would claim the thirteen movers above. +name_regex = "^(?:Doe, John Ed|Doe, John MA Ma|Doe, John Ma|Doe, Mary Jo Ma)$" +fields = ["_ambiguities"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#531) capitals take the do collision from the family-comma particle attachment" +# 'Doe, John DO', alone, because the word is alone in the class: `do` +# is the one ambiguous credential that is also particle vocabulary, so +# this slot and P6's attachment (rules.md#P6) want the same word. +# Derek's decision, recorded at decisions.md#S2: CAPITALS DECIDE, AND +# THE PARTICLE RULE KEEPS EVERY OTHER SPELLING. An all-caps member in +# a name written in more than one case carries a positive credential +# lean, so the credential reading wins and P6 stands down; every other +# spelling attaches exactly as it did, with P6's own +# `particle-or-given` and no second report. +# +# THE PAIRING IS THE ARGUMENT and the accepted cost is half of it. In +# ONE CASE the rule cannot tell 'NASCIMENTO, EDSON ARANTES DO' from +# 'SMITH, JOHN DO' and reads both as the particle -- right about the +# Brazilian record, wrong about the osteopath. Those two, and +# 'Doe, John Do', 'DOE, JOHN DO', 'doe, john do' and +# 'Nascimento, Edson Arantes do', are _MUST_NOT_MATCH probes: this +# rule has no business claiming a single one of them, and each is a +# name #531 measured to be byte-identical before and after. +# +# `fields` is per-baseline and the two halves of this file's set +# differ by which role the particle came OUT of, which is P6's own +# history: {middle, suffix} at 2.0.0 and 2.1.0, where the +# baseline read middle 'DO' because P6's family-comma attachment had +# not shipped, and {family, suffix} at 2.2.0 and 2.3.0, where it had +# and the baseline reads family 'DO Doe'. +name_regex = "^Doe, John DO$" +fields = ["_ambiguities", "family", "suffix"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#531) the trailing slot's positional reading reaches a caseless script" +# '田中, 太郎 MA' and '김, 민준 MA': middle 'MA' -> suffix 'MA', one +# report each. A caseless script answers True to the one-case test, so +# the lean is None -- and at this slot a None lean is not inert, +# because the POSITIONAL reading the lean would have overridden is +# itself the credential. "Caseless is inert" holds for the LEAN and +# not for the outcome, which is the sentence tests/v2/cases.py's two +# rows carry. +# +# Both are COMPOSED forms -- a family comma between a CJK family and a +# CJK given name, which no East Asian writing system produces -- so +# both are `tolerated` case rows on the radar tier +# (decisions.md#cjk-comma-demotion), and their diffs print under +# UNCLASSIFIED (radar) rather than blocking. They are classified all +# the same, on fix(#289/#516)'s stated ground: a radar diff an arc +# INTENDED is one a release note may be written from, and an +# unclassified one is not. +# +# Literal-anchored, and the probes are the CJK comma names that do NOT +# move at this slot: '田中 太郎, MA', '毛泽东, MA' and '마틴 킹, MA' put +# their member in the FIRST post-comma piece, which is #289's slot and +# not this one. +name_regex = "^(?:田中, 太郎 MA|김, 민준 MA)$" +fields = ["_ambiguities", "middle", "suffix"] +orders = ["DEFAULT"] diff --git a/tools/differential/expected_since_2.3.0.toml b/tools/differential/expected_since_2.3.0.toml index 8ae60026..73e6f21e 100644 --- a/tools/differential/expected_since_2.3.0.toml +++ b/tools/differential/expected_since_2.3.0.toml @@ -218,14 +218,40 @@ issue = "fix(#289) a written case contrast decides a bare ambiguous acronym" # Both sit on this rule because the rule is what states which way # each member of a pair goes. # +# 2026-09-19, #531: three names join, all of them lone post-comma +# credentials this rule's own second sentence already describes. +# 'Doe, MA' and 'Doe, MA PhD' are 'Smith, MA's family-word twins and +# 'Doe, Dr. MA' is the same slot behind a title; each reads family +# 'Doe' with the credential run in `suffix` -- 'MA', and 'MA PhD' +# on the one that has a run -- where the baseline read given 'MA'. +# They +# entered the corpora with #531's control rows and are NOT #531's +# doing -- verified byte-identical at cc78c960, the #530 merge +# (2026-09-19). No role joined the field list: the three move +# {given, suffix} and, on two of them, `_ambiguities`. +# +# 2026-09-19, #531 review round: 'Doe, Mr. MA PhD' joins them. It is +# 'Doe, Dr. MA' with a credential run behind the member -- the gate +# reads the segment whole, so the member joins the run rather than +# taking the given slot -- and it moves {given, suffix}, both already +# in the field list. It entered the corpora with that round's control +# rows and is NOT its doing: byte-identical at cc78c960 (measured +# 2026-09-19). +# # Literal-anchored to the measured movers, and ONE SET FOR THE FOUR # 2.x LEDGERS so the rule says the same thing at each of them. The -# 1.4.0 copy differs by exactly one member and holds 'John Prof. MA' +# 1.4.0 copy differs by FOUR members now. It holds 'John Prof. MA' # out: 1.4.0 read a trailing period-marked title as a title itself, # so that name's diff there belongs to fix(#316)'s rule and not to # this one. "One set for all five" was written before that carve-out # existed and was false in these four copies from the day it did -# (2026-09-18). The set is literal because the class is +# (2026-09-18). And it holds the three names above out for a +# different reason: at 1.4.0 their diff is already claimed by +# `fix(comma-family) lone post-comma piece routes to suffix/title, +# not first`, which stands far earlier in that file, so adding them +# there would reach a name without explaining it and would put two +# rules on one diff with nothing to arbitrate between them +# (2026-09-19). The set is literal because the class is # wide (every mixed-case name whose trailing piece is a listed # ambiguous acronym, plus every comma form with two name words before # the comma) and a regex for the class would claim the one-case @@ -233,7 +259,7 @@ issue = "fix(#289) a written case contrast decides a bare ambiguous acronym" # given baseline is simply one the reading already agreed with there. # `fields` is per-baseline, the union the run at THAT baseline # measures (#452). -name_regex = "^(?:Davis Royce, Ed|Freiherr von Berg MA|JOHN SMITH, MA|Jack MA|Jack MA\\.|Jack Wei Ma|John Prof\\. MA|John Smith Ma|John Smith, Ed|John Smith, MA|John Smith, Ma|John de Ma|John van der Berg Ma|Smith Jr\\., MA|Smith Jr\\., Ma|Smith, MA|abdul Smith Berg Ma|abdul Smith Jr Ma|abdul Smith Ma|john smith, ma)$" +name_regex = "^(?:Davis Royce, Ed|Doe, Dr\\. MA|Doe, MA|Doe, MA PhD|Doe, Mr\\. MA PhD|Freiherr von Berg MA|JOHN SMITH, MA|Jack MA|Jack MA\\.|Jack Wei Ma|John Prof\\. MA|John Smith Ma|John Smith, Ed|John Smith, MA|John Smith, Ma|John de Ma|John van der Berg Ma|Smith Jr\\., MA|Smith Jr\\., Ma|Smith, MA|abdul Smith Berg Ma|abdul Smith Jr Ma|abdul Smith Ma|john smith, ma)$" fields = ["family", "given", "middle", "suffix", "_ambiguities"] orders = ["DEFAULT"] @@ -297,12 +323,20 @@ issue = "fix(#289/#516) the ambiguous credential class reports at slots that wer # run it is, because 'DO' leans credential in a mixed-case name. That # is the one place this arc quiets a report rather than adding one. # +# 2026-09-19, #531 review round: 'Doe, MA Smith' joins, and it is the +# post-comma given slot with a NAME WORD behind the member: no role +# moves at any baseline -- the member is the given name here as it +# always was -- and what is new is only the report that emitter makes. +# Its silent twin 'Doe, Dr. MA Smith' puts a title in that first piece +# instead, which the emitter reads rather than the member, and agrees +# with every baseline on every field, so it is in no ledger at all. +# # `_ambiguities` alone, which keeps `_initials` out of these rows as # always (#484), and the rule cannot absorb a role diff on any of # these names for the same reason. Literal-anchored, one set for all # the 2.x ledgers; the 1.4.0 ledger has no twin, `_ambiguities` being # a v2 surface that cannot enter a diff below baseline 2.0. -name_regex = "^(?:J\\.A\\. K\\.D\\.|Jack X\\.Y\\.Z\\.|John Smith J\\.u\\.n\\.i\\.o\\.r\\.|John Smith R\\.A\\.I\\.|Royce, Ed|Smith Jr\\., A\\.B\\.|Smith, A\\.B\\.|Smith, Ma|Steven Hardman, MD, DO, DDS)$" +name_regex = "^(?:Doe, MA Smith|J\\.A\\. K\\.D\\.|Jack X\\.Y\\.Z\\.|John Smith J\\.u\\.n\\.i\\.o\\.r\\.|John Smith R\\.A\\.I\\.|Royce, Ed|Smith Jr\\., A\\.B\\.|Smith, A\\.B\\.|Smith, Ma|Steven Hardman, MD, DO, DDS)$" fields = ["_ambiguities"] orders = ["DEFAULT"] @@ -331,3 +365,151 @@ issue = "fix(#289/#516) the glued CJK honorific peel reads the case lean" name_regex = "^(?:Dr\\. 김민준씨, MA|Jo김민준씨, DO|Kim김민준씨, MA)$" fields = ["title", "given", "family", "suffix", "_ambiguities"] orders = ["DEFAULT"] + +# --------------------------------------------------------------- +# #531: THE GIVEN PART'S TRAILING SLOT. Four rules -- the thirteen +# the credential reading takes, the four the writing declines and that +# gain a REPORT alone, the one `do` spelling the capitals take from +# P6, and the two caseless comma forms on the radar tier. The decision is +# decisions.md#S2's 2026-09-18 pair; rules.md#S2, #S3, #C1 and #P6 +# carry the statements. +# --------------------------------------------------------------- + +[[change]] +issue = "fix(#531) a credential ending the given part of a family-comma listing reads as a credential" +# After a family comma, a word of the ambiguous credential class +# ending the given part is now read by the same rule the comma-less +# spelling uses for a word ending the name. The comma has already +# named the family and the first word after it is the given name, so +# the words to spare are there by construction and the count says +# nothing: the word reads as the credential unless its WRITING says +# otherwise. rules.md#S2 states it, rules.md#S3 carries the by-shape +# spelling, and decisions.md#S2 records the decision. +# +# Thirteen names, and they are the class read from every side the +# corpora reach it: the bare member ('Doe, John MA', 'Doe, John BA'); +# its one-case spellings, where the lean is inert and the POSITIONAL +# reading is not ('DOE, JOHN MA', 'doe, john ma', 'DOE, MARY JO MA'); +# the slot behind a title ('Doe, Dr. John MA') and behind an initial +# ('Doe, J. MA'); a run of two members, where the credential the +# member joins was already a suffix so `suffix` GROWS rather than +# filling ('Doe, John MA JD', 'Doe, John MA PhD', 'Doe, John PhD MA', +# 'Doe, John MA Jr'); the slot reached past a middle initial, which +# keeps its own middle ('Doe, John Q. MA', 'DOE, MARY JO MA'); and the +# by-shape member, whose lean is always None so the positional reading +# takes it outright ('Doe, John X.Y.Z.'). +# +# Do not read "thirteen role movers" off that list at every baseline: +# `fields` is the union the run at THIS baseline measures (#452), and +# a name here whose roles the baseline already agreed with contributes +# only its report. 'Doe, John BA' is that name at 2.1.0 and 2.2.0, and +# 'Doe, John MA JD' is the other end of the range, gaining TWO reports +# for its two members. +# +# `_ambiguities` is in the fields because every one of the thirteen +# gains the existing SUFFIX_OR_NAME -- no new kind, and the report +# tracks the FORK CONSULTED rather than which way it went, which is +# the rule fix(#289/#516) above already states for the other slots. +# #452's over-declaration check recomputes the union on every run. +# +# Literal-anchored for the reason the two rules above give, and it is +# sharper here: the class is a SLOT, so a regex for it would claim +# every family-comma name whose given part ends in an ambiguous +# acronym -- including the four the WRITING declines, which keep their +# name reading and have the rule below, and 'Doe, John MA Smith', +# where a name word behind the member ends the run so nothing moves +# and nothing reports. _MUST_NOT_MATCH carries both, with the `do` +# spellings P6 keeps and the third-comma-part name #144's restriction +# still owns. +name_regex = "^(?:DOE, JOHN MA|DOE, MARY JO MA|Doe, Dr\\. John MA|Doe, J\\. MA|Doe, John BA|Doe, John MA|Doe, John MA JD|Doe, John MA Jr|Doe, John MA PhD|Doe, John PhD MA|Doe, John Q\\. MA|Doe, John X\\.Y\\.Z\\.|doe, john ma)$" +fields = ["_ambiguities", "middle", "suffix"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#531) a member the writing declines keeps its name reading and reports the fork" +# The other direction of the same fork, and the whole of what these +# four names do: no role moves at any 2.x baseline and the decision +# starts being REPORTED. 'Doe, John Ma', 'Doe, John Ed' and +# 'Doe, Mary Jo Ma' carry a Title-cased member in a mixed-case name, +# which is written the way a name is written, so the member stays a +# middle name; 'Doe, John MA Ma' is a two-member run whose last member +# is Title-cased, so the walk declines there and the run stays middle +# text whole. +# +# Reporting a DECLINED fork is #530's stated rule -- the report tracks +# the fork consulted, not the lean -- and rules.md#A1's "a kind is +# worth adding only if a reader would hesitate too" is the standing +# objection it answers. It is also the only part of #531 that adds a +# report without moving a field, which is why it is a rule of its own +# rather than a widening of the one above: `fields` is +# `_ambiguities` alone, so this rule cannot absorb a role diff on any +# of the four. +# +# At 1.4.0 these same four DO move roles -- v1 read all four members +# as post-nominals -- and that divergence is carried there by +# fix(given-part-trailing-slot), whose comment records that #531 +# changed which names it holds rather than retiring it. +# +# Literal-anchored: the class is the slot's declining half, and a +# regex for it would claim the thirteen movers above. +name_regex = "^(?:Doe, John Ed|Doe, John MA Ma|Doe, John Ma|Doe, Mary Jo Ma)$" +fields = ["_ambiguities"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#531) capitals take the do collision from the family-comma particle attachment" +# 'Doe, John DO', alone, because the word is alone in the class: `do` +# is the one ambiguous credential that is also particle vocabulary, so +# this slot and P6's attachment (rules.md#P6) want the same word. +# Derek's decision, recorded at decisions.md#S2: CAPITALS DECIDE, AND +# THE PARTICLE RULE KEEPS EVERY OTHER SPELLING. An all-caps member in +# a name written in more than one case carries a positive credential +# lean, so the credential reading wins and P6 stands down; every other +# spelling attaches exactly as it did, with P6's own +# `particle-or-given` and no second report. +# +# THE PAIRING IS THE ARGUMENT and the accepted cost is half of it. In +# ONE CASE the rule cannot tell 'NASCIMENTO, EDSON ARANTES DO' from +# 'SMITH, JOHN DO' and reads both as the particle -- right about the +# Brazilian record, wrong about the osteopath. Those two, and +# 'Doe, John Do', 'DOE, JOHN DO', 'doe, john do' and +# 'Nascimento, Edson Arantes do', are _MUST_NOT_MATCH probes: this +# rule has no business claiming a single one of them, and each is a +# name #531 measured to be byte-identical before and after. +# +# `fields` is per-baseline and the two halves of this file's set +# differ by which role the particle came OUT of, which is P6's own +# history: {middle, suffix} at 2.0.0 and 2.1.0, where the +# baseline read middle 'DO' because P6's family-comma attachment had +# not shipped, and {family, suffix} at 2.2.0 and 2.3.0, where it had +# and the baseline reads family 'DO Doe'. +name_regex = "^Doe, John DO$" +fields = ["_ambiguities", "family", "suffix"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#531) the trailing slot's positional reading reaches a caseless script" +# '田中, 太郎 MA' and '김, 민준 MA': middle 'MA' -> suffix 'MA', one +# report each. A caseless script answers True to the one-case test, so +# the lean is None -- and at this slot a None lean is not inert, +# because the POSITIONAL reading the lean would have overridden is +# itself the credential. "Caseless is inert" holds for the LEAN and +# not for the outcome, which is the sentence tests/v2/cases.py's two +# rows carry. +# +# Both are COMPOSED forms -- a family comma between a CJK family and a +# CJK given name, which no East Asian writing system produces -- so +# both are `tolerated` case rows on the radar tier +# (decisions.md#cjk-comma-demotion), and their diffs print under +# UNCLASSIFIED (radar) rather than blocking. They are classified all +# the same, on fix(#289/#516)'s stated ground: a radar diff an arc +# INTENDED is one a release note may be written from, and an +# unclassified one is not. +# +# Literal-anchored, and the probes are the CJK comma names that do NOT +# move at this slot: '田中 太郎, MA', '毛泽东, MA' and '마틴 킹, MA' put +# their member in the FIRST post-comma piece, which is #289's slot and +# not this one. +name_regex = "^(?:田中, 太郎 MA|김, 민준 MA)$" +fields = ["_ambiguities", "middle", "suffix"] +orders = ["DEFAULT"]