From 1b37116aed35806e1a39843b5115d86618bc752f Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Sat, 19 Sep 2026 16:22:13 -0700 Subject: [PATCH 1/5] fix(#533): read a credential ending a maiden clause as a credential A maiden marker took every word after it up to the first suffix WORD or the trailing numeral, and the ambiguous credential class was not among the words that stopped it -- so the last trailing position in the library where a member of that class was read in silence was the end of a maiden clause. `parse("Jane Doe nee Smith MA")` gave maiden 'Smith MA' and reported nothing, where `John Smith MA` gives suffix 'MA' and reports. It was order-sensitive besides: `nee Smith MA PhD` gave maiden 'Smith MA' while `nee Smith PhD MA` gave maiden 'Smith', so whether the word was read at all depended on which side of the unambiguous credential the writer put it. The words a marker takes now end where a trailing credential begins, where the rule that reads the name left standing reads the word as one -- both as the name is written and as the take would leave it, which is the same double question the trailing roman numeral is asked and for the same reason: the count of words to spare includes the very words the marker removes. Which rule does the reading depends on where the clause stands. With no comma it is the trailing peel; after a family comma it is the reading the end of the given part takes, where the comma has already settled the count and the writing decides alone; before that comma, and past a second one, no trailing rule reads those words at all and the clause keeps them. A member that is the ONLY word after the marker stays the maiden name, whatever its writing says: the marker announces a name, and the rule gives a word up only where a maiden name is left standing. That is a clamp rather than a veto -- `Doe, J. nee MA ba` keeps maiden 'MA' and reads suffix 'ba'. Either reading is reported, on a channel of the walk's own: group suppresses its particle-chain emitter after a family comma, and a credential ending a clause is not that fork. One walk, one peel: the numeral reading stops going through `trailing_start` and calls the pair it wrapped, so the acronym half needs no second peel and a maiden name that does not trigger got one frame cheaper. #531's reading moves into `_pieces.credential_at_the_given_slot`, shared by both callers rather than spelled twice; measured over 12,630 parses in two processes, the refactor is byte-identical, and it costs one frame on a family-comma name carrying a class member. Over every corpus name under six policies the change moves 0 roles and adds 1 report. The agreement sweep it ships -- 2016 pairs of a name with a clause against the same name without one -- went from 984 disagreements to 0 outside the one-case-head class, whose 114 members are the accepted cost of `own_words` stopping at the marker. The rows this adds put three names inside both `fix(#274)` and `fix(#379)` in the 1.4.0 ledger, whose fields nest, so file order alone would pick the winner and test_every_order_decided_contest_is_declared goes red. The earlier rule carries a precedes_narrower block saying why it wins, and the negative control beside it records the contest either way; the remaining ledger work is its own commit. Review fixes, comments and tests only: `listed_lean`'s frame note names its third caller, SUFFIX_OR_NAME's silent positions count four, both maiden guards carry the negative controls they were measured against, the shared predicate costs one frame PER MEMBER, the `one_case` denominator is the recipe's own (10,740), and the span property test compares spans -- with a count that keeps it from being vacuous. Co-Authored-By: Claude Fable 5.1 --- docs/design/rules.md | 20 +- nameparser/_pipeline/_assign.py | 28 +- nameparser/_pipeline/_group.py | 268 +++++++- nameparser/_pipeline/_pieces.py | 55 +- nameparser/_types.py | 61 +- tests/v2/cases.py | 617 +++++++++++++++++- tests/v2/pipeline/test_assign.py | 31 +- tests/v2/pipeline/test_group.py | 164 ++++- tests/v2/test_facade_cases.py | 14 + tests/v2/test_ledger_guards.py | 119 +++- tests/v2/test_properties.py | 156 +++++ tools/differential/corpus_cjk_tolerated.jsonl | 1 + tools/differential/corpus_rules.jsonl | 6 + tools/differential/corpus_shapes.jsonl | 39 ++ tools/differential/expected_since_1.4.0.toml | 21 + 15 files changed, 1496 insertions(+), 104 deletions(-) diff --git a/docs/design/rules.md b/docs/design/rules.md index c9f7c7d8..5aff882a 100644 --- a/docs/design/rules.md +++ b/docs/design/rules.md @@ -1112,6 +1112,7 @@ S3. Rationale: credentials are often written run together with "John Smith Xyz." → family="Xyz." · boundary "John Smith 1.4" → family="1.4" · boundary "Doe, John X.Y.Z." → suffix="X.Y.Z." + "Jane Doe nee Smith X.Y.Z." → suffix="X.Y.Z." Accepted: the initialless-script clause carries no example line of its own. Every input that exercises it composes a script that writes no abbreviations with a period that only a Latin @@ -1224,7 +1225,11 @@ M2. Rationale: a maiden marker announces that what follows it is the word takes the words after it — up to any suffix word, or the trailing roman numeral assign reads as the suffix (S2), both as written and as the take would leave the name, the word before - the numeral being then the word before the marker — as the + the numeral being then the word before the marker, or a + trailing word of the ambiguous credential class (S2), asked + that same double way and stopping the take only where the rule + reading the name left standing reads the word as the + credential, and never the first word after the marker — as the maiden name, and the marker itself is dropped. A marker with nothing after it, or nothing before it, is just a word. @@ -1249,6 +1254,12 @@ M2. Rationale: a maiden marker announces that what follows it is the "Jane Smith née V" → suffix="V" "J. née Jones Smith V" → maiden="Jones Smith V" · boundary "Jane née Jones J. V" → maiden="Jones J. V" · boundary + "Jane Doe nee Smith MA" → maiden="Smith" + "Jane Doe nee Smith MA" → suffix="MA" + "Jane Doe nee Smith Ma" → maiden="Smith Ma" · boundary + "Jane Doe nee MA" → maiden="MA" · boundary + "Jane Doe nee MA Smith" → maiden="MA Smith" · boundary + "John née Jones Smith MA" → maiden="Jones Smith" "Jones née" → family="née" · boundary "née Jones" → family="Jones" · boundary "Jane van der Berg née Jones" → maiden="Jones" @@ -1272,9 +1283,10 @@ M2. Rationale: a maiden marker announces that what follows it is the would have bound the two into one name word (P3); the connective then builds a family name out of what is left. "Jane née Jr y Jones" → maiden="" - Accepted: a bare acronym the peel would take with words to spare - is maiden text all the same — the count it needs includes the - very words the marker removes, so the reading is left to assign. + Accepted: a bare acronym the reading declines is maiden text all + the same — the writing decides this one (S2), and the count such + a reading needs is taken over the name the take would leave + rather than over the words as they stand. "John née Jones Smith Ma" → maiden="Jones Smith Ma" history: decisions.md#M2 · interacts: P2, P3, P5, R2, M1, S2, H1, H5 · implemented: nameparser/_pipeline/_group.py diff --git a/nameparser/_pipeline/_assign.py b/nameparser/_pipeline/_assign.py index 4320b6c1..f77d13a6 100644 --- a/nameparser/_pipeline/_assign.py +++ b/nameparser/_pipeline/_assign.py @@ -72,7 +72,8 @@ effective_script, is_suffix_lenient, resolve_script_set, ) from nameparser._pipeline._pieces import ( - is_suffix_piece, leading_titles, listed_lean, peel_walk, + credential_at_the_given_slot, + is_suffix_piece, leading_titles, peel_walk, segment_suffix_reading, tail_reading, trailing_titles, ) from nameparser._pipeline._state import ( @@ -805,7 +806,6 @@ def reads_as_a_suffix(m: int, titled: tuple[int, ...]) -> bool: # reach, and the member is an ordinary middle # name read in silence. if m >= trailing_floor(m, titled): - lean = listed_lean(tok, state.one_case) # A member that is ALSO particle # vocabulary reads as the credential only # on a POSITIVE credential lean: P6's @@ -815,9 +815,27 @@ def reads_as_a_suffix(m: int, titled: tuple[int, ...]) -> bool: # would take it -- the wrong answer there, # not merely a stray report # (decisions.md#S2, 2026-09-18). - if lean == "credential" or ( - lean is None - and "particle" not in tok.tags): + # + # A FUNCTION since #533, not two conditions + # written to match: the maiden walk's + # second check asks this same question of + # the name a take would leave, and the + # drift would have been silent -- each + # site's own tests would have gone on + # passing (mechanisms.md + # #ONE-PREDICATE-PER-QUESTION). It costs + # one frame PER MEMBER asked at this slot, + # not one frame per name -- against + # 2f57ff21, 'Doe, John MA' is 310 -> 311 + # and 'Doe, John MA Ma MA', which asks + # four times, is 439 -> 443. A name with + # no member here never reaches it and pays + # nothing ('Smith, John' 206, 'MA JD' 185, + # both unchanged). Measured 2026-09-19 per + # `Parser.parse`; Derek took that trade + # deliberately. + if credential_at_the_given_slot( + tok, state.one_case): return True prev = previous_kept(m, titled) # trailing piece of a two-part name is unambiguously diff --git a/nameparser/_pipeline/_group.py b/nameparser/_pipeline/_group.py index 8d2ce4c0..7bdf1ddc 100644 --- a/nameparser/_pipeline/_group.py +++ b/nameparser/_pipeline/_group.py @@ -44,12 +44,14 @@ from nameparser._lexicon import _run_addresses_by_given from nameparser._pipeline._pieces import ( + credential_at_the_given_slot, is_leading_title, is_suffix_piece, is_title_piece, - leading_titles, peel_walk, tail_reading, trailing_start, + leading_titles, peel_trailing, peel_walk, tail_reading, + trailing_start, ) from nameparser._pipeline._state import ( - ParseState, PendingAmbiguity, Structure, WorkToken, - _AMBIGUOUS_CREDENTIAL_TAGS, + AMBIGUOUS_ACRONYM_TAG, ParseState, PendingAmbiguity, Structure, + WorkToken, _AMBIGUOUS_CREDENTIAL_TAGS, ) from nameparser._pipeline._vocab import D, PH from nameparser._pipeline._vocab import delimiter_cores @@ -68,6 +70,23 @@ MaidenTake = tuple[Piece, list[Piece]] +class TailReader(IntEnum): + """Which rule reads the words the maiden walk would leave standing + at the end of this segment -- the reader the acronym fork's second + check has to ask, since a stop is only right where that reader + takes the word (rules.md#M2, #533). + + NONE is a statement and not a default: before a family comma the + words are the family the comma already named, and a tail segment + is read as credentials whole, so no trailing rule is consulted + there and the clause keeps what it has -- a stop would hand a word + to `family` rather than to `suffix`.""" + + NONE = 0 # FAMILY_COMMA segment 0, and every tail segment + TRAILING = 1 # the S2 peel: NO_COMMA, SUFFIX_COMMA segment 0 + GIVEN_SLOT = 2 # #531's reading: FAMILY_COMMA segment 1 + + class BoundJoin(IntEnum): """v1 _join_bound_first_name's reserve_last, as the three states it actually has. IntEnum: the value IS the number of name pieces @@ -190,7 +209,10 @@ def _maiden_take(pieces: Sequence[Sequence[int]], ptags: Sequence[Set[str]], tokens: Sequence[WorkToken], cores: Set[str], - one_case: bool | None) -> tuple[list[int], list[int]] | None: + one_case: bool | None, + reader: TailReader, + ambiguities: list[PendingAmbiguity], + ) -> tuple[list[int], list[int]] | None: """The piece indices the marker pass removes, split the way MaidenTake declares them: the MARKER's pieces (one, or several for a phrase entry like 'z domu') and the maiden name's. None when the @@ -208,6 +230,15 @@ def _maiden_take(pieces: Sequence[Sequence[int]], Smith' takes only 'Jones'. The one reading the order costs; M2's Accepted row and decisions.md#M2 (#420) record it. + "Up to any trailing suffix" also means up to a trailing + CREDENTIAL since #533, where the rule that reads the name left + standing reads the word as one -- the TRAILING peel with no comma, + the given part's own slot after a family comma, and nobody before + that comma or past a second one, which is what `reader` says. A + member standing alone after the marker is never given up: the + marker announces a name, and the rule gives a word up only where + a maiden name is left standing. + A tail segment's delimiter cores (`cores`, empty elsewhere) are structure, not words, and group() drops them after the pass -- before the pass moved ahead of the joins it dropped them first. @@ -231,39 +262,57 @@ def _maiden_take(pieces: Sequence[Sequence[int]], # "up to any trailing suffix": a suffix WORD anywhere after the # marker ends the maiden name, and so does the trailing numeral as # assign will read it, which the suffix-piece test does not see - # (#424): 'John née Jones Smith V' took the V as maiden text. The - # numeral only -- trailing_start says why the acronym fork is - # left to assign here. Read from the MARKER, not after it: a + # (#424): 'John née Jones Smith V' took the V as maiden text. Both + # forks now -- the acronym one since #533, asked the same double + # way. Read from the MARKER, not after it: a # numeral straight after the marker then has the piece before it # the fork wants, and 'Jane Smith née V' declines like 'Jane Smith # née PhD' -- nothing after the marker but a suffix, so the marker # stays a word -- as 1.4.0 read it. # - # `one_case` is passed here and at the re-ask below and changes - # NOTHING, by construction: `numeral_only` answers off - # `peeled.numeral`, and the numeral fork is decided before the peel - # ever reads a lean -- the fact reaches only the bare-acronym fork, - # which this reading discards. Measured anyway, 2026-09-18, because - # "by construction" is the claim this repository gets wrong most - # often: dropping the argument at these TWO sites moves 0 of 9,852 - # parses (1,642 names -- the distinct union of every - # `tools/differential/corpus*.jsonl` entry, every `cases.py` text, - # and `tests/test_variations.TEST_NAMES` with its comma - # permutations -- under six policies: the default, both - # family-first orders, strict commas, and each 2.4 switch flipped; - # recompute recipe in decisions.md#S2). It stays passed rather - # than spelled `None` because `None` is a different statement -- - # "nobody asked" -- and a future numeral fork that DID read the - # writing would then be wrong silently. + # `one_case` is LIVE at these sites since #533, and it was not + # before: `numeral_only` answered off `peeled.numeral`, and the + # numeral fork is decided before the peel ever reads a lean, so + # the fact reached only the bare-acronym fork -- which that + # reading discarded. The acronym fork is asked now, so the writing + # decides here as it decides at the trailing slot of a name. + # + # Measured 2026-09-19 with a runtime wrapper that forces this + # function's `one_case` argument to None, over the population + # decisions.md#S2's 2026-09-18 recipe names -- the distinct union + # of every `tools/differential/corpus*.jsonl` entry, every + # `tests/v2/cases.py` text, and `tests/test_variations.TEST_NAMES` + # with the three comma permutations that entry names (no-comma, + # family-comma, and suffix-comma where the name has a suffix, + # built off the PARSE as `test_variations_of_TEST_NAMES` builds + # them -- not off a word split, which is what a first draft of + # this comment counted and why it read 2,429 names), empty + # strings dropped, under six policies (the default, both + # family-first orders, strict commas, and each 2.4 switch + # flipped). THE PAIR IS THE FINDING: + # over the corpus as it stood the day before this change it moved + # 0 parses, and over the corpus WITH this change's own rows it + # moves 36 of 10,740 (1,790 names), on 6 distinct names ('Doe, + # Jane nee Smith DO', 'Doe, Jane nee Smith Ma', 'Jane Doe nee + # Smith Ma', 'Jane Doe nee Smith Ma JD', 'Jane Doe nee Yo-Yo Ma', + # 'John née Jones Smith MA'). The plumbing was live either way; + # the corpus simply + # held no name that could show it, which is the blindness + # mechanisms.md's corpus field note asks to be measured before any + # "N names move" is written down. The 2026-09-18 record of 0 of + # 9,852 under the numeral-only reading stands as what was true + # then and is superseded here. # # The chain-tail measure below (`tail`, and the re-peel after the - # chain) is the opposite, and the same sweep says so: dropping it - # there moves 18 of the 9,852, on 'John van der Berg Ma', 'John de + # chain) is the opposite, and the 2026-09-18 sweep says so: + # dropping it moves 18 of that 9,852, on 'John van der Berg Ma', + # 'John de # Ma' and 'Freiherr von Berg MA' under every one of the six. A # review round called all three sites inert together; two are. skip = frozenset(range(len(pieces))) - frozenset(seen) - trailing = trailing_start(seen[m], pieces, ptags, tokens, skip, - numeral_only=True, one_case=one_case) + rest = peel_walk(seen[m], ptags, skip) + peeled = peel_trailing(rest, pieces, ptags, tokens, one_case) + trailing = rest[-1] if peeled.numeral is not None else len(pieces) # The fork reads the piece before the numeral, and the take # REMOVES that piece: afterwards assign sees the piece before the # marker there, and if that is initial-shaped the fork will not @@ -284,6 +333,114 @@ def _maiden_take(pieces: Sequence[Sequence[int]], numeral_only=True, one_case=one_case) == len(view): trailing = len(pieces) + # #533: the ACRONYM fork, asked the way the numeral is -- the peel + # over the pieces as they stand, then again over the name the take + # would leave, and a stop only where both read the word as the + # credential. `peeled.names` is the first piece the peel took, so + # the class member it stopped at is that piece and no walk of our + # own is needed. One peel, one view built once per take: O(pieces) + # for the take, not per member, and no re-entrancy -- the + # predicate never calls the walk that calls it. + if (reader is not TailReader.NONE and peeled.names < len(rest) + and m + run + 1 < len(seen)): + # OPTION 1: the stop never takes the FIRST word after the + # marker -- a class member standing alone there stays the + # maiden name. A clamp rather than a veto: where the peel + # consumed that word AND words behind it, only the first stays + # ('Doe, J. nee MA ba' keeps maiden 'MA' and reads suffix + # 'ba'; a veto handed 'ba' back to the clause too). The + # clamped piece may then be no member at all, and the test + # below declines -- which changes nothing, the walk stopping + # at that suffix word of its own accord. + stop = max(rest[peeled.names], seen[m + run + 1]) + head = pieces[stop] + # Both halves of the membership test below are DEFENSIVE, and + # measured inert on 2026-09-19 -- over 297,381 parses (1,533 + # corpus and case names plus 4,536 generated clause shapes, + # seven policies, seven lexicons) admitting either half moved + # no parse and, at the default vocabulary, no frame. What each + # one guards, and why it stays: + # + # `len(head) == 1` asks a LONE piece's question, and the + # answer below reads `head[0]` as if the piece were the word. + # The only multi-token piece the marker pass can see is the + # Ph. D. merge above, which carries the `suffix` ptag, so the + # walk stops there of its own accord and the stop would change + # nothing -- structural today, and the default vocabulary does + # not even put the tag on that piece's head: a caller listing + # `ph` ambiguous is what could. + # THE NEGATIVE CONTROL, so the "inert" above is checkable + # rather than asserted. A probe that fires wherever the tag + # test admits a head this length test then DECLINES -- the + # only sites where dropping it could matter -- recorded 0 over + # 21,480 parses: decisions.md#S2's population (1,790 names; + # the recipe is spelled out above) under six policies and two + # lexicons, the default and one listing `ph` ambiguous. Nor is + # there a + # price: with `ph` listed, dropping the test leaves 'Jane Doe + # nee Smith Ph. D.' at 372 frames, 'Doe, Jane nee Smith + # Ph. D.' at 399 and 'J. nee Jones Smith Ph. D.' at 336, + # unchanged to the frame. It is the walk, not this test, that + # keeps the merged piece from ever being the stop. Kept + # for the reason `_assign.previous_kept` is: an inert branch + # is cheaper than a question asked of the wrong shape, and the + # four sibling sites (`_pieces.tail_reading`, + # `_pieces.peel_trailing`, the GIVEN_SLOT branch below, the + # emitter at the end of this function) all pair the two. + # + # The tag is the CLASS the rule is stated in terms of, and it + # is not redundant with the walk the way the length test is: + # 'J. née Jones Smith V' reaches here on a piece + # `is_suffix_piece` REFUSES for being initial-shaped, and what + # declines it is the view check rather than the walk. What the + # tag buys is the cost, and that control is a price rather + # than a count: dropping it runs the view machinery over every + # ordinary credential the peel took, measured on a scratch + # copy of the package with the test deleted -- 'Jane Doe nee + # Smith PhD' 341 -> 353 frames and 'Doe, Jane nee Smith PhD' + # 367 -> 372, counted per `Parser.parse` the way + # tests/v2/test_benchmark counts them. No test pins those + # numbers: `_CALL_BASELINE` is per-interpreter and per entry + # point, and a row for one name would have to be guessed for + # the four interpreters only CI runs. + if (stop < trailing and len(head) == 1 + and AMBIGUOUS_ACRONYM_TAG in tokens[head[0]].tags): + left = [i for i in seen if i < seen[m] or i >= stop] + view = [pieces[i] for i in left] + view_tags = [ptags[i] for i in left] + # where the member stands in that view: everything before + # the marker, then the run the take would leave behind. + # The question is whether the reader takes THIS piece, not + # whether it takes something -- a suffix word behind the + # member answers yes to the weaker question while the + # member itself reads as the family name ('JOHN NEE JONES + # SMITH MA PHD' left 'JOHN MA PHD', whose MA is the + # family). + at = len(view) - (len(left) - left.index(stop)) + if reader is TailReader.GIVEN_SLOT: + # after a family comma the words to spare are there by + # construction, so the reader is #531's -- the member's + # own reading, asked through the one predicate that + # owns it, and that rule's FLOOR: the member ends the + # given part only where every piece behind it reads as + # a suffix too ('Doe, Jane MA do' reads middle 'MA', + # the particle not being a credential, so the clause + # keeps both words rather than handing one of them to + # the current name's middle). + takes = all( + is_suffix_piece(view[q], view_tags[q], tokens) + or (len(view[q]) == 1 + and AMBIGUOUS_ACRONYM_TAG in tokens[view[q][0]].tags + and credential_at_the_given_slot( + tokens[view[q][0]], one_case)) + for q in range(at, len(view))) + else: + takes = trailing_start( + leading_titles(view, view_tags, tokens), + view, view_tags, tokens, + one_case=one_case) <= at + if takes: + trailing = stop j = m + run while (j < len(seen) and seen[j] < trailing and not is_suffix_piece(pieces[seen[j]], ptags[seen[j]], @@ -295,6 +452,34 @@ def _maiden_take(pieces: Sequence[Sequence[int]], # second word is the marker, not the first word it takes. if j <= m + run: return None + # #533, mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE: "Emit at the + # site that takes the branch, not where an ambiguous tag sits" -- + # the walk that KEPT the word is where the fork was called, so it + # is where the report is raised. The LAST word of the maiden name + # is the one the trailing rule was asked about: everything behind + # it read as a suffix (that is what let the peel reach it), and a + # member the reading TOOK is not in the maiden name any more -- + # assign reports that one where it peels it, so no token is ever + # reported twice. A member with a name word behind it was never + # asked and stays silent, which rules.md#A1's hesitating reader is + # the reason for rather than the accident of. + # + # Gated on the reader for the same reason the walk is: where no + # trailing rule reads these words, nothing was decided and nothing + # may report. Gated on EITHER tag, as the chain emitter's is, so a + # by-shape member reports with the dotted switch off -- classify + # writes the shape tag there while the class does not admit it, + # which is the one place a declined fork can be recorded. + last = pieces[seen[j - 1]] + if (reader is not TailReader.NONE and len(last) == 1 + and not tokens[last[0]].tags.isdisjoint( + _AMBIGUOUS_CREDENTIAL_TAGS)): + ambiguities.append(PendingAmbiguity( + AmbiguityKind.SUFFIX_OR_NAME, + f"{tokens[last[0]].text!r} ending the maiden name is also " + f"a post-nominal; the maiden marker's clause keeps it " + f"rather than reading it as one", + tuple(last))) return seen[m:m + run], seen[m + run:j] @@ -328,6 +513,8 @@ def _group_segment(seg: tuple[int, ...], additional: int, opens_the_name: bool = False, *, one_case: bool | None, + reader: TailReader = TailReader.TRAILING, + maiden_ambiguities: list[PendingAmbiguity] | None = None, ) -> tuple[list[Piece], list[set[str]], MaidenTake | None]: pieces: list[Piece] = [[i] for i in seg] ptags: list[set[str]] = [set() for _ in seg] @@ -336,6 +523,15 @@ def _group_segment(seg: tuple[int, ...], additional: int, # list) suppresses reporting -- see group() for when that applies. if ambiguities is None: ambiguities = [] + # The maiden walk's own channel. group() passes `None` for the + # chain emitter after a family comma -- the comma fixed the + # family, so that fork is settled -- and #533's is not that fork: + # a credential ending a maiden clause is a question the comma + # settles nothing about. A second parameter rather than a widening + # of the first, so neither channel can quietly acquire the other's + # suppression. + if maiden_ambiguities is None: + maiden_ambiguities = ambiguities def title(k: int) -> bool: return is_title_piece(pieces[k], ptags[k], tokens) @@ -468,7 +664,8 @@ def merge(lo: int, hi: int, add: Set[str] = frozenset(), # The tokens are not touched here: this function reads them and # returns what it took, and group() records the drop and the roles. taken: MaidenTake | None = None - take = _maiden_take(pieces, ptags, tokens, cores, one_case) + take = _maiden_take(pieces, ptags, tokens, cores, one_case, + reader, maiden_ambiguities) if take is not None: marker_ks, maiden_ks = take taken = ([i for k in marker_ks for i in pieces[k]], @@ -927,13 +1124,26 @@ def group(state: ParseState) -> ParseState: # there is no fork left to report. tail = tail_start is not None and seg_idx >= tail_start seg_cores = cores if tail else frozenset() + # #533: which rule reads what the maiden walk would leave, off + # the three facts already in hand here. A tail segment is read + # as credentials whole and segment 0 of a family comma is the + # family the comma named, so neither consults a trailing rule. + if tail: + reader = TailReader.NONE + elif family_comma: + reader = (TailReader.GIVEN_SLOT if seg_idx == 1 + else TailReader.NONE) + else: + reader = TailReader.TRAILING pieces, ptags, taken = _group_segment( seg, additional, tokens, bound_join, None if family_comma else ambiguities, seg_cores, state.lexicon.given_name_titles, opens_the_name=(seg_idx == 0 and not family_comma), - one_case=state.one_case) + one_case=state.one_case, + reader=reader, + maiden_ambiguities=ambiguities) # the marker is dropped and the maiden name's tokens become # MAIDEN (#274); which pieces those are was settled in # _group_segment, before the joins diff --git a/nameparser/_pipeline/_pieces.py b/nameparser/_pipeline/_pieces.py index ad38208a..e6afac6a 100644 --- a/nameparser/_pipeline/_pieces.py +++ b/nameparser/_pipeline/_pieces.py @@ -417,14 +417,15 @@ def trailing_start(start: int, pieces: Sequence[Sequence[int]], or a bare acronym with words to spare, into the family or the maiden name. - `numeral_only` is the maiden walk's reading: the bare-acronym - fork counts pieces, and the walk removes the very pieces it - counted, so an acronym peeled over the pieces as they stand may - be the family of what is left ('John née Jones Smith Ma' read - maiden 'Jones Smith', family 'Ma'). The numeral fork reads one - piece, the one before the numeral, and _maiden_take re-asks it - with the piece the take leaves there; the acronym is left to - assign.""" + `numeral_only` answers off `Peel.numeral` alone: the bare-acronym + fork counts pieces, and a caller that REMOVES the pieces it + counted cannot use that count as it stands ('John née Jones Smith + Ma' peeled over the pieces as written reads the acronym as a + credential with words to spare, and once 'Jones Smith' has left + it is the family of what remains). Its one caller is the maiden + walk's numeral re-ask, over the view the take would leave; the + walk asks the acronym fork itself, over a view of its own + (#533).""" rest = peel_walk(start, ptags, skip) peeled = peel_trailing(rest, pieces, ptags, tokens, one_case) if numeral_only: @@ -436,7 +437,7 @@ def trailing_start(start: int, pieces: Sequence[Sequence[int]], # peel_trailing and segment_suffix_reading ask before reading the # lean -- shared here so the two cannot drift on what counts # (quality-review finding: it was spelled twice, once per site, -# before this). Both callers test "vocab:suffix-ambiguous" in tags +# before this). Those two test "vocab:suffix-ambiguous" in tags # INLINE, before calling this, rather than leaving that cheap check to # this function's own body: measured, a caller whose `elif` reaches # this on every piece (segment_suffix_reading's does, one per @@ -444,6 +445,18 @@ def trailing_start(start: int, pieces: Sequence[Sequence[int]], # regardless of what is inside it, and the inline pre-check is what # keeps a non-member piece ("Smith, John"'s "John") from ever making # the call at all. +# +# A THIRD caller since #533 -- credential_at_the_given_slot just +# below -- deliberately does NOT pre-check, because it is the one +# predicate that owns #531's reading and membership is its caller's +# to decide (its own docstring says so). It pushes the test out to +# ITS callers rather than asking it twice, and the frame argument +# holds transitively because both of them do ask it inline: assign's +# walkable pass tests `AMBIGUOUS_ACRONYM_TAG in tok.tags` after a +# `len(piece) == 1` before the call (_assign.py, the given part's +# trailing slot), and the maiden walk's view check spells the same +# pair inside its `all(...)` (_group.py, `_maiden_take`). So no +# non-member piece reaches this function down that route either. def listed_lean(token: WorkToken, one_case: bool | None) -> Lean | None: """`ambiguous_lean` for a LISTED bare-ambiguous token, or None if the token is not tagged a listed member, is admitted by SHAPE @@ -455,6 +468,30 @@ def listed_lean(token: WorkToken, one_case: bool | None) -> Lean | None: return ambiguous_lean(token.text, one_case) +def credential_at_the_given_slot(token: WorkToken, + one_case: bool | None) -> bool: + """#531's reading of a class MEMBER ending the given part after a + family comma: the credential unless the writing says otherwise. + The caller decides membership and that the piece ends that part. + + The words to spare are there by construction at that slot, so the + count says nothing and only the lean does; a member that is also + particle vocabulary reads as the credential on a POSITIVE lean + alone, P6's attachment keeping every other spelling. + + Two callers since #533 -- assign's walk over the given part, and + the maiden walk's second check over the name the take would leave + (rules.md#M2) -- so the reading is a function rather than a + condition written twice + (mechanisms.md#ONE-PREDICATE-PER-QUESTION). It is a + text-and-tags question, which is what puts it in this module + rather than beside either caller. + """ + lean = listed_lean(token, one_case) + return lean == "credential" or (lean is None + and "particle" not in token.tags) + + def peel_trailing(rest: Sequence[int], pieces: Sequence[Sequence[int]], ptags: Sequence[Set[str]], tokens: Sequence[WorkToken], diff --git a/nameparser/_types.py b/nameparser/_types.py index ab23d12b..27cca660 100644 --- a/nameparser/_types.py +++ b/nameparser/_types.py @@ -419,8 +419,8 @@ class AmbiguityKind(StrEnum): #: and this is the boundary rather than an omission to be read #: past. The emitters cover the trailing slot of a name, the FIRST #: PIECE after a family comma -- that piece and no further -- the - #: trailing slot of that listing's GIVEN part, and the extra - #: segments beyond it. + #: trailing slot of that listing's GIVEN part, the trailing slot + #: of a maiden marker's clause, and the extra segments beyond it. #: Since 2.4 the given part's trailing slot reports whichever way #: it read the word: "Doe, John MA" reads suffix ``MA`` and says #: so, "Doe, John Ma" keeps middle ``Ma`` and says so too. Not in @@ -429,15 +429,32 @@ class AmbiguityKind(StrEnum): #: site and this kind stays out of the way -- "Doe, John do" gives #: family ``do Doe`` and one ``PARTICLE_OR_GIVEN``, never two #: reports of one word. - #: FOUR positions stay silent, and all four are boundaries - #: rather than omissions. A member with a NAME WORD behind it was - #: never a fork -- "Doe, John MA Smith" reads middle ``MA Smith``, - #: the ordinary reading, and nothing consulted the class. The - #: maiden walk claims everything behind its marker before this - #: slot exists, so "Doe, Jane nee Smith MA" gives maiden - #: ``Smith MA`` in silence, exactly as the comma-less "John Smith - #: nee Jones R.A.I." does. And -- pre-existing, and untouched by - #: 2.4 -- a member with no name word IN FRONT of it is not at this + #: Since 2.4 a maiden marker's clause reports at ITS trailing slot + #: too, in both directions: "Doe, Jane nee Smith MA" gives maiden + #: ``Smith`` with suffix ``MA`` and says so, "Doe, Jane nee Smith + #: Ma" keeps maiden ``Smith Ma`` and says so too, and a member the + #: clause keeps because it is the only word after the marker + #: ("Jane Doe nee MA") reports as well. Where no trailing rule + #: reads the clause's tail, nothing was decided and nothing + #: reports: a clause in the FAMILY segment of a comma listing + #: ("Smith nee Jones MA, Jane" keeps maiden ``Jones MA``) and one + #: past a second comma ("Smith, John, Jr nee Jones MA" keeps + #: maiden ``Jones MA``) are both silent. + #: FOUR positions stay silent, and all four are boundaries rather + #: than omissions. The NO-READER clause just named is the first of + #: them, and the three that follow are about the member's own + #: surroundings. Second: a member with something BEHIND it that + #: the trailing reading does not take was never a fork -- "Doe, + #: John MA Smith" reads middle ``MA Smith``, the ordinary reading, + #: and nothing consulted the class, and inside a clause the same + #: holds of a name word ("Jane Doe nee MA Smith" keeps maiden + #: ``MA Smith``) and of a trailing TITLE, which breaks the + #: clause's peel before it can reach the member at all ("Jane Doe + #: nee Smith MA Prof." keeps maiden ``Smith MA Prof.``, where + #: "Jane Doe nee Smith Prof. MA" reads suffix ``MA`` and reports). + #: Third -- pre-existing, and 2.4 widening what reaches it rather + #: than moving it -- a member with no name word IN FRONT of it is + #: not at this #: slot either, because the slot is the end of a given part and #: there is none: a title took that position. "Doe, Dr. MA" gives #: suffix ``MA`` and "Doe, Mr. MA PhD" suffix ``MA PhD``, the @@ -448,8 +465,15 @@ class AmbiguityKind(StrEnum): #: immediately after the comma and nothing behind that piece: #: "Doe, MA Smith" reports its ``MA``, and "Doe, Dr. MA Smith" -- #: the same member, one title in front of it -- reads given ``MA`` - #: in silence. And -- pre-existing and untouched by 2.4 as well -- - #: a PARTICLE beside the member can + #: in silence. A maiden clause can put a name in that position: + #: "Doe, Dr. nee Smith MA" gives title ``Dr.``, suffix ``MA`` and + #: maiden ``Smith``, and moves in silence -- once ``Smith`` leaves + #: with the marker, segment 1 is ``Dr. MA``, a no-name segment the + #: credential-run gate reads whole. (The other direction still + #: reports, the clause's own emitter being what raises it: "Doe, + #: Dr. nee Smith Ma" keeps maiden ``Smith Ma`` and says so.) + #: And fourth -- pre-existing and untouched by 2.4 as well -- + #: a JOIN beside the member can #: take it out of this slot, from either side. Where a chain has #: swallowed the member into one piece there is no lone member to #: ask about, and the member may as well HEAD that piece as trail @@ -457,7 +481,12 @@ class AmbiguityKind(StrEnum): #: so does "Doe, John DO Ed", where the member is itself the #: particle the chain runs on and the name word behind it joins #: the piece -- though "Doe, John DO" alone reads the credential - #: and reports. Where a particle the suffix vocabulary + #: and reports. The particle chain is the commonest joiner but not + #: the only one: the bound-given join takes a member into its pair + #: the same way, so "Berg, abdul MA" reads given ``abdul MA`` in + #: silence -- and "Berg, abdul nee Jones MA" agrees with it, the + #: clause giving the member up and the join catching it before + #: this slot is reached. Where a particle the suffix vocabulary #: does not also claim stands BEHIND it, the given part ends at #: that particle as this walk reads it, and the attachment that #: moves the particle to the family runs a stage too late to @@ -468,8 +497,8 @@ class AmbiguityKind(StrEnum): #: it: "Doe, John van MA" reads family ``van Doe``, suffix ``MA`` #: and reports both forks, and "Doe, John MA vd" reads suffix #: ``MA`` past a ``vd`` the suffix vocabulary claims outright. - #: All four POSITIONS above are silent -- those last two names are - #: the boundary each one stops at, not instances of it. + #: All four POSITIONS above are silent -- those last two names + #: are the boundary each one stops at, not instances of it. SUFFIX_OR_NAME = "suffix-or-name" #: An input the title peel eats down to one last word which is #: itself title vocabulary still has to name somebody, so that diff --git a/tests/v2/cases.py b/tests/v2/cases.py index 20dd42ed..13934195 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -2807,15 +2807,616 @@ def _check_cjk_shape_purity(self) -> None: shape=2), Case("a_maiden_clause_takes_the_member_with_it", "Doe, Jane nee Smith MA", - {"given": "Jane", "family": "Doe", "maiden": "Smith MA"}, + {"given": "Jane", "family": "Doe", "suffix": "MA", + "maiden": "Smith"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="the row that named the silence, now naming the " + "reading. The maiden marker no longer claims a " + "trailing credential: the words it takes end where a " + "trailing credential begins, and after a family comma " + "the reader of what is left standing is the given " + "part's own trailing slot (#531), which reads 'MA' as " + "the credential. 1.4.0 had no maiden routing and read " + "middle 'nee Smith', suffix 'MA', so the SUFFIX is " + "1.4.0 parity and the maiden field is not; 2.0.0 " + "through 2.3.0 read maiden 'Smith MA' in silence " + "(measured 2026-09-19). Keeping the id: it is the same " + "question, answered the other way", + shape=2), + # ---- #533: the maiden clause's trailing credential ------------- + # The rule: the words a maiden marker takes end where a trailing + # credential begins, and a member of the ambiguous credential + # class is one of those -- but only where the rule that reads the + # name left standing reads it as the credential, both as written + # and as the take would leave it. That is the same double question + # the trailing roman numeral is asked, for the same reason: the + # count of words to spare includes the very words the marker + # removes. + Case("a_trailing_credential_ends_the_maiden_clause", + "Jane Doe nee Smith MA", + {"given": "Jane", "family": "Doe", "suffix": "MA", + "maiden": "Smith"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="the comma-less spelling of the row above, and the " + "statement of the rule. The peel over the pieces as " + "they stand takes 'MA', and the peel over the view the " + "take would leave ('Jane Doe MA') takes it too, so the " + "clause stops before it. 1.4.0 read middle 'Doe nee', " + "family 'Smith', suffix 'MA' -- the suffix restored, " + "the maiden field new since #274", + shape=1), + Case("the_clause_keeps_the_member_its_writing_declines", + "Jane Doe nee Smith Ma", + {"given": "Jane", "family": "Doe", "maiden": "Smith Ma"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="the other direction, and the one that carries most of " + "this change's visible effect at the default: Title " + "case in a mixed-case name is written the way a NAME " + "is written, so the peel declines the member and the " + "clause keeps it -- and the fork was consulted, so it " + "reports. The reading is unchanged from 2.0.0 through " + "2.3.0; only the report is new. 1.4.0 read suffix 'Ma'", + shape=1), + Case("the_all_caps_clause_reads_the_credential", + "JANE DOE NEE SMITH MA", + {"given": "JANE", "family": "DOE", "suffix": "MA", + "maiden": "SMITH"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="one case, so the lean is inert and the count decides " + "alone -- and with 'Jane Doe' standing in front of the " + "clause there are words to spare both as written and " + "as the take would leave the name. 1.4.0 parity on the " + "suffix", + shape=1), + Case("the_all_lower_clause_reads_the_credential", + "jane doe nee smith ma", + {"given": "jane", "family": "doe", "suffix": "ma", + "maiden": "smith"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="the other one-case spelling, same reason as its " + "all-caps twin. 1.4.0 parity on the suffix", + shape=1), + Case("the_clause_keeps_a_member_standing_alone_after_the_marker", + "Jane Doe nee MA", + {"given": "Jane", "family": "Doe", "maiden": "MA"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="the FLOOR, and it is deliberate that this differs " + "from what a suffix word or a roman numeral gets in " + "the same position ('Jane Smith nee PhD' and 'Jane " + "Smith nee V' leave the marker standing as an ordinary " + "word). The marker announces a NAME, and the rule " + "gives a word up only where a maiden name is left " + "standing; certain suffix vocabulary declines the " + "marker, an ambiguous word is kept by the clause it " + "ends. It reports all the same. 1.4.0 read family " + "'nee', suffix 'MA', so this half restores nothing and " + "does not try to", + shape=1), + Case("a_name_word_behind_the_member_leaves_the_clause_silent", + "Jane Doe nee MA Smith", + {"given": "Jane", "family": "Doe", "maiden": "MA Smith"}, + classification="fix(#274)", + ambiguities=(), + notes="the recorded negative control for the emitter: the " + "word the walk was asked about is the LAST piece of " + "the maiden name, and a member with a name word behind " + "it is never that piece. Nothing was consulted, so " + "nothing reports -- which is what rules.md#A1's " + "hesitating reader asks for rather than an accident of " + "where the emitter sits. Unchanged from 2.0.0; 1.4.0 " + "read middle 'Doe nee MA', family 'Smith'", + shape=1), + Case("the_clause_gives_up_the_whole_credential_run", + "Jane Doe nee Smith MA PhD", + {"given": "Jane", "family": "Doe", "suffix": "MA PhD", + "maiden": "Smith"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="the peel walks the run from the end, so 'PhD' is " + "settled vocabulary and 'MA' is the fork it reaches " + "behind it -- one report, not two. Rendered with a " + "SPACE: R1 derives suffix entries from the commas the " + "WRITER typed (#436/#437), where 1.4.0 wrote 'MA, " + "PhD'. 2.0.0 through 2.3.0 read maiden 'Smith MA' with " + "suffix 'PhD', which is the order-sensitivity the " + "issue reported", + shape=1), + Case("the_credential_run_reads_the_same_written_the_other_way", + "Jane Doe nee Smith PhD MA", + {"given": "Jane", "family": "Doe", "suffix": "PhD MA", + "maiden": "Smith"}, + classification="fix(#289)", + ambiguities=("suffix-or-name",), + notes="the control the row above needs, and the whole point " + "of the issue: this spelling already stopped at the " + "suffix WORD and read both credentials, so whether the " + "member was read at all used to depend on which side " + "of 'PhD' the writer put it. Unchanged here -- the two " + "orders now agree", + shape=1), + Case("the_declined_member_still_ends_the_clause_for_the_run", + "Jane Doe nee Smith Ma JD", + {"given": "Jane", "family": "Doe", "suffix": "JD", + "maiden": "Smith Ma"}, + classification="fix(#533)", + ambiguities=("suffix-or-name", "suffix-or-name"), + notes="both halves in one name: 'JD' is taken and reported " + "by assign where it peels it, 'Ma' is declined by its " + "writing and reported by the walk that kept it. TWO " + "reports, one per member, and never two for one word " + "-- the maiden pieces are gone before the chain runs. " + "1.4.0 read suffix 'Ma, JD'", + shape=1), + Case("two_members_ending_the_clause_report_once_each", + "Jane Doe nee Smith MA JD", + {"given": "Jane", "family": "Doe", "suffix": "MA JD", + "maiden": "Smith"}, + classification="fix(#533)", + ambiguities=("suffix-or-name", "suffix-or-name"), + notes="the same pair with both members taken. The peel " + "resolves each in turn and assign reports both; the " + "walk reports none, the last maiden piece being an " + "ordinary name word", + shape=1), + Case("the_dotted_member_ends_the_clause", + "John Smith nee Jones R.A.I.", + {"given": "John", "family": "Smith", "suffix": "R.A.I.", + "maiden": "Jones"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="the name #530's close-out reported from the other " + "side, and this row RESTORES 2.3.0 rather than " + "changing it: at 2.3.0 'R.A.I.' carried vocab:suffix " + "and the walk's suffix-piece test stopped at it, while " + "#516 retagged it shape:acronym plus " + "vocab:suffix-ambiguous, it stopped being a suffix " + "piece, and the walk took it -- maiden 'Jones R.A.I.' " + "on this tree, unrecorded because the name was in no " + "corpus file. 1.4.0 read middle 'Smith nee', family " + "'Jones', suffix 'R.A.I.' (all measured 2026-09-19)", + shape=1), + Case("the_dotted_member_is_kept_with_the_switch_off_and_reports", + "John Smith nee Jones R.A.I.", + {"given": "John", "family": "Smith", "maiden": "Jones R.A.I."}, + policy=Policy(unlisted_dotted_suffixes=False), + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="with the switch off classify writes the SHAPE tag and " + "the class does not admit the token, so the PEEL " + "declines to consume it -- it records the word in " + "`picks` and breaks, leaving nothing for the walk's " + "own reading gate to be asked about -- and the clause " + "keeps the word. The emitter's gate reads EITHER tag, " + "as the chain emitter's does, so the declined fork is " + "still reported. That asymmetry is the whole of the " + "two gates' difference and is what this row pins. " + "Measured 2026-09-19 by stepping the peel, not " + "reasoned from the gate's text"), + Case("the_caps_switch_reaches_the_clause", + "Jane Doe nee Smith XYZ", + {"given": "Jane", "family": "Doe", "suffix": "XYZ", + "maiden": "Smith"}, + policy=Policy(unlisted_caps_suffixes=True), + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="the opt-in class reaches this slot like any other, " + "the switch being what admits the token to the " + "ambiguous class in the first place"), + Case("an_unlisted_word_is_no_member_of_the_class", + "Jane Doe nee Smith XYZ", + {"given": "Jane", "family": "Doe", "maiden": "Smith XYZ"}, + classification="fix(#274)", + ambiguities=(), + notes="the default-policy control for the row above, and the " + "recorded negative control for the membership gate: " + "with the caps switch off 'XYZ' carries neither tag, " + "so the walk never asks and the clause keeps it in " + "silence. 1.4.0 read family 'XYZ'", + shape=1), + Case("the_peel_never_reaches_a_title_behind_the_member", + "Jane Doe nee Smith MA Prof.", + {"given": "Jane", "family": "Doe", "maiden": "Smith MA Prof."}, + classification="fix(#274)", + ambiguities=(), + notes="the H5 BOUNDARY, recorded rather than fixed: the " + "maiden walk has always read peel_trailing alone, and " + "the trailing-title chain lives in tail_reading, which " + "trailing_start does not run -- so the peel breaks at " + "'Prof.' and never reaches the member behind it. This " + "change inherits that boundary rather than creating " + "it, and a follow-up carries the question of whether " + "the walk should move onto tail_reading (both forks, " + "numeral included). Silent, because nothing was asked", + shape=1), + Case("a_title_in_front_of_the_member_is_the_other_spelling", + "Jane Doe nee Smith Prof. MA", + {"given": "Jane", "family": "Doe", "suffix": "MA", + "maiden": "Smith Prof."}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="the boundary's other side, and the pair is the " + "finding: the two spellings DISAGREE here, where at " + "the given part's own trailing slot they agree " + "(#531's 'Doe, John MA Prof.' and 'Doe, John Prof. " + "MA' land on one answer). The peel reaches 'MA' " + "because nothing stands behind it, so the clause stops " + "and 'Prof.' stays maiden text. 1.4.0 read family " + "'Prof.', suffix 'MA'", + shape=1), + Case("a_connective_behind_the_member_stops_the_peel", + "Jane Doe nee Smith MA y", + {"given": "Jane", "family": "Doe", "maiden": "Smith MA y"}, + classification="fix(#274)", + ambiguities=(), + notes="the recorded negative control for the PEEL's reach, " + "measured 2026-09-19 rather than reasoned: the marker " + "pass runs before every join, so 'MA' and 'y' are two " + "pieces here and no joined one -- and 'y' is no " + "suffix, so the peel takes nothing, the walk is never " + "asked about the member behind it, and the clause " + "keeps both words. The connective joins them a stage " + "later, into the maiden name. Silent for the reason " + "the row below is: nothing was decided", + shape=1), + Case("the_numeral_half_of_the_walk_is_unmoved", + "Jane Doe nee Smith V", + {"given": "Jane", "family": "Doe", "suffix": "V", + "maiden": "Smith"}, + classification="fix(#424)", + ambiguities=("suffix-or-name",), + notes="#424's own fork, pinned beside the acronym one " + "because the two now share a single peel: the numeral " + "half answers off Peel.numeral exactly as it did, and " + "this row is what says the shared call did not move " + "it. Unchanged since #424", + shape=1), + Case("the_digit_shaped_trailing_suffix_is_unmoved_and_silent", + "Jane Doe nee Smith 2", + {"given": "Jane", "family": "Doe", "suffix": "2", + "maiden": "Smith"}, + classification="fix(#424)", + ambiguities=(), + notes="the numeral half's silent twin -- a lone digit is " + "generational vocabulary and no fork, so the walk " + "stops and nothing reports. The pair with the row " + "above is what separates 'the walk stopped' from 'the " + "walk reported'", + shape=1), + Case("the_one_case_record_loses_its_second_birth_word", + "JANE DOE NEE YO-YO MA", + {"given": "JANE", "family": "DOE", "suffix": "MA", + "maiden": "YO-YO"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="an ACCEPTED COST, and the control beside it is what " + "makes it the one-case reading's cost rather than this " + "rule's: 'JANE YO-YO MA' reads suffix 'MA' too, so the " + "clause form now agrees with the bare form. In mixed " + "case the writing saves the name -- see the row below", + shape=1), + Case("the_one_case_records_control_without_the_clause", + "JANE YO-YO MA", + {"given": "JANE", "family": "YO-YO", "suffix": "MA"}, + classification="fix(#289)", + ambiguities=("suffix-or-name",), + notes="the recorded control for the accepted cost above. " + "Unchanged by this rule and read by #289's count, " + "which is the point: a clause must not change how a " + "word outside it reads, and here the clause form " + "joined the bare form rather than the other way round", + shape=1), + Case("the_mixed_case_record_keeps_its_second_birth_word", + "Jane Doe nee Yo-Yo Ma", + {"given": "Jane", "family": "Doe", "maiden": "Yo-Yo Ma"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="the same two-word birth surname where the writing " + "CAN speak: Title case in a mixed-case name declines " + "the peel, so 'Yo-Yo Ma' stays whole and the fork " + "reports. 1.4.0 read family 'Yo-Yo', suffix 'Ma'", + shape=1), + Case("the_view_check_asks_whether_the_reader_takes_that_word", + "JOHN NEE JONES SMITH MA PHD", + {"family": "JOHN", "suffix": "PHD", + "maiden": "JONES SMITH MA"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="the row that distinguishes the check this change " + "SHIPS from the weaker one. The take would leave " + "'JOHN MA PHD', whose peel takes 'PHD' and then " + "DECLINES 'MA' for want of words to spare -- so 'does " + "the reader take SOMETHING' answers yes while 'MA' " + "becomes the FAMILY name, which is #424's own disaster " + "one word further on. The check compares the view's " + "run start against the member's index in it, answers " + "no, and the clause keeps the word -- reporting, " + "because the fork was consulted and declined", + shape=1), + Case("the_view_checks_control_without_the_clause", + "JOHN MA PHD", + {"given": "JOHN", "family": "MA", "suffix": "PHD"}, + classification="fix(#289)", + ambiguities=("suffix-or-name",), + notes="the recorded control: this IS the view the take would " + "leave, and family 'MA' is what the row above must " + "not produce. Unchanged", + shape=1), + Case("a_dangling_connective_can_end_a_maiden_name", + "Jane Smith nee Jones and MA", + {"given": "Jane", "family": "Smith", "suffix": "MA", + "maiden": "Jones and"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="an ACCEPTED COST. The take runs before the joins " + "(#420), so the connective is a piece of its own when " + "the walk stops and the maiden name ends on it; M2's " + "Accepted row about the join order already owns this " + "shape. Untagged: the point is the stage order, not " + "the input shape"), + Case("a_marker_phrase_ends_at_the_credential_too", + "Maria Kowalska z domu Nowak MA", + {"given": "Maria", "family": "Kowalska", "suffix": "MA", + "maiden": "Nowak"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="the rule is about the marker's CLAUSE, not about a " + "one-word marker: the Polish phrase entry (#434) " + "reaches the same walk and the same stop. 1.4.0 read " + "middle 'Kowalska z domu', family 'Nowak', suffix 'MA'", + shape=1), + Case("the_german_marker_ends_at_the_credential_too", + "Jane Doe geb. Smith MA", + {"given": "Jane", "family": "Doe", "suffix": "MA", + "maiden": "Smith"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="a second vocabulary spelling, for the same reason as " + "the row above: nothing here is about the word 'nee'", + shape=1), + Case("a_kyusei_clause_ends_at_the_credential", + "田中 太郎 旧姓 佐藤 MA", + {"given": "太郎", "family": "田中", "suffix": "MA", + "maiden": "佐藤"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="the CJK marker reaches the walk like any other, and " + "the Latin credential behind it is read by the Latin " + "rule -- is_one_case answers True for a caseless " + "script, so the lean is inert and the count decides. " + "tolerated rather than shape-tagged: a Latin " + "credential wrapped around a CJK name is a composed " + "form (the 2026-09-01 demotion), read best-effort", + tolerated=True), + # ---- #533 after a family comma: #531's slot is the reader ------ + Case("the_comma_reader_declines_the_title_cased_member", + "Doe, Jane nee Smith Ma", + {"given": "Jane", "family": "Doe", "maiden": "Smith Ma"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="after a family comma the comma has already settled " + "the count, so the reader is #531's slot and the " + "writing decides alone -- Title case in a mixed-case " + "name keeps the word. Reported either way. 1.4.0 read " + "suffix 'Ma'", + shape=2), + Case("the_comma_reader_takes_the_all_lower_member", + "Doe, Jane nee Smith ma", + {"given": "Jane", "family": "Doe", "suffix": "ma", + "maiden": "Smith"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="the row that makes the COMMA reader load-bearing " + "rather than decorative: the count-based reader " + "declines an all-lower member with two pieces to " + "spare, and #531's declines nothing but a particle. " + "Measured -- this name moves under the comma reader " + "and would not under the count. 'Doe, Jane ma' already " + "reads suffix 'ma', which is what the clause form now " + "agrees with", + shape=2), + Case("the_do_pair_after_a_comma_keeps_the_particle_spelling", + "Doe, Jane nee Smith do", + {"given": "Jane", "family": "Doe", "maiden": "Smith do"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="'do' is the one class member that is also particle " + "vocabulary, so after a comma the clause's trailing " + "slot, #531's slot and P6's attachment all want it -- " + "and the reading is #531's, unchanged and shared " + "through one predicate. Lean None plus a particle tag " + "means P6's word, so the clause keeps it and reports " + "the fork it consulted", + shape=2), + Case("the_do_pair_after_a_comma_reads_the_capitals", + "Doe, Jane nee Smith DO", + {"given": "Jane", "family": "Doe", "suffix": "DO", + "maiden": "Smith"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="the same word where the capitals speak: a positive " + "credential lean is the one spelling that outranks " + "P6's attachment (decisions.md#S2, 2026-09-18), so the " + "clause gives the word up. 1.4.0 read suffix 'DO'", + shape=2), + Case("the_no_comma_do_reads_by_the_count_instead", + "Jane Doe nee Smith do", + {"given": "Jane", "family": "Doe", "suffix": "do", + "maiden": "Smith"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="the row the issue flagged as needing a decision, and " + "it goes the other way from its comma twin BECAUSE the " + "reader is different: with no comma the reader is the " + "S2 peel over the view, which reads the count, and " + "P6 does not run at all. The comma-less 'John Doe do' " + "reads suffix 'do' today, and a clause must not change " + "how a word outside it reads -- so the two now agree", + shape=1), + Case("the_no_comma_dos_control_without_the_clause", + "John Doe do", + {"given": "John", "family": "Doe", "suffix": "do"}, + classification="fix(#289)", + ambiguities=("suffix-or-name",), + notes="the recorded control for the row above. Unchanged, " + "and it is what the clause form was measured against", + shape=1), + Case("the_comma_floor_keeps_a_member_a_particle_follows", + "Doe, Jane nee Smith MA do", + {"given": "Jane", "family": "Doe", "maiden": "Smith MA do"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="where #531's FLOOR earns its place, and a check that " + "asked only about 'MA' got this wrong: the take would " + "leave 'Jane MA do', where 'do' does not read as a " + "suffix (P6 keeps it) so #531's slot reads 'MA' as a " + "MIDDLE name -- releasing it from the clause would " + "move a word from one person's name into another's. " + "With the floor the clause keeps 'MA do' whole, and " + "reports the 'do' it kept. rules.md#S2's own 'Doe, " + "John MA do' clause is the reading this rests on", + shape=2), + Case("the_clamp_never_takes_the_first_word_after_the_marker", + "Doe, J. nee MA ba", + {"given": "J.", "family": "Doe", "suffix": "ba", + "maiden": "MA"}, + classification="fix(#533)", + ambiguities=("suffix-or-name", "suffix-or-name"), + notes="the floor is a CLAMP, not a veto, and this row is " + "why. The peel takes 'ba' and then 'MA', so the first " + "piece the peel took IS the only maiden word; a veto " + "that cancelled the stop whenever nothing would be " + "left handed 'ba' back to the clause too, giving " + "maiden 'MA ba' where 'Doe, J. ba' reads suffix 'ba'. " + "Clamped to the piece after the marker, the clause " + "keeps 'MA' and gives 'ba' up. TWO reports: the walk's " + "for the word it kept, assign's for the word it took", + shape=2), + Case("the_clamps_control_without_the_clause", + "Doe, J. ba", + {"given": "J.", "family": "Doe", "suffix": "ba"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="the recorded control for the clamp: this is what the " + "released 'ba' must go on reading, and it is #531's " + "slot doing the reading. 'ba' is the fifth listed " + "member, the one decisions.md#suffix-acronym-collisions " + "marked ambiguous rather than removing. 1.4.0 parity", + shape=2), + Case("the_clause_leaves_a_middle_initial_alone", + "Doe, Jane Q. nee Smith MA", + {"given": "Jane", "middle": "Q.", "family": "Doe", + "suffix": "MA", "maiden": "Smith"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="the member leaves the clause and 'Q.' stays the " + "middle initial it always was -- the stop reaches the " + "clause's trailing word, not the name in front of it", + shape=2), + Case("a_no_name_segment_moves_in_silence", + "Doe, Dr. nee Smith MA", + {"title": "Dr.", "family": "Doe", "suffix": "MA", + "maiden": "Smith"}, + classification="fix(#533)", + ambiguities=(), + notes="an ACCEPTED COST and a recorded silence. Once 'Smith' " + "leaves with the marker, segment 1 is 'Dr. MA' -- a " + "no-name segment, which the credential-run gate reads " + "whole without ever reaching #531's emitter, and the " + "first-post-comma emitter reads 'Dr.'. That is " + "_types.py's third pre-existing silence (a member with " + "no name word IN FRONT of it), now reachable through a " + "clause. 1.4.0 read given 'nee', middle 'Smith', " + "suffix 'MA'", + shape=2), + Case("the_bound_given_join_takes_the_released_member", + "Berg, abdul nee Jones MA", + {"given": "abdul MA", "family": "Berg", "maiden": "Jones"}, + classification="fix(#533)", + ambiguities=(), + notes="an ACCEPTED COST, and the control beside it says it " + "is not this change's defect: the take releases 'MA' " + "and P5's LENIENT post-comma join swallows it into the " + "bound-given pair before assign can read it -- exactly " + "as it does in 'Berg, abdul MA'. So the clause form " + "now AGREES with the clause-less form. Silent on both, " + "and _types.py already documents that silence (a chain " + "has swallowed the member into one piece, so there is " + "no lone member to ask about)", + shape=2), + Case("the_bound_given_joins_control_without_the_clause", + "Berg, abdul MA", + {"given": "abdul MA", "family": "Berg"}, + classification="parity", + ambiguities=(), + notes="the recorded control for the row above, and 1.4.0 " + "read it identically (first 'abdul MA', last 'Berg'). " + "Unchanged by this rule", + shape=2), + Case("no_trailing_rule_reads_the_family_segments_clause", + "Smith nee Jones, Jane MA", + {"given": "Jane", "family": "Smith", "suffix": "MA", + "maiden": "Jones"}, + classification="fix(#531)", + ambiguities=("suffix-or-name",), + notes="the clause is in segment 0 of a family comma, where " + "the comma has ALREADY named the family: those words " + "are family text and a stop would hand one to `family` " + "rather than to `suffix`, so no trailing rule is " + "consulted and the clause keeps what it has. The 'MA' " + "that does read as a credential here is in segment 1 " + "and is #531's slot's, not this rule's. Unchanged", + shape=2), + Case("no_trailing_rule_reads_a_third_comma_part", + "Smith, John, Jr nee Jones MA", + {"given": "John", "family": "Smith", "suffix": "Jr", + "maiden": "Jones MA"}, classification="fix(#274)", - notes="the silence no walk decides: the maiden marker claims " - "everything behind it, so the member is inside the " - "maiden clause and this slot never exists for it. The " - "comma-less 'John Smith nee Jones R.A.I.' is silent " - "for the same reason. 1.4.0 had no maiden routing and " - "read middle 'nee Smith', suffix 'MA'; 2.0.0 and 2.3.0 " - "read this maiden (measured 2026-09-19)"), + ambiguities=("comma-structure",), + notes="the other NONE reader, and the row that killed the " + "first prototype: a segment past the second comma is " + "read as credentials whole, so no trailing rule is " + "consulted, the clause keeps 'MA' -- and nothing " + "reports, because nothing was decided. Untagged: shape " + "2 is a TWO-part listing. Unchanged from 2.0.0", + ), + # ---- #533: the policy sweep, all core-only ----------------------- + Case("the_clause_reads_the_same_under_the_strict_comma_knob", + "Doe, Jane nee Smith MA", + {"given": "Jane", "family": "Doe", "suffix": "MA", + "maiden": "Smith"}, + policy=Policy(lenient_comma_suffixes=False), + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="the knob governs the LENIENT trailing predicate, " + "which this slot does not inherit, so the reading is " + "the default's"), + Case("the_clause_reads_the_same_under_family_first", + "Doe, Jane nee Smith MA", + {"given": "Jane", "family": "Doe", "suffix": "MA", + "maiden": "Smith"}, + policy=Policy(name_order=FAMILY_FIRST), + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="name_order does not enter it: the peel is " + "order-independent and the comma has already named the " + "family, so all three orders move the same names " + "(measured over the whole corpus under six policies). " + "UNTAGGED, and a shape 4 tag would be wrong -- this is " + "a comma listing, not the family-first arrangement"), + Case("the_clause_reads_the_same_under_ff_given_last", + "Doe, Jane nee Smith MA", + {"given": "Jane", "family": "Doe", "suffix": "MA", + "maiden": "Smith"}, + policy=Policy(name_order=FAMILY_FIRST_GIVEN_LAST), + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="the third order, for the same reason as the row above"), Case("a_leading_title_takes_the_slot_the_member_would_have_had", "Doe, Dr. MA Smith", {"title": "Dr.", "given": "MA", "middle": "Smith", diff --git a/tests/v2/pipeline/test_assign.py b/tests/v2/pipeline/test_assign.py index 8d89e8ad..0dbc10ab 100644 --- a/tests/v2/pipeline/test_assign.py +++ b/tests/v2/pipeline/test_assign.py @@ -6,6 +6,7 @@ from nameparser._pipeline._classify import classify from nameparser._pipeline._extract import extract_delimited from nameparser._pipeline._group import group +from nameparser._pipeline._pieces import credential_at_the_given_slot from nameparser._pipeline._segment import segment from nameparser._pipeline._state import ParseState from nameparser._pipeline._tokenize import tokenize @@ -755,7 +756,9 @@ def test_the_trailing_given_slot_steps_over_a_title_inside_the_run() -> None: def test_the_trailing_given_slot_reads_the_lean_three_ways() -> None: """credential / name / no-lean, the three answers listed_lean - gives, each landing where the rule says.""" + gives -- read for this slot through + `credential_at_the_given_slot`, which assign calls -- each + landing where the rule says.""" assert _by_role(_assigned("Doe, John MA", lexicon=Lexicon.default()), Role.SUFFIX) == "MA" # lean credential assert _by_role(_assigned("Doe, John Ma", lexicon=Lexicon.default()), @@ -768,7 +771,8 @@ def test_the_trailing_given_slot_falls_through_for_a_by_shape_member( ) -> None: """listed_lean returns None wherever the shape tag rides, so a by-shape member never leans and takes the positional reading -- - which at this slot is the credential.""" + which at this slot is the credential. Reached through + `credential_at_the_given_slot`, whose no-lean arm this is.""" out = _assigned("Doe, John X.Y.Z.", lexicon=Lexicon.default()) assert _by_role(out, Role.SUFFIX) == "X.Y.Z." @@ -792,6 +796,29 @@ def test_the_trailing_given_slot_ignores_the_two_segment_floor() -> None: assert _by_role(out, Role.MIDDLE) == "V" +def test_the_shared_given_slot_predicate_reads_the_lean_three_ways( +) -> None: + """credential_at_the_given_slot is the ONE place #531's reading + lives since #533, and its two callers sit in different stages -- + assign's walk over the given part, and the maiden walk's second + check. A test of its own is what keeps the three answers pinned + where a caller's test would only pin the reading it needs.""" + lex = Lexicon.default() + state = _assigned("Doe, John MA", lexicon=lex) + member = next(t for t in state.tokens if t.text == "MA") + assert credential_at_the_given_slot(member, False) is True + state = _assigned("Doe, John Ma", lexicon=lex) + member = next(t for t in state.tokens if t.text == "Ma") + assert credential_at_the_given_slot(member, False) is False + state = _assigned("doe, john ma", lexicon=lex) + member = next(t for t in state.tokens if t.text == "ma") + assert credential_at_the_given_slot(member, True) is True + # the particle carve-out: no positive lean, so P6 keeps the word + state = _assigned("Doe, John do", lexicon=lex) + member = next(t for t in state.tokens if t.text == "do") + assert credential_at_the_given_slot(member, False) is False + + def test_a_name_word_behind_the_member_ends_the_run() -> None: out = _assigned("Doe, John MA Smith", lexicon=Lexicon.default()) assert _by_role(out, Role.MIDDLE) == "MA Smith" diff --git a/tests/v2/pipeline/test_group.py b/tests/v2/pipeline/test_group.py index ca50e2c5..439762ff 100644 --- a/tests/v2/pipeline/test_group.py +++ b/tests/v2/pipeline/test_group.py @@ -6,14 +6,16 @@ from nameparser._lexicon import Lexicon from nameparser._pipeline._classify import classify from nameparser._pipeline._extract import extract_delimited, _maiden_marked -from nameparser._pipeline._group import group, marker_run_length +from nameparser._pipeline._group import ( + _group_segment, group, marker_run_length, +) from nameparser._pipeline._script_segment import script_segment from nameparser._pipeline._segment import segment -from nameparser._pipeline._state import ParseState +from nameparser._pipeline._state import ParseState, PendingAmbiguity from nameparser._pipeline._tokenize import tokenize from nameparser._pipeline._vocab import maiden_marker_run from nameparser._policy import Policy, Script -from nameparser._types import Role +from nameparser._types import AmbiguityKind, Role _LEX = Lexicon( titles=frozenset({"mr", "mrs", "secretary", "the", "state"}), @@ -935,25 +937,159 @@ def test_the_maiden_walk_stops_before_the_numeral_too() -> None: assert _piece_texts(out) == [["John", "V"]] -def test_the_maiden_walk_leaves_the_acronym_fork_to_assign() -> None: - # The bare-acronym fork counts pieces, and the walk removes the - # pieces it counted: peeled over the pieces as they stand, 'Ma' - # would be a credential with words to spare, but once 'Jones - # Smith' has left the name it is the family of a two-piece name. - # So the walk takes it as maiden text, as it always did, and - # stops only at the numeral fork. +def test_the_maiden_walk_keeps_the_acronym_its_writing_declines( +) -> None: + # The walk asks the acronym fork the way it asks the numeral one + # (#533): the peel over the pieces as they stand, then again over + # the name the take would leave. 'Ma' is Title case in a + # mixed-case name, so the peel declines it either way and the + # clause keeps it -- the same answer this test pinned when the + # fork was left to assign, for a different reason. The WRITING + # keeps the word, not the count. out = _grouped("John née Jones Smith Ma", lexicon=_AMBIGUOUS_LEX) assert [t.text for t in out.tokens if t.role is Role.MAIDEN] == \ ["Jones", "Smith", "Ma"] - # The numeral-only reading is what the acronym BETWEEN the maiden - # name and the numeral shows: the general peel would stop at the - # acronym, the re-ask would veto it, and the walk would take the - # numeral too (the test review's surviving mutant). + # and the capitals go the other way, which is what makes the row + # above a reading rather than a floor + out = _grouped("John née Jones Smith MA", lexicon=_AMBIGUOUS_LEX) + assert [t.text for t in out.tokens if t.role is Role.MAIDEN] == \ + ["Jones", "Smith"] + # The numeral half is untouched: the acronym BETWEEN the maiden + # name and the numeral still declines, and the walk still stops at + # the numeral fork (the test review's surviving mutant). out = _grouped("Jane Smith née Jones Ma V", lexicon=_AMBIGUOUS_LEX) assert [t.text for t in out.tokens if t.role is Role.MAIDEN] == \ ["Jones", "Ma"] +# -- #533: the acronym fork, the reader, the clamp and the emitter + +def _maiden_texts(state: ParseState) -> list[str]: + return [t.text for t in state.tokens if t.role is Role.MAIDEN] + + +def _suffix_forks(state: ParseState) -> list[str]: + return [a.detail for a in state.ambiguities + if a.kind is AmbiguityKind.SUFFIX_OR_NAME] + + +def test_the_clause_stops_before_a_credential_the_reader_takes( +) -> None: + out = _grouped("Jane Doe née Smith MA", lexicon=_AMBIGUOUS_LEX) + assert _maiden_texts(out) == ["Smith"] + + +def test_the_clause_never_gives_up_the_first_word_after_the_marker( +) -> None: + """Option 1's floor, as a CLAMP. A member standing alone after the + marker stays the maiden name; where the peel consumed that word + AND words behind it, only the first stays -- a veto that cancelled + the stop outright handed the words behind it back to the clause + too.""" + out = _grouped("Jane Doe née MA", lexicon=_AMBIGUOUS_LEX) + assert _maiden_texts(out) == ["MA"] + out = _grouped("Doe, J. née MA ba", + lexicon=_AMBIGUOUS_LEX.add( + suffix_acronyms={"ba"}, + suffix_acronyms_ambiguous={"ba"})) + assert _maiden_texts(out) == ["MA"] + + +def test_the_clamped_stop_may_land_on_no_member_and_declines() -> None: + """The clamp can move the stop onto a piece that is no class + member at all, and then the test declines and nothing changes -- + the walk stopping at that suffix word of its own accord.""" + out = _grouped("Jane Doe née MA Jr", lexicon=_AMBIGUOUS_LEX) + assert _maiden_texts(out) == ["MA"] + + +def test_the_view_check_asks_about_the_member_and_not_about_the_run( +) -> None: + """The take would leave 'JOHN MA PHD', whose peel takes 'PHD' and + then declines 'MA' for want of words to spare. A check asking + whether the reader takes SOMETHING answers yes there, and the + member becomes the FAMILY name.""" + # NOTE the accented marker: this module's `_LEX` ships + # maiden_markers={"née", "geb"} and NOT the unaccented "nee", so + # the plain spelling takes nothing at all here and the test would + # pass vacuously. cases.py's row of the same shape uses the + # DEFAULT vocabulary, where both spellings are markers. + lex = _AMBIGUOUS_LEX.add(suffix_acronyms={"phd"}) + out = _grouped("JOHN NÉE JONES SMITH MA PHD", lexicon=lex) + assert _maiden_texts(out) == ["JONES", "SMITH", "MA"] + assert len(_suffix_forks(out)) == 1 + + +def test_the_reader_is_none_in_the_family_segment() -> None: + """Segment 0 of a family comma: the comma has already named the + family, so no trailing rule reads those words and the clause keeps + them -- and nothing reports, nothing having been decided.""" + out = _grouped("Smith née Jones MA, Jane", lexicon=_AMBIGUOUS_LEX) + assert _maiden_texts(out) == ["Jones", "MA"] + assert _suffix_forks(out) == [] + + +def test_the_reader_is_none_in_a_third_comma_part() -> None: + """A segment past the second comma is read as credentials whole, + so no trailing rule is consulted there either.""" + out = _grouped("Smith, John, Jr née Jones MA", + lexicon=_AMBIGUOUS_LEX.add(suffix_words={"jr"})) + assert _maiden_texts(out) == ["Jones", "MA"] + assert _suffix_forks(out) == [] + + +def test_the_emitter_fires_on_the_last_maiden_piece_and_only_there( +) -> None: + """The word the trailing rule was asked about is the LAST piece of + the maiden name: everything behind it read as a suffix, which is + what let the peel reach it. A member with a name word behind it + was never asked.""" + out = _grouped("Jane Doe née Smith Ma", lexicon=_AMBIGUOUS_LEX) + assert _suffix_forks(out) == [ + "'Ma' ending the maiden name is also a post-nominal; the " + "maiden marker's clause keeps it rather than reading it as one"] + out = _grouped("Jane Doe née MA Smith", lexicon=_AMBIGUOUS_LEX) + assert _suffix_forks(out) == [] + + +def test_the_emitter_reports_a_by_shape_member_too() -> None: + """The emitter's gate reads EITHER tag, as the chain emitter's + does, so a member admitted by SHAPE reports even where the class + does not admit it and the peel declined to consume it -- it + records the word in `picks` and breaks, so the walk's own reading + gate is never asked about it (measured 2026-09-19).""" + out = _grouped("John Smith née Jones R.A.I.", + policy=Policy(unlisted_dotted_suffixes=False)) + assert _maiden_texts(out) == ["Jones", "R.A.I."] + assert len(_suffix_forks(out)) == 1 + + +def test_the_maiden_report_survives_the_family_comma_suppression( +) -> None: + """group() passes `None` for the chain emitter, deliberately -- + the comma fixed the family. The maiden fork is not that fork, so + it travels on its own channel and reports after a comma too.""" + out = _grouped("Doe, Jane née Smith Ma", lexicon=_AMBIGUOUS_LEX) + assert _maiden_texts(out) == ["Smith", "Ma"] + assert len(_suffix_forks(out)) == 1 + + +def test_an_unnamed_maiden_channel_falls_back_to_the_general_one( +) -> None: + """The second channel's DEFAULT, which group() never takes -- it + names both lists at every call. A caller of the segment function + that names only the one gets the one: the maiden fork reports + into it rather than into a list nobody reads, which is what makes + the parameter a routing choice rather than a second switch.""" + state = classify(segment(tokenize(extract_delimited(ParseState( + original="Jane Doe née Smith Ma", lexicon=_AMBIGUOUS_LEX, + policy=Policy()))))) + reported: list[PendingAmbiguity] = [] + _group_segment(state.segments[0], 0, state.tokens, + ambiguities=reported, one_case=state.one_case) + assert [a.kind for a in reported] == [AmbiguityKind.SUFFIX_OR_NAME] + + def test_a_marker_followed_only_by_the_numeral_is_just_a_word() -> None: # The peel is read from the marker, so 'née V' is two pieces and # the fork fires on the V: nothing follows the marker but a diff --git a/tests/v2/test_facade_cases.py b/tests/v2/test_facade_cases.py index c8804c75..0620e0cc 100644 --- a/tests/v2/test_facade_cases.py +++ b/tests/v2/test_facade_cases.py @@ -181,6 +181,20 @@ "the_trailing_slot_ignores_the_strict_comma_knob", "the_trailing_slot_reads_the_same_under_family_first", "the_trailing_slot_reads_the_same_under_ff_given_last", + # #533: the maiden clause's trailing credential. Five policy rows, + # each core-only for a field-by-field reason rather than a blanket + # one -- `unlisted_dotted_suffixes`, `unlisted_caps_suffixes`, + # `lenient_comma_suffixes` and `name_order` are each already in + # _UNTRANSLATED above, the first three having no v1 Constants + # manager at all and the fourth no v1 spelling. The DEFAULT-policy + # rows of the same change are ordinary rows and run here, which is + # what pins the facade parity that matters: HumanName("Jane Doe + # nee Smith MA") gives first Jane, last Doe, suffix MA. + "the_dotted_member_is_kept_with_the_switch_off_and_reports", + "the_caps_switch_reaches_the_clause", + "the_clause_reads_the_same_under_the_strict_comma_knob", + "the_clause_reads_the_same_under_family_first", + "the_clause_reads_the_same_under_ff_given_last", }) diff --git a/tests/v2/test_ledger_guards.py b/tests/v2/test_ledger_guards.py index 7c97f5d4..826cd2d9 100644 --- a/tests/v2/test_ledger_guards.py +++ b/tests/v2/test_ledger_guards.py @@ -2808,12 +2808,24 @@ def _claim(rule: dict) -> _Claim: # round's case rows added, '田中, 太郎 MA' and '김, 민준 MA', # both tolerated rows and so both in the radar corpus. Growth # into new corpus again; no role joined the list. + # 2026-09-19, #533: 137 -> 138. One new corpus name, + # '田中 太郎 旧姓 佐藤 MA' -- the tolerated row this change + # added, so radar corpus. Growth into new corpus again; no + # role joined the list. "fix(#271/#272/#298) native-script CJK: family-first order, hangul segmentation, the kana license and the dots": - _Claim(137, ('family', 'given', 'middle'), "94b869596b3e", None), + _Claim(138, ('family', 'given', 'middle'), "397de8444acc", None), + # 2026-09-19, #533: 33 -> 68. The count grew with the CORPUS + # rather than with the rule -- this change added 35 + # maiden-clause names as rules.md example lines and + # shape-tagged case rows, and this regex reaches every name + # carrying a marker. Read name by name against the regex; no + # role joined the list. "fix(#274) maiden markers consumed": - _Claim(33, ('family', 'maiden', 'middle'), "6f8bf7136b09", None), + _Claim(68, ('family', 'maiden', 'middle'), "19914ae9948e", None), + # 2026-09-19, #533: 5 -> 6, the same one new corpus name + # '田中 太郎 旧姓 佐藤 MA' as the CJK rule above. "fix(cjk-maiden-marker) maiden marker consumed, compounding with the CJK order flip": - _Claim(5, ('family', 'given', 'maiden', 'middle'), "bc0e10dd7ec8", None), + _Claim(6, ('family', 'given', 'maiden', 'middle'), "c72a3d5724f5", None), # 2026-09-18, #531: 13 -> 19. Six corpus names whose trailing # word is particle vocabulary this regex already lists -- # 'Doe, John DO', 'Doe, John Do', 'SMITH, JOHN DO', @@ -2835,8 +2847,14 @@ def _claim(rule: dict) -> _Claim: # the three, so it is ineligible and the fix(#380) rule the # round added explains it. Verified to be that name and no # other. + # 2026-09-19, #533: 21 -> 24. Three new corpus names whose + # trailing word is particle vocabulary this regex already + # lists -- 'Doe, Jane nee Smith DO', 'Doe, Jane nee Smith MA + # do' and 'Doe, Jane nee Smith do', the do pair this change + # added. Reach, not explanation: all three are the contest + # fix(#274) is now declared to outrank. "fix(#379) a tussenvoegsel after a family comma attaches to the family": - _Claim(21, ('family', 'middle'), "b017f45d04f5", None), + _Claim(24, ('family', 'middle'), "0c62afd7f400", None), "fix(#380) a trailing vd after a family comma is the tussenvoegsel, not a post-nominal": _Claim(2, ('family', 'suffix'), "ec0d45289dc1", None), # 279 -> 280 with #371, and the growth is corpus, not behavior: @@ -2891,8 +2909,12 @@ def _claim(rule: dict) -> _Claim: # inferred from the arithmetic. Reach again, not explanation: # four of the seven diff in fields this rule cannot admit and # are explained further down the file. + # 2026-09-19, #533: 343 -> 356, the thirteen new corpus names + # carrying a comma. Reach, not explanation: the regex is a + # comma and every family-comma row this change added matches + # it. "fix(comma-family) lone post-comma piece routes to suffix/title, not first": - _Claim(343, ('given', 'suffix', 'title'), "75e89d1dfd04", None), + _Claim(356, ('given', 'suffix', 'title'), "16c571069b0c", None), "fix(comma-family) a comma followed only by titles keeps the given/family split": _Claim(2, ('family', 'given'), "5bd9c6d96c38", None), "fix(comma-family) a comma followed only by titles keeps the given/family split, the C1 example": @@ -2940,8 +2962,10 @@ def _claim(rule: dict) -> _Claim: # comma names as the rule above. # 2026-09-19, #531 fix round: 336 -> 343, the same seven new # comma names as the rule above and for the same reason. + # 2026-09-19, #533: 343 -> 356, the same thirteen new comma + # names as the rule above and for the same reason. "fix(comma-precomma-family) pre-comma run reads as family, not given": - _Claim(343, ('family', 'given'), "75e89d1dfd04", None), + _Claim(356, ('family', 'given'), "16c571069b0c", None), "fix(#397) NOT WANTED: a trailing Catalan/Polish linking 'i' is read as a generation marker and the family is lost": _Claim(1, ('family', 'suffix'), "498602f3cfd0", None), "fix(suffix-delimiter-rendering) no-space delimiter core token kept whole": @@ -3018,8 +3042,13 @@ def _claim(rule: dict) -> _Claim: _Claim(1, ('family', 'given'), "efa60ca42d4a", None), "fix(nickname-typographic-pairs) two typographic quote spans read as one nickname set": _Claim(1, ('family', 'given', 'middle', 'nickname'), "3cf566c78800", None), + # 2026-09-19, #533: 1 -> 2. One new corpus name, 'Berg, + # abdul nee Jones MA' -- this change's record that P5's + # lenient post-comma join takes the released member before + # assign can read it, so the clause form agrees with the + # bare 'Berg, abdul MA'. Reach, not explanation. "fix(#411) the bound-given reserve stops counting words the maiden name takes": - _Claim(1, ('given', 'maiden', 'middle'), "7515923c9613", None), + _Claim(2, ('given', 'maiden', 'middle'), "b2500b6f6dfc", None), "fix(#400/#274) bound-given join and maiden consumption in one name": _Claim(1, ('family', 'given', 'maiden', 'middle'), "6bed6d349342", None), "fix(#411/S2) a declining bound-given join leaves the suffix reading after a family comma": @@ -3040,8 +3069,16 @@ def _claim(rule: dict) -> _Claim: _Claim(1, ('given', 'middle'), "2010cc79a34d", None), "fix(#424) the particle chain stops before the trailing numeral": _Claim(1, ('family', 'suffix'), "2c99162bc9cf", None), + # 2026-09-19, #533: 1 -> 2, and the new name looks like an + # accident and is not. This rule is an (?i) alternation + # carrying 'John n[ée]e Jones Smith Ma', so the example + # line this change added, 'John née Jones Smith MA', + # matches it case-insensitively -- correctly, both being + # the same given->family shape written two ways. Reach, + # not explanation: the caps spelling is the one whose + # reading this change MOVED, and its own rule explains it. "fix(#424/#445) accepted: the maiden walk keeps a bare acronym, and the lone name word is the family": - _Claim(1, ('family', 'given', 'maiden', 'middle', 'suffix'), "f2c6cd2e3001", None), + _Claim(2, ('family', 'given', 'maiden', 'middle', 'suffix'), "6728ea181c01", None), "fix(#424) an unlisted abbreviation is as transparent as a listed title to the leading particle, the P4 example": _Claim(1, ('family', 'given'), "42b69cf1b320", None), "fix(#424) accepted: the maiden walk keeps the numeral an initial before the marker vetoes": @@ -3148,8 +3185,12 @@ def _claim(rule: dict) -> _Claim: # what this number counts. "fix(initials-per-word) a connective run initials each word (facade, since 2.0.0)": _Claim(101, ('_initials',), "e91031622dca", ('DEFAULT',)), + # 2026-09-19, #533: 41 -> 43. Two new corpus names opening + # with a bound-given word, 'Berg, abdul MA' and 'Berg, abdul + # nee Jones MA' -- the P5 pair this change added to record + # that the clause form now agrees with the bare one. "fix(initials-per-word) a bound-given run initials each word (facade, since 2.0.0)": - _Claim(41, ('_initials',), "e99f56c955d5", ('DEFAULT',)), + _Claim(43, ('_initials',), "2a1c728285b8", ('DEFAULT',)), # 2026-09-18: 109 -> 110. One new corpus name, # 'john van der berg ma' -- rules.md#P2's one-case contrast, # and a particle chain like every other member. @@ -3405,7 +3446,10 @@ def _claim(rule: dict) -> _Claim: # 2026-09-19, #531 fix round: 20 -> 21, the same one new # corpus name as the 1.4.0 copy ('Doe, John MA do'), reached # on the trailing tussenvoegsel and explained by neither. - _Claim(21, ('_ambiguities', 'family', 'middle'), "b017f45d04f5", None), + # 2026-09-19, #533: 21 -> 24, the same three new corpus + # names as the 1.4.0 copy -- the do pair this change added + # after a family comma. + _Claim(24, ('_ambiguities', 'family', 'middle'), "0c62afd7f400", None), # 2026-09-18: 126 -> 131. Five corpus names arrived with # #289/#516's own case rows -- 'J.씨', 'John Smith 田.中.', # '毛泽东, MA', '田中 太郎, MA', '마틴 킹, MA' -- all of them @@ -3413,8 +3457,10 @@ def _claim(rule: dict) -> _Claim: # corpus, not a widened regex. # 2026-09-18, #531: 135 -> 137, the same two CJK comma forms # as the 1.4 twin. + # 2026-09-19, #533: 137 -> 138, the same one new tolerated + # corpus name as the 1.4.0 copy, '田中 太郎 旧姓 佐藤 MA'. "fix(#271/#272/#298) native-script CJK: family-first order, hangul segmentation, the kana license and the dots": - _Claim(137, ('_ambiguities', 'family', 'given', 'middle'), "94b869596b3e", None), + _Claim(138, ('_ambiguities', 'family', 'given', 'middle'), "397de8444acc", None), # 37 -> 35 with the same 2026-09-05 narrowing as the 1.4 twin, # whose entry carries the reason. Here the one name that # changed hands, '김민준 박사님', goes to the spaced rule @@ -3426,8 +3472,10 @@ def _claim(rule: dict) -> _Claim: _Claim(44, ('family', 'given', 'suffix'), "1587ce883dab", None), "fix(#307/#308/#320) spaced CJK postnominal honorific routed to suffix": _Claim(16, ('family', 'given', 'middle', 'suffix'), "6d390e518bd2", None), + # 2026-09-19, #533: 5 -> 6, the same one new tolerated + # corpus name as the CJK rule above. "fix(#309) 旧姓 maiden marker consumed, compounding with the CJK order flip": - _Claim(5, ('family', 'given', 'maiden', 'middle'), "bc0e10dd7ec8", None), + _Claim(6, ('family', 'given', 'maiden', 'middle'), "c72a3d5724f5", None), "fix(#272) nakaguro inside delimited content renders as a space, compounding with the CJK order flip": _Claim(1, ('family', 'given', 'nickname'), "d4069d459f23", None), "fix(#298) 间隔号 division changes the comma reading, sending the credential from title to suffix": @@ -3448,8 +3496,13 @@ def _claim(rule: dict) -> _Claim: _Claim(1, ('family', 'given'), "db724fb9c779", None), "fix(#272/#308) nakaguro division and a glued hangul honorific in one name": _Claim(1, ('family', 'given', 'middle', 'suffix'), "2fbf1a94f122", None), + # 2026-09-19, #533: 1 -> 2. One new corpus name, 'Berg, + # abdul nee Jones MA' -- this change's record that P5's + # lenient post-comma join takes the released member before + # assign can read it, so the clause form agrees with the + # bare 'Berg, abdul MA'. Reach, not explanation. "fix(#411) the bound-given reserve stops counting words the maiden name takes": - _Claim(1, ('given', 'maiden', 'middle'), "7515923c9613", None), + _Claim(2, ('given', 'maiden', 'middle'), "b2500b6f6dfc", None), "fix(#412) a connective join no longer absorbs the maiden marker beside it": _Claim(2, ('family', 'maiden'), "51c0eb36b5c5", None), "fix(#418) the connective carve-out counts the name the maiden clause leaves behind": @@ -3525,8 +3578,15 @@ def _claim(rule: dict) -> _Claim: _Claim(1, ('_ambiguities', 'family', 'given'), "42b69cf1b320", None), "fix(#424/#445) the maiden walk stops before the trailing numeral, and the lone name word is the family": _Claim(1, ('_ambiguities', 'family', 'given', 'maiden', 'suffix'), "cbe5bdd97317", None), + # 2026-09-19, #533: 6 -> 7, and as at 1.4.0 the new name + # looks like an accident and is not: this rule is an (?i) + # alternation carrying 'John n[ée]e Jones Smith Ma', which + # the example line 'John née Jones Smith MA' matches + # case-insensitively and correctly, the two being one + # given->family shape written two ways. Reach, not + # explanation. "fix(#445) a maiden marker makes the lone name word the family": - _Claim(6, ('family', 'given'), "f521c94c79fc", None), + _Claim(7, ('family', 'given'), "73c2f924d257", None), "fix(#445) the lone name word beside a marker a connective join no longer absorbs": _Claim(1, ('family', 'given', 'maiden', 'middle'), "52544a41dd62", None), "fix(#424) a marker followed only by the numeral is just a word": @@ -4037,7 +4097,10 @@ def _claim(rule: dict) -> _Claim: # 2026-09-19, #531 fix round: 20 -> 21, the same one new # corpus name as the 1.4.0 copy ('Doe, John MA do'), reached # on the trailing tussenvoegsel and explained by neither. - _Claim(21, ('_ambiguities', 'family', 'middle'), "b017f45d04f5", None), + # 2026-09-19, #533: 21 -> 24, the same three new corpus + # names as the 1.4.0 copy -- the do pair this change added + # after a family comma. + _Claim(24, ('_ambiguities', 'family', 'middle'), "0c62afd7f400", None), "fix(#424) an unlisted abbreviation is as transparent as a listed title to the leading particle": _Claim(1, ('_ambiguities', 'family', 'given'), "ca7b37af6cf8", None), "fix(#367) a title no longer displaces a leading particle out of the leading position": @@ -4052,8 +4115,13 @@ def _claim(rule: dict) -> _Claim: _Claim(11, ('given', 'middle'), "1eaed91fc574", None), "fix(#367) a title no longer displaces a leading never-given particle": _Claim(1, ('family', 'given'), "db724fb9c779", None), + # 2026-09-19, #533: 1 -> 2. One new corpus name, 'Berg, + # abdul nee Jones MA' -- this change's record that P5's + # lenient post-comma join takes the released member before + # assign can read it, so the clause form agrees with the + # bare 'Berg, abdul MA'. Reach, not explanation. "fix(#411) the bound-given reserve stops counting words the maiden name takes": - _Claim(1, ('given', 'maiden', 'middle'), "7515923c9613", None), + _Claim(2, ('given', 'maiden', 'middle'), "b2500b6f6dfc", None), "fix(#412) a connective join no longer absorbs the maiden marker beside it": _Claim(2, ('family', 'maiden'), "51c0eb36b5c5", None), "fix(#418) the connective carve-out counts the name the maiden clause leaves behind": @@ -4129,8 +4197,15 @@ def _claim(rule: dict) -> _Claim: _Claim(1, ('_ambiguities', 'family', 'given'), "42b69cf1b320", None), "fix(#424/#445) the maiden walk stops before the trailing numeral, and the lone name word is the family": _Claim(1, ('_ambiguities', 'family', 'given', 'maiden', 'suffix'), "cbe5bdd97317", None), + # 2026-09-19, #533: 6 -> 7, and as at 1.4.0 the new name + # looks like an accident and is not: this rule is an (?i) + # alternation carrying 'John n[ée]e Jones Smith Ma', which + # the example line 'John née Jones Smith MA' matches + # case-insensitively and correctly, the two being one + # given->family shape written two ways. Reach, not + # explanation. "fix(#445) a maiden marker makes the lone name word the family": - _Claim(6, ('family', 'given'), "f521c94c79fc", None), + _Claim(7, ('family', 'given'), "73c2f924d257", None), "fix(#445) the lone name word beside a marker a connective join no longer absorbs": _Claim(1, ('family', 'given', 'maiden', 'middle'), "52544a41dd62", None), "fix(#424) a marker followed only by the numeral is just a word": @@ -6010,6 +6085,16 @@ def test_a_rule_reaching_no_corpus_name_says_why_it_is_kept() -> None: #: re-measure before editing it. _ORDER_EXEMPTION_EFFECT: dict[str, list[tuple[str, str, int]]] = { "expected_since_1.4.0.toml": [ + # NEW on 2026-09-19 (#533): the three names carrying a maiden + # marker AND a trailing tussenvoegsel after a family comma + # ('Doe, Jane nee Smith do', its capital spelling, and 'Doe, + # Jane nee Smith MA do') are the first corpus names inside + # both regexes, so file order alone arbitrates the pair. The + # earlier rule now carries the [[change.precedes_narrower]] + # block saying why it wins; this control records the contest + # whether or not it is declared away. + ("fix(#274) maiden markers consumed", + "fix(#379) a tussenvoegsel after a family comma attaches to the family", 3), ("fix(comma-family) a comma followed only by titles keeps the given/family split, the C1 example", "fix(comma-precomma-family) pre-comma run reads as family, not given", 2), ("fix(#296) a credential-only comma string reads a name and its postnominal", diff --git a/tests/v2/test_properties.py b/tests/v2/test_properties.py index f37b0a50..bac10e7a 100644 --- a/tests/v2/test_properties.py +++ b/tests/v2/test_properties.py @@ -243,6 +243,162 @@ def test_the_comma_agreement_exceptions_are_all_still_exceptions( assert not parser.parse(comma_form).suffix, comma_form +#: The one-case-head exception class, and the recorded size of it. +#: Structural rather than a name list: a name whose OWN words are +#: written in one case, where the member's own writing is the only +#: contrast in the string, so the clause hides it. `own_words` stops +#: at the maiden marker (rules.md#P3), so a member inside the clause +#: cannot contribute the case contrast `one_case` is computed from -- +#: which is mechanisms.md's "the predicate keeps the judged token in +#: the span" failing structurally, the judged token never being in +#: the span at this slot. Accepted by Derek 2026-09-19 and recorded in +#: decisions.md#S2 as the M2 instance of #492's deferred question. +#: The COUNT is asserted beside the class because a structural +#: allowlist cannot notice a 115th member of it: 114 of 2016 pairs on +#: 2026-09-19, against 186 allowlisted and 984 failing before the +#: change. +_MAIDEN_AGREEMENT_EXCEPTIONS = 114 + + +def _one_case(text: str) -> bool | None: + """ParseState.one_case for a whole parse -- `is_one_case` over the + name's own words, which is exactly the span the allowlist asks + about.""" + return run(ParseState(original=text, lexicon=Lexicon.default(), + policy=Policy())).one_case + + +def test_a_maiden_clause_does_not_change_how_a_trailing_word_reads( +) -> None: + """#533's invariant: appending a maiden clause to a name must not + change the CLASS a trailing word is read in. + + A member of the ambiguous credential class ending a name that + carries a maiden clause, and the same member ending the same name + with the clause removed, must land in the same class -- credential + (role SUFFIX) or name (any other role). CLASS, not FIELD. + + The clause bodies are NAME WORDS ONLY on purpose: a suffix word or + a title inside the clause is a word the count reads, so removing + the clause changes the question rather than answering it ('J. nee + Smith Jr MA' against 'J. MA' is not a pair). + + Measured on this tree before #533: 984 of 2016 pairs disagreed + outside the allowlist. After: 0, and the allowlist holds exactly + its recorded size. + """ + members = ("ba", "do", "ed", "jd", "ma", "x.y.z.", "r.a.i.") + heads = ("Jane Doe", "Doe, Jane", "John", "J.", "Dr.", "Jane", + "Jane van der Berg", "JANE DOE", "jane doe", "DOE, JANE", + "doe, jane", "Jane Q. Doe", "Doe, Dr. Jane", "Doe, J.", + "Smith, Jane", "Jane Doe Jr.") + bodies = ("Smith", "Yo-Yo", "van der Berg", "Jones Smith", "MA", + "Ma") + parser = Parser() + + def side(text: str, word: str) -> str: + name = parser.parse(text) + hits = [t for t in name.tokens if t.text == word] + if not hits: + return "gone" + return ("credential" if hits[-1].role is Role.SUFFIX + else "name") + + pairs = allowed = 0 + failures = [] + for head in heads: + for body in bodies: + for base in members: + for word in (base.lower(), base.title(), base.upper()): + clause = f"{head} nee {body} {word}" + plain = f"{head} {word}" + pairs += 1 + if side(clause, word) == side(plain, word): + continue + if _one_case(clause) and not _one_case(plain): + allowed += 1 + continue + failures.append( + f"{clause!r} reads {side(clause, word)} but " + f"{plain!r} reads {side(plain, word)}") + assert not failures, ( + f"{len(failures)} of {pairs} pair(s) disagree outside the " + f"one-case-head class:\n" + "\n".join(failures[:15])) + assert allowed == _MAIDEN_AGREEMENT_EXCEPTIONS, ( + f"the one-case-head class holds {allowed} of {pairs} pairs, " + f"recorded as {_MAIDEN_AGREEMENT_EXCEPTIONS} on 2026-09-19; a " + f"structural allowlist cannot notice its own growth, so this " + f"count is the control. Re-record it deliberately, saying why") + + +def test_no_two_ambiguities_name_the_same_token_span() -> None: + """The maiden walk and assign's trailing peel both report at this + class, and group's particle-chain emitter stands beside them. None + of the three may report a word another already did -- the maiden + pieces are removed before the chain runs, and a member the reading + TOOK is out of the clause by the time roles are assigned. + + Per TOKEN, not per whole span: a first draft compared the + ambiguities' token tuples for equality, which sees two reports of + the same span and misses the likelier defect -- one report naming + `Smith MA` while another names `MA`. The check below is over the + spans the reports claim, so an OVERLAP fails it however the two + spans differ in length. + + Measured 2026-09-19: 0 over 20,412 parses, every particle shape + among them ('nee van der Berg Ma', 'nee de Ma', 'nee van Ma'). + A COUNT of the parses that carry two reports rides along, because + a comparison over one report is vacuous and nothing else would + say so: the grid as it stands has 2,160 of them, all from the two + particle heads, and a later edit that drops them fails here + rather than silently turning this into a test of nothing. + """ + members = ("ba", "do", "ed", "jd", "ma", "x.y.z.", "r.a.i.") + heads = ("Jane Doe", "Doe, Jane", "John", "J.", "Dr.", "Jane", + "Jane van der Berg", "JANE DOE", "jane doe", "DOE, JANE", + "doe, jane", "Jane Q. Doe", "Doe, Dr. Jane", "Doe, J.", + "Smith, Jane", "Jane Doe Jr.", "Doe, Jane van", + "Doe, Jane van der") + bodies = ("Smith", "Yo-Yo", "van der Berg", "de", "van", + "Jones Smith") + parsers = [Parser(), + Parser(policy=Policy(unlisted_dotted_suffixes=False)), + Parser(policy=Policy(unlisted_caps_suffixes=True))] + failures = [] + multi = 0 + for head in heads: + for body in bodies: + for base in members: + for word in (base.lower(), base.title(), base.upper()): + for marker in ("nee", "née", "geb."): + text = f"{head} {marker} {body} {word}" + for parser in parsers: + claimed: set[object] = set() + overlap = False + spans = [] + reports = parser.parse(text).ambiguities + multi += len(reports) > 1 + for a in reports: + # a synthetic token has no span; fall + # back on its identity so it cannot + # collide with another report's + span = {t.span if t.span is not None + else id(t) for t in a.tokens} + spans.append(sorted(map(str, span))) + overlap = overlap or bool(claimed & span) + claimed |= span + if overlap: + failures.append(f"{text!r}: {spans}") + assert not failures, ( + f"{len(failures)} parse(s) report one token twice:\n" + + "\n".join(failures[:15])) + assert multi == 2160, ( + f"{multi} of these parses carry more than one report, recorded " + f"as 2160 on 2026-09-19. A parse with one report cannot fail " + f"the check above, so this is what keeps the grid honest: move " + f"the number deliberately, and never to 0") + + @pytest.mark.parametrize("text", _FORK_CORPUS) def test_a_fork_is_never_reported_twice_on_a_real_name(text: str) -> None: state = run(ParseState(original=text, lexicon=Lexicon.default(), diff --git a/tools/differential/corpus_cjk_tolerated.jsonl b/tools/differential/corpus_cjk_tolerated.jsonl index b72e078a..da1f6590 100644 --- a/tools/differential/corpus_cjk_tolerated.jsonl +++ b/tools/differential/corpus_cjk_tolerated.jsonl @@ -13,6 +13,7 @@ "毛泽东 MA" "毛泽东, MA" "王先生, V." +"田中 太郎 旧姓 佐藤 MA" "田中 太郎, MA" "田中, 太郎 MA" "田中, 太郎さん" diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index d3c7b438..e07a24cf 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -85,6 +85,11 @@ "Jack Wei Ma" "Jack X.Y.I." "Jane (née Jones) Smith" +"Jane Doe nee MA" +"Jane Doe nee MA Smith" +"Jane Doe nee Smith MA" +"Jane Doe nee Smith Ma" +"Jane Doe nee Smith X.Y.Z." "Jane Smith (Nee)" "Jane Smith (Nee) (Jones)" "Jane Smith (née Jones)" @@ -146,6 +151,7 @@ "John Smith, Mr. Jr." "John Smith, PhD" "John Smith, V." +"John née Jones Smith MA" "John née Jones Smith Ma" "John née Jones Smith V" "John van Mc" diff --git a/tools/differential/corpus_shapes.jsonl b/tools/differential/corpus_shapes.jsonl index 1b330df7..b161642e 100644 --- a/tools/differential/corpus_shapes.jsonl +++ b/tools/differential/corpus_shapes.jsonl @@ -5,7 +5,12 @@ {"name": "J.A. K.D.", "shape": 1} {"name": "J.R.R. Tolkien", "shape": 1} {"name": "JACK MA", "shape": 1} +{"name": "JANE DOE NEE SMITH MA", "shape": 1} +{"name": "JANE DOE NEE YO-YO MA", "shape": 1} +{"name": "JANE YO-YO MA", "shape": 1} {"name": "JOHN E SMITH", "shape": 1} +{"name": "JOHN MA PHD", "shape": 1} +{"name": "JOHN NEE JONES SMITH MA PHD", "shape": 1} {"name": "JOHN SMITH MA", "shape": 1} {"name": "JOSE E MARIA SANTOS", "shape": 1} {"name": "JOSEP CAROD I ROVIRA", "shape": 1} @@ -16,10 +21,28 @@ {"name": "Jack Ma", "shape": 1} {"name": "Jack X.Y.I.", "shape": 1} {"name": "Jack X.Y.Z.", "shape": 1} +{"name": "Jane Doe geb. Smith MA", "shape": 1} +{"name": "Jane Doe nee MA", "shape": 1} +{"name": "Jane Doe nee MA Smith", "shape": 1} +{"name": "Jane Doe nee Smith 2", "shape": 1} +{"name": "Jane Doe nee Smith MA", "shape": 1} +{"name": "Jane Doe nee Smith MA JD", "shape": 1} +{"name": "Jane Doe nee Smith MA PhD", "shape": 1} +{"name": "Jane Doe nee Smith MA Prof.", "shape": 1} +{"name": "Jane Doe nee Smith MA y", "shape": 1} +{"name": "Jane Doe nee Smith Ma", "shape": 1} +{"name": "Jane Doe nee Smith Ma JD", "shape": 1} +{"name": "Jane Doe nee Smith PhD MA", "shape": 1} +{"name": "Jane Doe nee Smith Prof. MA", "shape": 1} +{"name": "Jane Doe nee Smith V", "shape": 1} +{"name": "Jane Doe nee Smith XYZ", "shape": 1} +{"name": "Jane Doe nee Smith do", "shape": 1} +{"name": "Jane Doe nee Yo-Yo Ma", "shape": 1} {"name": "Jean DUPONT", "shape": 1} {"name": "Jean Pierre DUPONT", "shape": 1} {"name": "John \"Jack\" Kennedy", "shape": 1} {"name": "John Doe JD.CPA", "shape": 1} +{"name": "John Doe do", "shape": 1} {"name": "John Jack Andrew Kennedy", "shape": 1} {"name": "John Smith", "shape": 1} {"name": "John Smith 1.4", "shape": 1} @@ -34,6 +57,7 @@ {"name": "John Smith X.Y.Z.", "shape": 1} {"name": "John Smith XYZ", "shape": 1} {"name": "John Smith Xyz.", "shape": 1} +{"name": "John Smith nee Jones R.A.I.", "shape": 1} {"name": "John V. Smith", "shape": 1} {"name": "John de Ma", "shape": 1} {"name": "John e Smith", "shape": 1} @@ -43,6 +67,7 @@ {"name": "Juan Garcia Y Lopez", "shape": 1} {"name": "Juan de la Vega", "shape": 1} {"name": "Lt.Gov. John Doe", "shape": 1} +{"name": "Maria Kowalska z domu Nowak MA", "shape": 1} {"name": "Md Abdul Karim", "shape": 1} {"name": "Minjun KIM", "shape": 1} {"name": "Mr MA", "shape": 1} @@ -52,6 +77,7 @@ {"name": "anh van do", "shape": 1} {"name": "anh van mc", "shape": 1} {"name": "jack ma", "shape": 1} +{"name": "jane doe nee smith ma", "shape": 1} {"name": "john e smith", "shape": 1} {"name": "john smith MA", "shape": 1} {"name": "john smith x.y.z.", "shape": 1} @@ -64,12 +90,24 @@ {"name": "محمد و علي", "shape": 1} {"name": "Beethoven, Ludwig van", "shape": 2} {"name": "Berg, Jan mc", "shape": 2} +{"name": "Berg, abdul MA", "shape": 2} +{"name": "Berg, abdul nee Jones MA", "shape": 2} {"name": "DOE, JOHN MA", "shape": 2} {"name": "DOE, MARY JO MA", "shape": 2} {"name": "Doe, Dr. John MA", "shape": 2} {"name": "Doe, Dr. MA", "shape": 2} {"name": "Doe, Dr. MA Smith", "shape": 2} +{"name": "Doe, Dr. nee Smith MA", "shape": 2} {"name": "Doe, J. MA", "shape": 2} +{"name": "Doe, J. ba", "shape": 2} +{"name": "Doe, J. nee MA ba", "shape": 2} +{"name": "Doe, Jane Q. nee Smith MA", "shape": 2} +{"name": "Doe, Jane nee Smith DO", "shape": 2} +{"name": "Doe, Jane nee Smith MA", "shape": 2} +{"name": "Doe, Jane nee Smith MA do", "shape": 2} +{"name": "Doe, Jane nee Smith Ma", "shape": 2} +{"name": "Doe, Jane nee Smith do", "shape": 2} +{"name": "Doe, Jane nee Smith ma", "shape": 2} {"name": "Doe, John A.", "shape": 2} {"name": "Doe, John BA", "shape": 2} {"name": "Doe, John DO", "shape": 2} @@ -106,6 +144,7 @@ {"name": "Smith Jr., A.B.", "shape": 2} {"name": "Smith Jr., John", "shape": 2} {"name": "Smith Jr., MA", "shape": 2} +{"name": "Smith nee Jones, Jane MA", "shape": 2} {"name": "Smith, A.B.", "shape": 2} {"name": "Smith, A.B.C.", "shape": 2} {"name": "Smith, Dr. John", "shape": 2} diff --git a/tools/differential/expected_since_1.4.0.toml b/tools/differential/expected_since_1.4.0.toml index 187f096c..0d6555d8 100644 --- a/tools/differential/expected_since_1.4.0.toml +++ b/tools/differential/expected_since_1.4.0.toml @@ -333,6 +333,27 @@ issue = "fix(#274) maiden markers consumed" name_regex = "(?i)(? Date: Sat, 19 Sep 2026 17:31:48 -0700 Subject: [PATCH 2/5] test(#533): record the ledger diffs at every baseline MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Written from the gate's own output rather than predicted beside the change. Over the corpus AS IT STOOD, #533 moves ONE name -- `John née Jones Smith Ma`, which keeps every field and gains a report -- so at 2.0.0 and 2.1.0 the fix(#445) rule that already carried it gains `_ambiguities` and nothing else, and at 2.2.0 and 2.3.0 it joins the report-only rule below. Everything else here is about the 29 corpus names this change's own rows admit. Those 29 are not all this change's, and each was measured at the parent 2f57ff21 as well as at its baseline to find out. Where the tree reads a name exactly as it read before #533, the rule names the change that DOES move it: fix(#531) gains `Doe, J. ba` and `Smith nee Jones, Jane MA`, fix(#436/#437) gains `Jane Doe nee Smith PhD MA` at the three 2.x baselines that still render its run with a comma, a fix(#424) rule carries `Jane Doe nee Smith V` at 2.0.0 and 2.1.0, and 1.4.0 gains two rules named for #274 -- one for the six names whose clause keeps a credential v1 read as a post-nominal, one for the name whose clause an UNAMBIGUOUS credential had already ended. A `fix(#533)` label on any of them would attribute a released reading to this change. One existing rule needed narrowing rather than widening. 1.4.0's `fix(#424/#445) accepted: the maiden walk keeps a bare acronym` was (?i)-anchored, so it reached `John née Jones Smith MA` as well as the Title-cased spelling -- and that name now does the opposite of what the rule says, the clause giving the MA up. The anchor is narrowed to the two spellings whose reading it describes, `JOHN NEE JONES SMITH MA PHD` joins it, and the caps-MA spelling gets a compound rule naming #445 and #533 together. The #533 rules themselves split by what they SAY rather than by field set: the clause giving a credential up; the clause keeping one and reporting it; the dotted by-shape spelling, which leaves v1 rather than restoring anything and is recorded as the accepted cost it is; the marker phrase; and the two names where the released member lands somewhere other than the trailing peel, silently, so their rule declares no `_ambiguities` to absorb a report with. At 1.4.0 the change's rows also create an order-decided contest, declared in commit 1 with a `precedes_narrower` block on the wider rule. Its `why` said "the whole of the diff is the marker leaving the name", which two of its three names falsify -- both also empty the `suffix` v1 read, which is in neither rule's fields -- so the block now says what the contest is over and where those two actually land. One name is a RESTORATION and its rule says so: `John Smith nee Jones R.A.I.` read suffix `R.A.I.` at 2.3.0, moved into the maiden name during this unreleased cycle when #516 retagged the token out of the certain-suffix class, and reads 2.3.0's way again here. It was in no corpus file, so no gate had ever compared it. The one radar name this change intended is classified rather than left on the radar unread, the way #531's caseless rule was: `田中 太郎 旧姓 佐藤 MA` at 2.1.0, 2.2.0 and 2.3.0, the marker's own rule already carrying it at 1.4.0 and 2.0.0. Radar unclassified goes 0/5/7/8/8 to 0/5/6/7/7. All five gates exit 0 with `unexplained: 0`: 1.4.0 corpus 1304 names; intentional diffs 508 2.0.0 corpus 1311 names; intentional diffs 455 2.1.0 corpus 1311 names; intentional diffs 367 2.2.0 corpus 1311 names; intentional diffs 223 2.3.0 corpus 1311 names; intentional diffs 108 No parser, case row or corpus file is touched. Co-Authored-By: Claude Fable 5.1 --- tests/v2/test_ledger_guards.py | 475 ++++++++++++++++++- tools/differential/expected_since_1.4.0.toml | 195 +++++++- tools/differential/expected_since_2.0.0.toml | 218 ++++++++- tools/differential/expected_since_2.1.0.toml | 242 +++++++++- tools/differential/expected_since_2.2.0.toml | 171 ++++++- tools/differential/expected_since_2.3.0.toml | 161 ++++++- 6 files changed, 1421 insertions(+), 41 deletions(-) diff --git a/tests/v2/test_ledger_guards.py b/tests/v2/test_ledger_guards.py index 826cd2d9..3f7587e5 100644 --- a/tests/v2/test_ledger_guards.py +++ b/tests/v2/test_ledger_guards.py @@ -1273,6 +1273,105 @@ def test_case_shape_ids_exist_in_the_inventory() -> None: ("Jean DUPONT", "Jean Pierre DUPONT", "Minjun KIM", "John Smith XYZ", "Smith, LEED AP", "John Smith, LEED AP", "Mr XXX"), + # #533's rules, and the probes are the boundaries each rule's own + # comment argues for. The class is a SLOT, so every one of these + # is a name carrying the same vocabulary in a position the rule + # does not reach -- which is what a regex written over the WORDS + # rather than over the names would claim. + # + # The movers must not reach the declining half ('Jane Doe nee + # Smith Ma', 'Doe, Jane nee Smith MA do'), the boundaries where + # no fork was consulted at all ('Jane Doe nee MA Smith', a name + # word behind the member; 'Jane Doe nee Smith MA Prof.', a + # trailing title breaking the peel; 'Smith, John, Jr nee Jones + # MA', past a second comma), the numeral shape another rule owns + # ('Jane Doe nee Smith V'), or the CLAUSE-LESS controls whose + # readings pre-date this change ('Doe, John MA' is #531's, + # 'John Smith MA' is S2's). + "fix(#533) a credential ending a maiden clause reads as a credential": + ("Jane Doe nee Smith Ma", "Doe, Jane nee Smith MA do", + "Jane Doe nee MA Smith", "Jane Doe nee Smith MA Prof.", + "Smith, John, Jr nee Jones MA", "Jane Doe nee Smith V", + "Doe, John MA", "John Smith MA", "Berg, abdul MA", + "Jane Doe nee Smith XYZ"), + # The declining half must not reach the movers, and the same + # boundaries apply from this side. + "fix(#533) the maiden clause reports the credential it keeps": + ("Jane Doe nee Smith MA", "Doe, Jane nee Smith MA", + "Jane Doe nee MA Smith", "Jane Doe nee Smith MA Prof.", + "Smith, John, Jr nee Jones MA", "Doe, John Ma", + "John Smith Ma", "Jane Doe nee Smith V"), + # The by-shape rule must stay on the one spelling the switch + # makes credential-SHAPED: not the listed member beside it, not + # the comma-less control, and not the same acronym one period + # short. + "fix(#533) accepted: an unlisted dotted acronym ending a maiden clause": + ("Jane Doe nee Smith MA", "John Doe X.Y.Z.", + "Doe, John X.Y.Z.", "Jane Doe nee Smith X.Y.Z", + "Jane Doe nee Smith XYZ"), + # The two names where the released member lands elsewhere must + # not reach their own clause-less controls -- which is the pair + # the rule's argument rests on -- nor the spellings that decline. + "fix(#533) accepted: the released credential lands where the segment it is released into puts it": + ("Berg, abdul MA", "Doe, Dr. MA", "Doe, Dr. nee Smith Ma", + "Berg, abdul nee Jones Ma", "Doe, Dr. nee Smith MA PhD"), + # The restoration is one name and one spelling: not the same + # acronym without the clause, and not a spelling the dotted gate + # reads differently. + "fix(#533) restores the suffix reading a 2.4 retag had moved into the maiden name": + ("John Smith R.A.I.", "John Smith nee Jones RAI", + "John Smith nee Jones R.A.I", "Jane Doe nee Smith R.A.I."), + # 1.4.0's pre-existing half must not reach the names whose clause + # this change made give the member up. + "fix(#274/#424) accepted: a maiden clause keeps a trailing credential": + ("Jane Doe nee Smith MA", "Doe, Jane nee Smith MA", + "Jane Doe nee MA Smith", "Jane Doe nee Smith V", + "John Smith MA"), + # ...and the name whose clause an UNAMBIGUOUS credential ended + # must not reach the ambiguous ones, in either word order. + "fix(#274/#436/#437) a clause the unambiguous credential ended": + ("Jane Doe nee Smith MA PhD", "Jane Doe nee Smith MA", + "Jane Doe nee Smith PhD", "John Doe PhD MA"), + # 1.4.0's #533 half must not reach the pre-existing one, and the + # capitals are the evidence: 'Jane Doe nee Smith Ma JD' is in the + # rule and 'Jane Doe nee Smith Ma' is not. + "fix(#533) the clause gives the credential up, and the run it joins renders as the writer spaced it": + ("Jane Doe nee Smith PhD MA", "Jane Doe nee Smith MA", + "Jane Doe nee Smith Ma", "Doe, J. nee MA", "Doe, J. ba"), + # The compound rules are case-SENSITIVE by construction, so each + # takes the other spellings of its own words as probes. + "fix(#445/#533) the clause gives the credential up, and the one name word it leaves is the family": + ("John née Jones Smith Ma", "JOHN NEE JONES SMITH MA PHD", + "John née Jones Smith", "Jane Doe nee Smith MA"), + "fix(#445/#533) a one-case clause keeps the credential, and the one name word it leaves is the family": + ("John nee Jones Smith MA PHD", "John née Jones Smith Ma", + "JOHN NEE JONES SMITH MA", "John née Jones Smith MA"), + # The phrase rule must not reach the bare phrase name the + # fix(#434) rules own, nor the n[ée]e spellings. + "fix(#434/#533) a marker PHRASE takes the maiden name": + ("Maria Kowalska z domu Nowak", "Maria Kowalska z domu Nowak Ma", + "Jane Doe nee Smith MA", "Maria Kowalska nee Nowak MA"), + # The native-script rule is one literal, and the widening it + # invites is a 旧姓 regex: the marker's other corpus names do not + # end in a member and must not be claimed, nor must the Latin + # spelling of the same reading. + "fix(#533) the maiden clause ends at the credential in a native-script name too": + ("田中 太郎 旧姓 佐藤", "Jane van der Berg 旧姓 Jones", + "田中 太郎 旧姓 佐藤 Ma", "Jane Doe nee Smith MA"), + # The numeral rule is #424's and must not reach the acronym + # shapes #533 moved, nor the two numeral names that already have + # rules of their own. + "fix(#424) the maiden walk stops before the trailing numeral, with the family name left standing": + ("Jane Doe nee Smith MA", "John née Jones Smith V", + "Jane Smith née V", "J. née Jones Smith V"), + # 2026-09-19, #533: the narrowed anchor's boundary is the + # spelling it no longer claims. 'John née Jones Smith MA' reads + # the opposite way and has fix(#445/#533); 'John nee Jones Smith + # MA PHD' is the mixed-case spelling of the name that joined, + # and reads the opposite way too. + "fix(#424/#445) accepted: the maiden walk keeps a bare acronym": + ("John née Jones Smith MA", "John nee Jones Smith MA PHD", + "JOHN NEE JONES SMITH MA"), } @@ -1981,14 +2080,28 @@ class _LatinCopy(NamedTuple): # what selects these names: 'John Smith Mc V' is here on a # particle that is also suffix vocabulary and 'Kenneth Clarke QC # MP' on two acronyms, and a member matching either set would - # reach names that do not move. One set, identical in all four - # ledgers. + # reach names that do not move. TWO sets since 2026-09-19 (#533): + # 'Jane Doe nee Smith PhD MA' joined the THREE 2.x spellings of + # that rule and not the 1.4.0 one, so the four ledgers no longer + # carry one identical set. The name is one more corpus name whose + # run those baselines write with a comma and the tree writes with + # a space; the SEPARATOR is still the subject, and the maiden + # clause in front of it decides nothing there -- #533 moves no + # role on it, measured against 2f57ff21. At 1.4.0 it does move + # roles, under a rule of that ledger's own. frozenset({"JOHN DOE PHD MD", "John Doe MD PhD", "John Smith MD PhD", "John Smith Mc V", "Kenneth Clarke QC MP", "Smith, John PhD I\\.", "The Rt Hon Kenneth Clarke QC MP, HMG", "Washington Jr\\. MD, Franklin", "abdul Smith Jr Ma", "abdul Smith Jr V"}), + frozenset({"JOHN DOE PHD MD", "Jane Doe nee Smith PhD MA", + "John Doe MD PhD", + "John Smith MD PhD", "John Smith Mc V", + "Kenneth Clarke QC MP", "Smith, John PhD I\\.", + "The Rt Hon Kenneth Clarke QC MP, HMG", + "Washington Jr\\. MD, Franklin", "abdul Smith Jr Ma", + "abdul Smith Jr V"}), # #346's movers, one corpus name per alternative -- a list of # names, not a copy of any wordlist, so there is no vocabulary for # it to drift from. The rule's subject is a SHAPE the vocabulary @@ -2080,12 +2193,18 @@ class _LatinCopy(NamedTuple): # Smith', where a name word behind the member ends the run, and # the whole `do` family, which rules.md#P6 keeps -- every one of # them measured not to move. _MUST_NOT_MATCH carries the probes. + # 2026-09-19, #533: two names joined that first set and it is + # fifteen. 'Doe, J. ba' and 'Smith nee Jones, Jane MA' are the + # same slot read the same way, and both read so at 2f57ff21 -- + # the corpus grew, the rule did not. Neither is a step toward a + # wordlist: 'ba' is already in the class the first thirteen draw + # on, and what selects the second is still the SLOT. frozenset({"DOE, JOHN MA", "DOE, MARY JO MA", r"Doe, Dr\. John MA", - r"Doe, J\. MA", "Doe, John BA", "Doe, John MA", - "Doe, John MA JD", "Doe, John MA Jr", + r"Doe, J\. MA", r"Doe, J\. ba", "Doe, John BA", + "Doe, John MA", "Doe, John MA JD", "Doe, John MA Jr", "Doe, John MA PhD", "Doe, John PhD MA", r"Doe, John Q\. MA", r"Doe, John X\.Y\.Z\.", - "doe, john ma"}), + "Smith nee Jones, Jane MA", "doe, john ma"}), frozenset({"Doe, John Ed", "Doe, John MA Ma", "Doe, John Ma", "Doe, Mary Jo Ma"}), # The third set. What selects these five is a WORD that is both @@ -2342,6 +2461,70 @@ class _LatinCopy(NamedTuple): # lopez', on the 2.0.0 per-word grouping -- are no part of the six. frozenset({"john e smith", "john e jones", "jones, john e", "JUAN Y GARCIA"}), + # #533's rules, one corpus name per alternative -- lists of names, + # not copies of any wordlist. What selects every one of them is a + # SLOT the vocabulary participates in only at one end: a member of + # the ambiguous credential class ending a maiden marker's clause. + # A member copying SUFFIX_ACRONYMS_AMBIGUOUS would reach 'Jane Doe + # nee MA Smith', where a name word behind the member ends the + # clause, and 'Jane Doe nee Smith MA Prof.', where a title breaks + # the peel before it -- both measured not to move. + # _MUST_NOT_MATCH carries those probes and the rest. + # + # Six sets, and the pairs differ by BASELINE rather than by + # reading: what a rule claims is what its baseline can see + # (#452), so 'Doe, J. nee MA ba' is a report at 2.0.0 and a mover + # at 2.2.0, and 'John née Jones Smith MA' needs a compound rule of + # its own below 2.2.0 where #445's move is still in the diff. + # + # The movers, at 2.2.0 and 2.3.0 (seventeen): + frozenset({r"Doe, J\. nee MA ba", r"Doe, Jane Q\. nee Smith MA", + "Doe, Jane nee Smith DO", "Doe, Jane nee Smith MA", + "Doe, Jane nee Smith ma", "JANE DOE NEE SMITH MA", + "JANE DOE NEE YO-YO MA", r"Jane Doe geb\. Smith MA", + "Jane Doe nee Smith MA", "Jane Doe nee Smith MA JD", + "Jane Doe nee Smith MA PhD", "Jane Doe nee Smith Ma JD", + r"Jane Doe nee Smith Prof\. MA", "Jane Doe nee Smith do", + "John née Jones Smith MA", "Maria Kowalska z domu Nowak MA", + "jane doe nee smith ma"}), + # and at 2.0.0 and 2.1.0 (fourteen). + frozenset({r"Doe, Jane Q\. nee Smith MA", "Doe, Jane nee Smith DO", + "Doe, Jane nee Smith MA", "Doe, Jane nee Smith ma", + "JANE DOE NEE SMITH MA", "JANE DOE NEE YO-YO MA", + r"Jane Doe geb\. Smith MA", "Jane Doe nee Smith MA", + "Jane Doe nee Smith MA JD", "Jane Doe nee Smith MA PhD", + "Jane Doe nee Smith Ma JD", + r"Jane Doe nee Smith Prof\. MA", "Jane Doe nee Smith do", + "jane doe nee smith ma"}), + # The names the clause KEEPS and now reports, at 2.2.0 and 2.3.0 + # (eight), + frozenset({"Doe, Jane nee Smith MA do", "Doe, Jane nee Smith Ma", + "Doe, Jane nee Smith do", "JOHN NEE JONES SMITH MA PHD", + "Jane Doe nee MA", "Jane Doe nee Smith Ma", + "Jane Doe nee Yo-Yo Ma", "John née Jones Smith Ma"}), + # and at 2.0.0 and 2.1.0 (five). + frozenset({r"Doe, J\. nee MA ba", "Doe, Jane nee Smith Ma", + "Jane Doe nee MA", "Jane Doe nee Smith Ma", + "Jane Doe nee Yo-Yo Ma"}), + # The two names where the released member lands somewhere other + # than the trailing peel. One set, shared by the 1.4.0 rule and + # the 2.2.0/2.3.0 one; at 2.0.0 and 2.1.0 the rule holds one name + # and has no alternation to declare. + frozenset({"Berg, abdul nee Jones MA", r"Doe, Dr\. nee Smith MA"}), + # The 1.4.0 pair and the 1.4.0 halves. The clause KEEPS the member + # in six of them, which at that baseline is not a report but the + # `suffix` v1 read emptying; it gives the member up in four, whose + # runs v1 also wrote with commas. The two-member set is the + # narrowed fix(#424/#445) anchor, which holds the two spellings + # whose reading that rule describes and no longer reaches the + # third by (?i). + frozenset({"Doe, Jane nee Smith MA do", "Doe, Jane nee Smith Ma", + "Doe, Jane nee Smith do", "Jane Doe nee MA", + "Jane Doe nee Smith Ma", "Jane Doe nee Yo-Yo Ma"}), + frozenset({r"Doe, J\. nee MA ba", "Jane Doe nee Smith MA JD", + "Jane Doe nee Smith MA PhD", "Jane Doe nee Smith Ma JD"}), + frozenset({"JOHN NEE JONES SMITH MA PHD", + "John n[ée]e Jones Smith Ma"}), }) def _unjustified_reach(name_regex: str, members: set[str]) -> list[str]: @@ -3069,16 +3252,20 @@ def _claim(rule: dict) -> _Claim: _Claim(1, ('given', 'middle'), "2010cc79a34d", None), "fix(#424) the particle chain stops before the trailing numeral": _Claim(1, ('family', 'suffix'), "2c99162bc9cf", None), - # 2026-09-19, #533: 1 -> 2, and the new name looks like an - # accident and is not. This rule is an (?i) alternation - # carrying 'John n[ée]e Jones Smith Ma', so the example - # line this change added, 'John née Jones Smith MA', - # matches it case-insensitively -- correctly, both being - # the same given->family shape written two ways. Reach, - # not explanation: the caps spelling is the one whose - # reading this change MOVED, and its own rule explains it. + # 2026-09-19, #533: 1 -> 2, and the two are not the two an + # earlier draft of this note predicted. The rule was an (?i) + # alternation carrying 'John n[ée]e Jones Smith Ma', so the + # example line this change added, 'John née Jones Smith MA', + # matched it case-insensitively -- and that spelling now + # reads the OPPOSITE way, the clause giving the MA up. The + # anchor is narrowed to the two spellings whose reading this + # rule describes ('John née Jones Smith Ma' and 'JOHN NEE + # JONES SMITH MA PHD'), the caps-MA spelling is carried by + # fix(#445/#533) below, and the digest moves although the + # count does not -- which is the one kind of growth a names + # count alone cannot see. "fix(#424/#445) accepted: the maiden walk keeps a bare acronym, and the lone name word is the family": - _Claim(2, ('family', 'given', 'maiden', 'middle', 'suffix'), "6728ea181c01", None), + _Claim(2, ('family', 'given', 'maiden', 'middle', 'suffix'), "348bf7114c9c", None), "fix(#424) an unlisted abbreviation is as transparent as a listed title to the leading particle, the P4 example": _Claim(1, ('family', 'given'), "42b69cf1b320", None), "fix(#424) accepted: the maiden walk keeps the numeral an initial before the marker vetoes": @@ -3337,6 +3524,50 @@ def _claim(rule: dict) -> _Claim: ('DEFAULT',)), "fix(#380) a trailing mc or do after a family comma is the tussenvoegsel, not a post-nominal": _Claim(5, ('family', 'suffix'), "13fad71c96d0", ('DEFAULT',)), + # #533's rows, and the rule is not #533's: six corpus names + # whose reading this change leaves alone, carrying the + # `suffix` v1 read beside the three fields fix(#274) declares. + # A number that grows here without a corpus row growing with + # it is this rule reaching a name whose clause DOES give the + # word up, which is the rule below. + "fix(#274/#424) accepted: a maiden clause keeps a trailing credential v1 read as a post-nominal": + _Claim(6, ('family', 'maiden', 'middle', 'suffix'), "447e065d4852", None), + # One corpus name, four roles. The rule is a compound of two + # released changes and #533 moves nothing on it, so a growth + # here is this rule reaching a name whose clause the + # ambiguous member ended instead. + "fix(#274/#436/#437) a clause the unambiguous credential ended, and the run it left renders with spaces": + _Claim(1, ('family', 'maiden', 'middle', 'suffix'), "51b39568664f", None), + # Four corpus names, four roles. The `suffix` is where the + # released member lands and `maiden` is what it left, so a + # widening that took one without the other would change the + # roles here before it reached the gate. + "fix(#533) the clause gives the credential up, and the run it joins renders as the writer spaced it": + _Claim(4, ('family', 'maiden', 'middle', 'suffix'), "3b2a2ee29ece", None), + # One corpus name. `suffix` is NOT in the 1.4.0 roles and + # that is the point of the row: v1 read the member as a + # post-nominal and so does the tree, which is what #533 + # restored here. + "fix(#445/#533) the clause gives the credential up, and the one name word it leaves is the family": + _Claim(1, ('family', 'given', 'maiden', 'middle'), "6403e56cea29", None), + # One corpus name. The by-shape member has no lean to read, + # so a growth here is the rule reaching a LISTED member -- + # a different reading under this rule's sentence. + "fix(#533) accepted: an unlisted dotted acronym ending a maiden clause leaves v1's family name": + _Claim(1, ('family', 'maiden', 'middle', 'suffix'), "0cc7bd35a9b3", None), + # One corpus name, three roles and no `suffix`: v1 read the + # trailing MA as a post-nominal and so does the tree. A + # `suffix` appearing here would mean the phrase rule had + # reached a name whose member moved. + "fix(#434/#533) a marker PHRASE takes the maiden name, and its clause ends at the credential": + _Claim(1, ('family', 'maiden', 'middle'), "bf8359f65c8a", None), + # Two corpus names at 1.4.0 and 2.2.0/2.3.0, one at 2.0.0 and + # 2.1.0 where fix(#411) still carries the bound-given half. + # No `_ambiguities` at any baseline, which is the row's own + # check: both readings are silent, and a report appearing + # here means one of them started speaking. + "fix(#533) accepted: the released credential lands where the segment it is released into puts it": + _Claim(2, ('given', 'maiden', 'middle', 'suffix'), "b3280b888d7c", None), }, "expected_since_2.0.0.toml": { # #436/#437's Latin alternation, first in every ledger. @@ -3344,7 +3575,7 @@ def _claim(rule: dict) -> _Claim: # SEPARATOR and no role, so a widening that took a role would # change the row here before it reached the gate. "fix(#436/#437) a space-separated post-nominal run renders with spaces, not commas": - _Claim(10, ('suffix',), "30f5314a2662", None), + _Claim(11, ('suffix',), "e665eae5c5df", None), # #449's six rules, second in every 2.x ledger. The # alternation reaches twenty-two corpus names and # `_ambiguities` alone: no role moves anywhere in this change, @@ -3586,7 +3817,7 @@ def _claim(rule: dict) -> _Claim: # given->family shape written two ways. Reach, not # explanation. "fix(#445) a maiden marker makes the lone name word the family": - _Claim(7, ('family', 'given'), "73c2f924d257", None), + _Claim(7, ('_ambiguities', 'family', 'given'), "73c2f924d257", None), "fix(#445) the lone name word beside a marker a connective join no longer absorbs": _Claim(1, ('family', 'given', 'maiden', 'middle'), "52544a41dd62", None), "fix(#424) a marker followed only by the numeral is just a word": @@ -3747,7 +3978,7 @@ def _claim(rule: dict) -> _Claim: # {family, suffix} from 2.2.0 on, which is P6's own shipping # date showing through. "fix(#531) a credential ending the given part of a family-comma listing reads as a credential": - _Claim(13, ('_ambiguities', 'middle', 'suffix'), "c577ebf116f2", ('DEFAULT',)), + _Claim(15, ('_ambiguities', 'middle', 'suffix'), "f17532bb4ff1", ('DEFAULT',)), "fix(#531) a member the writing declines keeps its name reading and reports the fork": _Claim(4, ('_ambiguities',), "c6d26d145ee1", ('DEFAULT',)), "fix(#531) capitals take the do collision from the family-comma particle attachment": @@ -3760,6 +3991,62 @@ def _claim(rule: dict) -> _Claim: # already read family 'mc Berg' and carry no rule for it. "fix(#380) a trailing mc after a family comma is the tussenvoegsel, not a post-nominal": _Claim(1, ('_ambiguities', 'family', 'suffix'), "105229644206", ('DEFAULT',)), + # The change's own movers: seventeen corpus names at 2.2.0 + # and 2.3.0, fourteen at 2.0.0 and 2.1.0 where three of them + # move roles those baselines cannot see and have rules of + # their own. `suffix`, `maiden` and `_ambiguities` together: + # the member lands in one, leaves the other and is reported, + # so a widening taking any of the three alone would change + # the roles here before it reached the gate. + "fix(#533) a credential ending a maiden clause reads as a credential": + _Claim(14, ('_ambiguities', 'maiden', 'suffix'), "2147fddf943a", None), + # The declining half: eight corpus names at 2.2.0 and 2.3.0, + # five at 2.0.0 and 2.1.0. `_ambiguities` alone, so a role + # appearing here is this rule reaching a name whose clause + # gave the member up. + "fix(#533) the maiden clause reports the credential it keeps": + _Claim(5, ('_ambiguities',), "a51ac503d15d", None), + # One corpus name. The by-shape member has no lean to read, + # so a growth here is the rule reaching a LISTED member -- + # a different reading under this rule's sentence. + "fix(#533) accepted: an unlisted dotted acronym ending a maiden clause is read by position": + _Claim(1, ('_ambiguities', 'maiden', 'suffix'), "0cc7bd35a9b3", None), + # Two corpus names at 1.4.0 and 2.2.0/2.3.0, one at 2.0.0 and + # 2.1.0 where fix(#411) still carries the bound-given half. + # No `_ambiguities` at any baseline, which is the row's own + # check: both readings are silent, and a report appearing + # here means one of them started speaking. + "fix(#533) accepted: the released credential lands where the segment it is released into puts it": + _Claim(1, ('maiden', 'suffix'), "4cc6318d5d14", None), + # One corpus name, `_ambiguities` alone at every 2.x + # baseline: the roles are the ones these releases already + # read, which is what makes the row a restoration. A role + # appearing here would mean the restoration had stopped + # being one. + "fix(#533) restores the suffix reading a 2.4 retag had moved into the maiden name": + _Claim(1, ('_ambiguities',), "0a7295a0cb19", None), + # One corpus name. `suffix` is NOT in the 1.4.0 roles and + # that is the point of the row: v1 read the member as a + # post-nominal and so does the tree, which is what #533 + # restored here. + "fix(#445/#533) the clause gives the credential up, and the one name word it leaves is the family": + _Claim(1, ('_ambiguities', 'family', 'given', 'maiden', 'suffix'), "6403e56cea29", None), + # One corpus name, at 2.0.0 and 2.1.0 only. Case-sensitive by + # construction, so a growth here is the anchor having picked + # up the mixed-case spelling, which reads the other way. + "fix(#445/#533) a one-case clause keeps the credential, and the one name word it leaves is the family": + _Claim(1, ('_ambiguities', 'family', 'given'), "a1816f915523", None), + # One corpus name, three roles and no `suffix`: v1 read the + # trailing MA as a post-nominal and so does the tree. A + # `suffix` appearing here would mean the phrase rule had + # reached a name whose member moved. + "fix(#434/#533) a marker PHRASE takes the maiden name, and its clause ends at the credential": + _Claim(1, ('family', 'maiden', 'middle'), "bf8359f65c8a", None), + # One corpus name, and the rule is #424's rather than #533's: + # the reading is unchanged on this tree since 2f57ff21. A + # growth here is the numeral rule reaching an acronym name. + "fix(#424) the maiden walk stops before the trailing numeral, with the family name left standing": + _Claim(1, ('_ambiguities', 'maiden', 'suffix'), "170a53c37765", None), }, # The 2.3 cycle's first rule, and a facade-only render fix: every # role is identical, so `_initials` alone. Reach and digest as in @@ -3771,7 +4058,7 @@ def _claim(rule: dict) -> _Claim: # SEPARATOR and no role, so a widening that took a role would # change the row here before it reached the gate. "fix(#436/#437) a space-separated post-nominal run renders with spaces, not commas": - _Claim(10, ('suffix',), "30f5314a2662", None), + _Claim(11, ('suffix',), "e665eae5c5df", None), # #449's six rules, second in every 2.x ledger. The # alternation reaches twenty-two corpus names and # `_ambiguities` alone: no role moves anywhere in this change, @@ -3973,13 +4260,53 @@ def _claim(rule: dict) -> _Claim: # shipped in 2.3; at 2.0.0 and 2.1.0 the same name moves # {middle, suffix} instead. "fix(#531) a credential ending the given part of a family-comma listing reads as a credential": - _Claim(13, ('_ambiguities', 'middle', 'suffix'), "c577ebf116f2", ('DEFAULT',)), + _Claim(15, ('_ambiguities', 'middle', 'suffix'), "f17532bb4ff1", ('DEFAULT',)), "fix(#531) a member the writing declines keeps its name reading and reports the fork": _Claim(4, ('_ambiguities',), "c6d26d145ee1", ('DEFAULT',)), "fix(#531) capitals take the do collision from the family-comma particle attachment": _Claim(1, ('_ambiguities', 'family', 'suffix'), "8ad64f404621", ('DEFAULT',)), "fix(#531) the trailing slot's positional reading reaches a caseless script": _Claim(2, ('_ambiguities', 'middle', 'suffix'), "8419a6f53c3e", ('DEFAULT',)), + # The change's own movers: seventeen corpus names at 2.2.0 + # and 2.3.0, fourteen at 2.0.0 and 2.1.0 where three of them + # move roles those baselines cannot see and have rules of + # their own. `suffix`, `maiden` and `_ambiguities` together: + # the member lands in one, leaves the other and is reported, + # so a widening taking any of the three alone would change + # the roles here before it reached the gate. + "fix(#533) a credential ending a maiden clause reads as a credential": + _Claim(17, ('_ambiguities', 'maiden', 'suffix'), "3051b7cde472", None), + # The declining half: eight corpus names at 2.2.0 and 2.3.0, + # five at 2.0.0 and 2.1.0. `_ambiguities` alone, so a role + # appearing here is this rule reaching a name whose clause + # gave the member up. + "fix(#533) the maiden clause reports the credential it keeps": + _Claim(8, ('_ambiguities',), "0b36d3482ba6", None), + # One corpus name. The by-shape member has no lean to read, + # so a growth here is the rule reaching a LISTED member -- + # a different reading under this rule's sentence. + "fix(#533) accepted: an unlisted dotted acronym ending a maiden clause is read by position": + _Claim(1, ('_ambiguities', 'maiden', 'suffix'), "0cc7bd35a9b3", None), + # Two corpus names at 1.4.0 and 2.2.0/2.3.0, one at 2.0.0 and + # 2.1.0 where fix(#411) still carries the bound-given half. + # No `_ambiguities` at any baseline, which is the row's own + # check: both readings are silent, and a report appearing + # here means one of them started speaking. + "fix(#533) accepted: the released credential lands where the segment it is released into puts it": + _Claim(2, ('given', 'maiden', 'suffix'), "b3280b888d7c", None), + # One corpus name, `_ambiguities` alone at every 2.x + # baseline: the roles are the ones these releases already + # read, which is what makes the row a restoration. A role + # appearing here would mean the restoration had stopped + # being one. + "fix(#533) restores the suffix reading a 2.4 retag had moved into the maiden name": + _Claim(1, ('_ambiguities',), "0a7295a0cb19", None), + # One corpus name, on the radar tier. A growth here is this + # rule reaching a second CJK-marker name, which would be a + # corpus row to look at rather than a widening to wave + # through. + "fix(#533) the maiden clause ends at the credential in a native-script name too": + _Claim(1, ('_ambiguities', 'maiden', 'suffix'), "6bab87214ddf", None), }, "expected_since_2.1.0.toml": { # #436/#437's Latin alternation, first in every ledger. @@ -3987,7 +4314,7 @@ def _claim(rule: dict) -> _Claim: # SEPARATOR and no role, so a widening that took a role would # change the row here before it reached the gate. "fix(#436/#437) a space-separated post-nominal run renders with spaces, not commas": - _Claim(10, ('suffix',), "30f5314a2662", None), + _Claim(11, ('suffix',), "e665eae5c5df", None), # #449's six rules, second in every 2.x ledger. The # alternation reaches twenty-two corpus names and # `_ambiguities` alone: no role moves anywhere in this change, @@ -4205,7 +4532,7 @@ def _claim(rule: dict) -> _Claim: # given->family shape written two ways. Reach, not # explanation. "fix(#445) a maiden marker makes the lone name word the family": - _Claim(7, ('family', 'given'), "73c2f924d257", None), + _Claim(7, ('_ambiguities', 'family', 'given'), "73c2f924d257", None), "fix(#445) the lone name word beside a marker a connective join no longer absorbs": _Claim(1, ('family', 'given', 'maiden', 'middle'), "52544a41dd62", None), "fix(#424) a marker followed only by the numeral is just a word": @@ -4355,7 +4682,7 @@ def _claim(rule: dict) -> _Claim: # {family, suffix} from 2.2.0 on, which is P6's own shipping # date showing through. "fix(#531) a credential ending the given part of a family-comma listing reads as a credential": - _Claim(13, ('_ambiguities', 'middle', 'suffix'), "c577ebf116f2", ('DEFAULT',)), + _Claim(15, ('_ambiguities', 'middle', 'suffix'), "f17532bb4ff1", ('DEFAULT',)), "fix(#531) a member the writing declines keeps its name reading and reports the fork": _Claim(4, ('_ambiguities',), "c6d26d145ee1", ('DEFAULT',)), "fix(#531) capitals take the do collision from the family-comma particle attachment": @@ -4368,6 +4695,68 @@ def _claim(rule: dict) -> _Claim: # already read family 'mc Berg' and carry no rule for it. "fix(#380) a trailing mc after a family comma is the tussenvoegsel, not a post-nominal": _Claim(1, ('_ambiguities', 'family', 'suffix'), "105229644206", ('DEFAULT',)), + # The change's own movers: seventeen corpus names at 2.2.0 + # and 2.3.0, fourteen at 2.0.0 and 2.1.0 where three of them + # move roles those baselines cannot see and have rules of + # their own. `suffix`, `maiden` and `_ambiguities` together: + # the member lands in one, leaves the other and is reported, + # so a widening taking any of the three alone would change + # the roles here before it reached the gate. + "fix(#533) a credential ending a maiden clause reads as a credential": + _Claim(14, ('_ambiguities', 'maiden', 'suffix'), "2147fddf943a", None), + # The declining half: eight corpus names at 2.2.0 and 2.3.0, + # five at 2.0.0 and 2.1.0. `_ambiguities` alone, so a role + # appearing here is this rule reaching a name whose clause + # gave the member up. + "fix(#533) the maiden clause reports the credential it keeps": + _Claim(5, ('_ambiguities',), "a51ac503d15d", None), + # One corpus name. The by-shape member has no lean to read, + # so a growth here is the rule reaching a LISTED member -- + # a different reading under this rule's sentence. + "fix(#533) accepted: an unlisted dotted acronym ending a maiden clause is read by position": + _Claim(1, ('_ambiguities', 'maiden', 'suffix'), "0cc7bd35a9b3", None), + # Two corpus names at 1.4.0 and 2.2.0/2.3.0, one at 2.0.0 and + # 2.1.0 where fix(#411) still carries the bound-given half. + # No `_ambiguities` at any baseline, which is the row's own + # check: both readings are silent, and a report appearing + # here means one of them started speaking. + "fix(#533) accepted: the released credential lands where the segment it is released into puts it": + _Claim(1, ('maiden', 'suffix'), "4cc6318d5d14", None), + # One corpus name, `_ambiguities` alone at every 2.x + # baseline: the roles are the ones these releases already + # read, which is what makes the row a restoration. A role + # appearing here would mean the restoration had stopped + # being one. + "fix(#533) restores the suffix reading a 2.4 retag had moved into the maiden name": + _Claim(1, ('_ambiguities',), "0a7295a0cb19", None), + # One corpus name. `suffix` is NOT in the 1.4.0 roles and + # that is the point of the row: v1 read the member as a + # post-nominal and so does the tree, which is what #533 + # restored here. + "fix(#445/#533) the clause gives the credential up, and the one name word it leaves is the family": + _Claim(1, ('_ambiguities', 'family', 'given', 'maiden', 'suffix'), "6403e56cea29", None), + # One corpus name, at 2.0.0 and 2.1.0 only. Case-sensitive by + # construction, so a growth here is the anchor having picked + # up the mixed-case spelling, which reads the other way. + "fix(#445/#533) a one-case clause keeps the credential, and the one name word it leaves is the family": + _Claim(1, ('_ambiguities', 'family', 'given'), "a1816f915523", None), + # One corpus name, three roles and no `suffix`: v1 read the + # trailing MA as a post-nominal and so does the tree. A + # `suffix` appearing here would mean the phrase rule had + # reached a name whose member moved. + "fix(#434/#533) a marker PHRASE takes the maiden name, and its clause ends at the credential": + _Claim(1, ('family', 'maiden', 'middle'), "bf8359f65c8a", None), + # One corpus name, and the rule is #424's rather than #533's: + # the reading is unchanged on this tree since 2f57ff21. A + # growth here is the numeral rule reaching an acronym name. + "fix(#424) the maiden walk stops before the trailing numeral, with the family name left standing": + _Claim(1, ('_ambiguities', 'maiden', 'suffix'), "170a53c37765", None), + # One corpus name, on the radar tier. A growth here is this + # rule reaching a second CJK-marker name, which would be a + # corpus row to look at rather than a widening to wave + # through. + "fix(#533) the maiden clause ends at the credential in a native-script name too": + _Claim(1, ('_ambiguities', 'maiden', 'suffix'), "6bab87214ddf", None), }, "expected_since_2.3.0.toml": { # #383/#479's three rules, the first this ledger carries. The @@ -4450,13 +4839,53 @@ def _claim(rule: dict) -> _Claim: # shipped in 2.3; at 2.0.0 and 2.1.0 the same name moves # {middle, suffix} instead. "fix(#531) a credential ending the given part of a family-comma listing reads as a credential": - _Claim(13, ('_ambiguities', 'middle', 'suffix'), "c577ebf116f2", ('DEFAULT',)), + _Claim(15, ('_ambiguities', 'middle', 'suffix'), "f17532bb4ff1", ('DEFAULT',)), "fix(#531) a member the writing declines keeps its name reading and reports the fork": _Claim(4, ('_ambiguities',), "c6d26d145ee1", ('DEFAULT',)), "fix(#531) capitals take the do collision from the family-comma particle attachment": _Claim(1, ('_ambiguities', 'family', 'suffix'), "8ad64f404621", ('DEFAULT',)), "fix(#531) the trailing slot's positional reading reaches a caseless script": _Claim(2, ('_ambiguities', 'middle', 'suffix'), "8419a6f53c3e", ('DEFAULT',)), + # The change's own movers: seventeen corpus names at 2.2.0 + # and 2.3.0, fourteen at 2.0.0 and 2.1.0 where three of them + # move roles those baselines cannot see and have rules of + # their own. `suffix`, `maiden` and `_ambiguities` together: + # the member lands in one, leaves the other and is reported, + # so a widening taking any of the three alone would change + # the roles here before it reached the gate. + "fix(#533) a credential ending a maiden clause reads as a credential": + _Claim(17, ('_ambiguities', 'maiden', 'suffix'), "3051b7cde472", None), + # The declining half: eight corpus names at 2.2.0 and 2.3.0, + # five at 2.0.0 and 2.1.0. `_ambiguities` alone, so a role + # appearing here is this rule reaching a name whose clause + # gave the member up. + "fix(#533) the maiden clause reports the credential it keeps": + _Claim(8, ('_ambiguities',), "0b36d3482ba6", None), + # One corpus name. The by-shape member has no lean to read, + # so a growth here is the rule reaching a LISTED member -- + # a different reading under this rule's sentence. + "fix(#533) accepted: an unlisted dotted acronym ending a maiden clause is read by position": + _Claim(1, ('_ambiguities', 'maiden', 'suffix'), "0cc7bd35a9b3", None), + # Two corpus names at 1.4.0 and 2.2.0/2.3.0, one at 2.0.0 and + # 2.1.0 where fix(#411) still carries the bound-given half. + # No `_ambiguities` at any baseline, which is the row's own + # check: both readings are silent, and a report appearing + # here means one of them started speaking. + "fix(#533) accepted: the released credential lands where the segment it is released into puts it": + _Claim(2, ('given', 'maiden', 'suffix'), "b3280b888d7c", None), + # One corpus name, `_ambiguities` alone at every 2.x + # baseline: the roles are the ones these releases already + # read, which is what makes the row a restoration. A role + # appearing here would mean the restoration had stopped + # being one. + "fix(#533) restores the suffix reading a 2.4 retag had moved into the maiden name": + _Claim(1, ('_ambiguities',), "0a7295a0cb19", None), + # One corpus name, on the radar tier. A growth here is this + # rule reaching a second CJK-marker name, which would be a + # corpus row to look at rather than a widening to wave + # through. + "fix(#533) the maiden clause ends at the credential in a native-script name too": + _Claim(1, ('_ambiguities', 'maiden', 'suffix'), "6bab87214ddf", None), }, } diff --git a/tools/differential/expected_since_1.4.0.toml b/tools/differential/expected_since_1.4.0.toml index 0d6555d8..53aa9b99 100644 --- a/tools/differential/expected_since_1.4.0.toml +++ b/tools/differential/expected_since_1.4.0.toml @@ -344,14 +344,24 @@ strict subset of these, so nothing but file order keeps them here. The discriminator is WHICH MOVE the diff is. What 1.4.0 read as middle 'nee Smith' plus a trailing post-nominal, the tree reads as a -maiden name: the whole of the diff is the marker leaving the name and -taking words with it, and the tussenvoegsel rule cannot say that -- -it describes a particle joining a family the comma already named, -which is a reading these three share with 1.4.0 rather than one that -moved. Read the other way round the narrower rule would stand ready to -explain a maiden/middle regression on every comma name ending in a -particle, which is the absorption this file's wide-first discipline -exists to prevent. +maiden name: the marker leaving the name and taking words with it is +what every one of the three diffs is ABOUT, and the tussenvoegsel +rule cannot say that -- it describes a particle joining a family the +comma already named, which is a reading these three share with 1.4.0 +rather than one that moved. Read the other way round the narrower +rule would stand ready to explain a maiden/middle regression on every +comma name ending in a particle, which is the absorption this file's +wide-first discipline exists to prevent. + +Not "the whole of the diff", which an earlier draft of this block +said and which two of the three names falsify: 'Doe, Jane nee Smith +do' and 'Doe, Jane nee Smith MA do' also empty the `suffix` v1 read, +and `suffix` is in neither rule's fields, so neither classifies them +at all -- they are below, under 'fix(#274/#424) accepted: a maiden +clause keeps a trailing credential v1 read as a post-nominal'. The +contest is over the names both regexes REACH, not the ones either +explains, which is what keeps it live with only 'Doe, Jane nee Smith +DO' actually landing here. """ [[change]] @@ -1949,7 +1959,24 @@ issue = "fix(#424/#445) accepted: the maiden walk keeps a bare acronym, and the # which is exactly how the gate reported it before this edit. The # name is #445's only mover whose diff is not the shape the fix(#445) # rule below carries, the suffix v1 read being the difference. -name_regex = "(?i)^john\\s+n[ée]e\\s+jones\\s+smith\\s+ma$" +# +# 2026-09-19, #533: NARROWED from (?i) to the two spellings whose +# reading this rule describes, and widened to the second of them. +# The (?i) anchor reached 'John née Jones Smith MA' as well, and +# that name now does the OPPOSITE of what this rule says -- the +# clause gives the MA up and it reads suffix 'MA' -- so leaving +# the anchor alone would have this rule explain a diff under the +# reasoning of the reading it lost. It is carried below, by +# 'fix(#445/#533) the clause gives the credential up, and the one +# name word it leaves is the family'. 'JOHN NEE JONES SMITH MA +# PHD' joins instead: one case, so no lean, and a suffix word +# BEHIND the member ends the peel before it reaches the member -- +# the clause keeps the MA and the lone name word is the family, +# which is this rule's sentence exactly. Its mixed-case spelling +# does not read that way ('John nee Jones Smith MA PHD' reads +# suffix 'MA PHD', measured), which is why the anchor cannot go +# back to (?i) to pick it up. +name_regex = "^(?:JOHN NEE JONES SMITH MA PHD|John n[ée]e Jones Smith Ma)$" fields = ["given", "middle", "family", "suffix", "maiden"] [[change]] @@ -3861,3 +3888,153 @@ issue = "fix(#380) a trailing mc or do after a family comma is the tussenvoegsel name_regex = "^(?:Berg, Jan mc|Doe, John Do|NASCIMENTO, EDSON ARANTES DO|Nascimento, Edson Arantes do|SMITH, JOHN DO)$" fields = ["family", "suffix"] orders = ["DEFAULT"] + +[[change]] +issue = "fix(#274/#424) accepted: a maiden clause keeps a trailing credential v1 read as a post-nominal" +# Six names whose reading this change does not touch: each reads on +# this tree exactly as it read at 2f57ff21, measured name by name, +# and the whole of the 1.4.0 diff is v1 having no maiden field. The +# marker takes the words after it, the trailing member among them -- +# rules.md#M2's Accepted reading, which has shipped since #274 and +# which #533 leaves standing wherever the WRITING declines the word +# (a Title-cased member in a mixed-case name: 'Jane Doe nee Smith +# Ma', 'Jane Doe nee Yo-Yo Ma', 'Doe, Jane nee Smith Ma'), wherever +# the member is the only word the marker would leave ('Jane Doe nee +# MA'), and wherever P6's attachment claims the word instead ('Doe, +# Jane nee Smith do', 'Doe, Jane nee Smith MA do'). What #533 adds on +# these six is a REPORT, and 1.4.0 has no surface to compare one +# against -- so nothing of this change is visible from here, and the +# rule is named for the change that is. +# +# Its own rule rather than a widening of fix(#274) above, and the +# reason is that rule's `fields`: it stops at maiden/middle/family +# because a Latin marker moves only those, while these six also move +# the `suffix` v1 read. Folding them in would have fix(#274) stand +# ready to explain a suffix regression on every name carrying a +# marker -- the trade 'fix(#424/#445) accepted: the maiden walk keeps +# a bare acronym' already refused for the same reason and records. +# +# Literal-anchored: the subject is a SLOT, and a regex for it would +# claim the names whose writing gives the word UP as readily as +# these. +name_regex = "^(?:Doe, Jane nee Smith MA do|Doe, Jane nee Smith Ma|Doe, Jane nee Smith do|Jane Doe nee MA|Jane Doe nee Smith Ma|Jane Doe nee Yo-Yo Ma)$" +fields = ["family", "maiden", "middle", "suffix"] + +[[change]] +issue = "fix(#274/#436/#437) a clause the unambiguous credential ended, and the run it left renders with spaces" +# 'Jane Doe nee Smith PhD MA', and #533 moves nothing on it either: +# the unambiguous PhD already ended the walk before this change, so +# the clause gave the MA up then as it does now -- maiden 'Smith', +# suffix 'PhD MA', the same reading as at 2f57ff21. Two changes meet +# in the 1.4.0 diff and one rule has to explain the whole of it: the +# marker leaving the name (#274) and R1's space-separated rendering +# of the run v1 wrote 'PhD, MA' (#436/#437). +# +# The name is the order-sensitivity #533 was filed about, seen from +# the side that never moved: 'Jane Doe nee Smith MA PhD' reads the +# same way now and did not before, and it is in the rule below. +name_regex = "^Jane Doe nee Smith PhD MA$" +fields = ["family", "maiden", "middle", "suffix"] + +[[change]] +issue = "fix(#533) the clause gives the credential up, and the run it joins renders as the writer spaced it" +# Four names where #533 changes which words the clause holds, and the +# 1.4.0 diff carries two more changes besides. What the marker takes +# now ends where a trailing credential begins (rules.md#M2, +# decisions.md#M2), so 'Jane Doe nee Smith MA PhD' reads suffix 'MA +# PhD' where the tree read maiden 'Smith MA' at 2f57ff21, and 'Jane +# Doe nee Smith MA JD' reads suffix 'MA JD' where it read maiden +# 'Smith MA JD'. 'Jane Doe nee Smith Ma JD' is the pair's other +# half: the Title-cased Ma stays in the clause and only the JD behind +# it leaves. 'Doe, J. nee MA ba' is the CLAMP -- a member that is the +# first word after the marker stays the maiden name, so maiden 'MA' +# is kept and suffix 'ba' is given up. +# +# The other two changes are the marker leaving the name (#274, which +# v1 has no field for) and R1's space-separated rendering of the run +# the member joins, which v1 wrote with commas: 'MA, PhD' -> 'MA +# PhD', 'MA, JD' -> 'MA JD', 'MA, ba' -> 'ba' (#436/#437). One rule +# explains the whole of each diff, which is why this is not three. +# +# NOT case-insensitive, and measured: 'Jane Doe nee Smith Ma JD' and +# 'Jane Doe nee Smith MA JD' read differently from each other -- the +# capitals are the evidence the lean reads -- so an (?i) anchor here +# would stand ready to explain either with the other's reasoning. +name_regex = "^(?:Doe, J\\. nee MA ba|Jane Doe nee Smith MA JD|Jane Doe nee Smith MA PhD|Jane Doe nee Smith Ma JD)$" +fields = ["family", "maiden", "middle", "suffix"] + +[[change]] +issue = "fix(#445/#533) the clause gives the credential up, and the one name word it leaves is the family" +# 'John née Jones Smith MA', whose diff has three causes from 1.4.0 +# and one rule has to explain the whole of it. v1 read given 'John', +# middle 'née Jones', last 'Smith', suffix 'MA'; the tree reads +# family 'John', suffix 'MA', maiden 'Jones Smith'. The marker takes +# the words after it (#274), rules.md#M4 makes the one name word the +# clause leaves the family (#445), and #533 is what takes the MA back +# out of the maiden name -- at 2f57ff21 this read maiden 'Jones Smith +# MA' with no suffix at all, so `suffix` was in the diff then and is +# not now. +# +# Its own rule because 'fix(#424/#445) accepted: the maiden walk +# keeps a bare acronym, and the lone name word is the family' above +# says the OPPOSITE of what this name now does, and until this commit +# its (?i) anchor claimed both spellings. That rule is narrowed to +# the spellings whose reading it describes; this one carries the +# spelling that gives the member up. +name_regex = "^John née Jones Smith MA$" +fields = ["family", "given", "maiden", "middle"] + +[[change]] +issue = "fix(#533) accepted: an unlisted dotted acronym ending a maiden clause leaves v1's family name" +# 'Jane Doe nee Smith X.Y.Z.': v1 read middle 'Doe nee Smith', last +# 'X.Y.Z.' and no suffix at all; the tree reads family 'Doe', suffix +# 'X.Y.Z.', maiden 'Smith'. A by-shape member carries no writing +# convention to read -- `listed_lean` returns None where the switch +# made the token credential-SHAPED and the vocabulary did not list it +# -- so the positional reading decides alone and the clause gives the +# word up. The same trade #531 recorded at the given part's slot and +# for the same reason: taken to agree with the comma-less 'John Doe +# X.Y.Z.' and rules.md#S3's shape rule, not to restore anything. +# +# Accepted, and the divergence from v1 is the whole of it: v1 made +# the dotted acronym the family name, and nothing here proposes to. +name_regex = "^Jane Doe nee Smith X\\.Y\\.Z\\.$" +fields = ["family", "maiden", "middle", "suffix"] + +[[change]] +issue = "fix(#434/#533) a marker PHRASE takes the maiden name, and its clause ends at the credential" +# 'Maria Kowalska z domu Nowak MA'. fix(#274)'s regex is the +# n[ée]e/geb alternation and cannot reach a phrase entry at all -- +# its own comment says why, at length -- and the fix(#434) rules +# below are anchored to the bare 'Maria Kowalska z domu Nowak'. +# Separate rather than a widening of that anchor: the diff here is +# the same three fields but the CAUSE is two changes, and a rule +# naming only #434 would attribute this change's stop to the +# vocabulary addition. +# +# What #533 does here is not in the field list: `maiden` moves for +# the phrase's sake either way, and what changed is the TEXT it +# holds -- 'Nowak MA' at 2f57ff21, 'Nowak' now, with the MA a suffix +# v1 also read as one. +name_regex = "^Maria Kowalska z domu Nowak MA$" +fields = ["family", "maiden", "middle"] + +[[change]] +issue = "fix(#533) accepted: the released credential lands where the segment it is released into puts it" +# Two names where the member leaves the clause and something other +# than the trailing peel catches it. Both are accepted and both are +# silent -- no report, because the rule that would raise one never +# reads the word. +# +# 'Doe, Dr. nee Smith MA': once 'Smith' leaves with the marker, +# segment 1 is 'Dr. MA' -- a no-name segment the credential-run gate +# reads whole (rules.md#C1) -- so the member is a suffix and the +# given part's own emitter never sees it. v1 read given 'nee', middle +# 'Smith'. 'Berg, abdul nee Jones MA': P5's lenient post-comma join +# takes the released member into the bound-given pair before assign +# can read it, exactly as it does in the clause-less 'Berg, abdul +# MA', so the clause form now AGREES with the bare form -- which is +# what makes this a consequence rather than a defect. v1 read given +# 'abdul nee', middle 'Jones', suffix 'MA'. +name_regex = "^(?:Berg, abdul nee Jones MA|Doe, Dr\\. nee Smith MA)$" +fields = ["given", "maiden", "middle", "suffix"] diff --git a/tools/differential/expected_since_2.0.0.toml b/tools/differential/expected_since_2.0.0.toml index c2ed9e4f..4ee895a2 100644 --- a/tools/differential/expected_since_2.0.0.toml +++ b/tools/differential/expected_since_2.0.0.toml @@ -54,7 +54,15 @@ # never claim. [[change]] issue = "fix(#436/#437) a space-separated post-nominal run renders with spaces, not commas" -name_regex = "^(?:JOHN DOE PHD MD|John Doe MD PhD|John Smith MD PhD|John Smith Mc V|Kenneth Clarke QC MP|Smith, John PhD I\\.|The Rt Hon Kenneth Clarke QC MP, HMG|Washington Jr\\. MD, Franklin|abdul Smith Jr Ma|abdul Smith Jr V)$" +# 2026-09-19, #533: one name joins the alternation and the rule +# does not change. 'Jane Doe nee Smith PhD MA' is a two-word +# credential run this baseline renders 'PhD, MA' and the tree +# renders 'PhD MA' -- the whole of its diff at this baseline, and +# the same diff at 2f57ff21, measured. #533 moves nothing on it: +# the unambiguous PhD already ended the walk, so the clause gave +# the MA up before this change as it does after it. What grew is +# the CORPUS, and the run rendering is what the diff is about. +name_regex = "^(?:JOHN DOE PHD MD|Jane Doe nee Smith PhD MA|John Doe MD PhD|John Smith MD PhD|John Smith Mc V|Kenneth Clarke QC MP|Smith, John PhD I\\.|The Rt Hon Kenneth Clarke QC MP, HMG|Washington Jr\\. MD, Franklin|abdul Smith Jr Ma|abdul Smith Jr V)$" fields = ["suffix"] # The six #449 rules go SECOND, not first: the rule above @@ -1629,8 +1637,16 @@ issue = "fix(#445) a maiden marker makes the lone name word the family" # rule of its own: this baseline already reads it maiden 'Jones Smith # Ma', so the acronym is no part of the diff. At 1.4.0, where v1 read # suffix 'Ma', it is the one name of this class that needs one. +# 2026-09-19, #533: `_ambiguities` joins the two, and no name +# leaves or enters. The roles are unchanged by that change -- +# 'John née Jones Smith Ma' has its Title-cased member declined +# by its own writing, exactly as the count declined it before -- +# but the fork is now CONSULTED at the end of the clause and +# REPORTED, and the gate compares `_ambiguities` as a field. The +# other five names carry no member and contribute nothing to the +# union, which is why this widening is one field and not a rule. name_regex = "(?i)^(?:Janey n[ée]e Jones|Jane n[ée]e Jones J\\. V|Jane n[ée]e Jones Smith|John n[ée]e Jones Smith Ma|Smith n[ée]e Jones|Smith n[ée]e Jones PhD)$" -fields = ["given", "family"] +fields = ["given", "family", "_ambiguities"] [[change]] issue = "fix(#445) the lone name word beside a marker a connective join no longer absorbs" @@ -2588,7 +2604,17 @@ issue = "fix(#531) a credential ending the given part of a family-comma listing # and nothing reports. _MUST_NOT_MATCH carries both, with the `do` # spellings P6 keeps and the third-comma-part name #144's restriction # still owns. -name_regex = "^(?:DOE, JOHN MA|DOE, MARY JO MA|Doe, Dr\\. John MA|Doe, J\\. MA|Doe, John BA|Doe, John MA|Doe, John MA JD|Doe, John MA Jr|Doe, John MA PhD|Doe, John PhD MA|Doe, John Q\\. MA|Doe, John X\\.Y\\.Z\\.|doe, john ma)$" +# 2026-09-19, #533: TWO names join the alternation and the rule does +# not change. 'Doe, J. ba' and 'Smith nee Jones, Jane MA' are this +# slot read exactly as the thirteen above are -- a lower-case member +# ending the given part after an initial, and the same member behind +# a maiden clause that sits in the FAMILY segment, where no trailing +# rule reads the clause's own tail and the given part's slot is +# reached as usual. Both read the same way at 2f57ff21, before #533, +# measured name by name: what grew is the CORPUS, not the rule, and +# a `fix(#533)` rule for them would attribute a #531 reading to the +# wrong change. +name_regex = "^(?:DOE, JOHN MA|DOE, MARY JO MA|Doe, Dr\\. John MA|Doe, J\\. MA|Doe, J\\. ba|Doe, John BA|Doe, John MA|Doe, John MA JD|Doe, John MA Jr|Doe, John MA PhD|Doe, John PhD MA|Doe, John Q\\. MA|Doe, John X\\.Y\\.Z\\.|Smith nee Jones, Jane MA|doe, john ma)$" fields = ["_ambiguities", "middle", "suffix"] orders = ["DEFAULT"] @@ -2706,3 +2732,189 @@ issue = "fix(#380) a trailing mc after a family comma is the tussenvoegsel, not name_regex = "^Berg, Jan mc$" fields = ["_ambiguities", "family", "suffix"] orders = ["DEFAULT"] + +[[change]] +issue = "fix(#533) a credential ending a maiden clause reads as a credential" +# The words a maiden marker takes now end where a trailing credential +# begins, where the rule that reads the name left standing reads the +# word as one -- both as the name is written and as the take would +# leave it (rules.md#M2, decisions.md#M2). `suffix` fills as `maiden` +# shortens, and the fork is reported: the end of a maiden clause was +# the last trailing position in the library where a member of the +# ambiguous credential class was read in silence. +# +# Fourteen names here, three fewer than the same rule carries at +# 2.2.0 and 2.3.0, and the three are a fact about THIS baseline +# rather than about the change. 'Maria Kowalska z domu Nowak MA' and +# 'John née Jones Smith MA' move roles this baseline has never read, +# so each needs a rule that explains the whole of its diff; 'Doe, J. +# nee MA ba' moves no role from here at all and is with the reports +# below. Width is baseline-relative, and so is membership. +# +# What is left is the class read from every side the corpora reach +# it: the bare member ('Jane Doe nee Smith MA', 'Doe, Jane nee Smith +# MA'); its one-case spellings, where the lean is inert and the +# POSITIONAL reading is not ('JANE DOE NEE SMITH MA', 'JANE DOE NEE +# YO-YO MA', 'jane doe nee smith ma', 'Doe, Jane nee Smith ma'); the +# German abbreviation ('Jane Doe geb. Smith MA'); a member beside +# another credential, where `suffix` GROWS rather than filling ('Jane +# Doe nee Smith MA JD', 'Jane Doe nee Smith MA PhD', 'Jane Doe nee +# Smith Ma JD' -- whose Title-cased Ma stays in the clause while the +# JD behind it leaves); past a trailing title, which the peel steps +# over ('Jane Doe nee Smith Prof. MA'); the slot reached past a +# middle initial ('Doe, Jane Q. nee Smith MA'); and the `do` +# collision, where the capitals decide ('Doe, Jane nee Smith DO' +# beside 'Jane Doe nee Smith do'). +# +# Literal-anchored for the reason fix(#531)'s rules give: the subject +# is a SLOT, so a regex for it would claim the names whose WRITING +# declines the word as readily as the ones it takes. +name_regex = "^(?:Doe, Jane Q\\. nee Smith MA|Doe, Jane nee Smith DO|Doe, Jane nee Smith MA|Doe, Jane nee Smith ma|JANE DOE NEE SMITH MA|JANE DOE NEE YO-YO MA|Jane Doe geb\\. Smith MA|Jane Doe nee Smith MA|Jane Doe nee Smith MA JD|Jane Doe nee Smith MA PhD|Jane Doe nee Smith Ma JD|Jane Doe nee Smith Prof\\. MA|Jane Doe nee Smith do|jane doe nee smith ma)$" +fields = ["_ambiguities", "maiden", "suffix"] + +[[change]] +issue = "fix(#533) the maiden clause reports the credential it keeps" +# The other direction of the same fork, and the whole of what these +# five names do at this baseline: no role moves and the decision +# starts being REPORTED. A Title-cased member in a mixed-case name is +# written the way a name is written, so the clause keeps it ('Jane +# Doe nee Smith Ma', 'Jane Doe nee Yo-Yo Ma', 'Doe, Jane nee Smith +# Ma'); a member that is the ONLY word after the marker stays the +# maiden name whatever its writing says ('Jane Doe nee MA'); and +# 'Doe, J. nee MA ba' is the CLAMP, whose roles this baseline already +# reads the way the tree does -- maiden 'MA', suffix 'ba' -- so from +# here the clamp is visible only as the report it raises. +# +# Reporting a DECLINED fork is #530's stated rule -- the report +# tracks the fork consulted, not the lean -- and it is why this is a +# rule of its own rather than a widening of the one above: `fields` +# is `_ambiguities` alone, so it cannot absorb a role diff on any of +# the five. +name_regex = "^(?:Doe, J\\. nee MA ba|Doe, Jane nee Smith Ma|Jane Doe nee MA|Jane Doe nee Smith Ma|Jane Doe nee Yo-Yo Ma)$" +fields = ["_ambiguities"] + +[[change]] +issue = "fix(#533) accepted: an unlisted dotted acronym ending a maiden clause is read by position" +# 'Jane Doe nee Smith X.Y.Z.'. A by-shape member carries no writing +# convention to read -- `listed_lean` returns None where the switch +# made the token credential-SHAPED and the vocabulary did not list it +# -- so the positional reading decides alone and the clause gives the +# word up. The same trade #516 recorded at its own slot and #531 at +# the given part's, taken to agree with the comma-less 'John Doe +# X.Y.Z.' and rules.md#S3's shape rule rather than to restore +# anything. +# +# Its own rule rather than one of the movers above, because the +# ARGUMENT is different: every name up there has a writing the lean +# could have read, and this one has none. +name_regex = "^Jane Doe nee Smith X\\.Y\\.Z\\.$" +fields = ["_ambiguities", "maiden", "suffix"] + +[[change]] +issue = "fix(#533) accepted: the released credential lands where the segment it is released into puts it" +# 'Doe, Dr. nee Smith MA', and it is SILENT, which is why it is not +# with the movers above: a rule declaring `_ambiguities` over a name +# that gains no report is the over-declaration #452's check names. +# Once 'Smith' leaves with the marker, segment 1 is 'Dr. MA' -- a +# no-name segment the credential-run gate reads whole (rules.md#C1) +# -- so the member is a suffix and the given part's own emitter never +# sees it. +# +# 'Berg, abdul nee Jones MA' is the other name of this shape and is +# NOT here: from this baseline its diff is P5's, and fix(#411) above +# carries it. It joins this rule at 2.2.0 and 2.3.0, where the +# reserve's own change has already shipped. +name_regex = "^Doe, Dr\\. nee Smith MA$" +fields = ["maiden", "suffix"] + +[[change]] +issue = "fix(#533) restores the suffix reading a 2.4 retag had moved into the maiden name" +# 'John Smith nee Jones R.A.I.', and it is a RESTORATION rather than +# a change: this baseline reads suffix 'R.A.I.' with maiden 'Jones', +# and so does the tree, so only the report diffs here. What moved in +# between was this unreleased 2.4 cycle. At 2.3.0 the token carried +# `vocab:suffix` and the maiden walk's suffix-piece test stopped at +# it; #516 retags it `shape:acronym` plus `vocab:suffix-ambiguous`, +# it stops being a suffix piece, and the walk took it -- maiden +# 'Jones R.A.I.' on this tree at 2f57ff21, measured. The name was in +# no corpus file, so no gate ever compared it and no 2.4.0 release +# bullet records the move. +name_regex = "^John Smith nee Jones R\\.A\\.I\\.$" +fields = ["_ambiguities"] + +[[change]] +issue = "fix(#445/#533) the clause gives the credential up, and the one name word it leaves is the family" +# 'John née Jones Smith MA', whose diff has TWO causes from here and +# one rule has to explain the whole of it -- the shape the +# fix(#424/#445) rules above already record for the numeral and for +# the Title-cased acronym. #445 moves the one name word the clause +# leaves into the family (given 'John' -> family 'John'), which this +# baseline has never read, and #533 takes the MA out of the maiden +# name and into `suffix`. Attributing either to the other would be +# false, and a rule declaring only one pair of fields would explain +# nothing while the name went UNEXPLAINED. +# +# Case-SENSITIVE, unlike the fix(#445) rule above, and measured: the +# capitals are what decide here, so 'John née Jones Smith Ma' keeps +# its member in the clause and 'John née Jones Smith MA' gives it up. +# An (?i) anchor would stand ready to explain either name's diff with +# the other's reasoning. +name_regex = "^John née Jones Smith MA$" +fields = ["_ambiguities", "family", "given", "maiden", "suffix"] + +[[change]] +issue = "fix(#445/#533) a one-case clause keeps the credential, and the one name word it leaves is the family" +# 'JOHN NEE JONES SMITH MA PHD'. One case, so no lean; a suffix word +# BEHIND the member ends the peel before it can reach the member, so +# the clause KEEPS the MA and only the clause's own emitter speaks. +# The role move is #445's -- the lone name word the clause leaves +# becomes the family -- and this baseline has not read it, so the +# diff is that move plus the new report. +# +# Its own rule rather than an alternative in the fix(#445) rule +# above, and case-sensitivity is the reason: that rule is (?i), and +# the mixed-case spelling of these words reads the other way. 'John +# nee Jones Smith MA PHD' gives the MA up and reads suffix 'MA PHD', +# measured 2026-09-19, so an (?i) alternative here would stand ready +# to explain a regression on a name whose reading is the opposite of +# this one's. +name_regex = "^JOHN NEE JONES SMITH MA PHD$" +fields = ["_ambiguities", "family", "given"] + +[[change]] +issue = "fix(#434/#533) a marker PHRASE takes the maiden name, and its clause ends at the credential" +# 'Maria Kowalska z domu Nowak MA'. The marker PHRASE is what moves +# the roles from here -- this baseline had no `z domu` entry, so it +# read middle 'Kowalska z domu', family 'Nowak' -- and the tree reads +# given 'Maria', family 'Kowalska', maiden 'Nowak'. #533's own effect +# on the name is invisible at this baseline: the trailing MA is a +# suffix on both sides, and both sides report it. What #533 changed +# is which words the maiden name holds, and `maiden` is already in +# the diff for the phrase's sake. +# +# Named for both changes because the maiden TEXT is the compound: at +# 2f57ff21 the same three fields moved and `maiden` read 'Nowak MA'. +# A rule naming only #434 would attribute this change's stop to the +# vocabulary addition; one naming only #533 would attribute the +# phrase to it. +name_regex = "^Maria Kowalska z domu Nowak MA$" +fields = ["family", "maiden", "middle"] + +[[change]] +issue = "fix(#424) the maiden walk stops before the trailing numeral, with the family name left standing" +# 'Jane Doe nee Smith V', and #533 has nothing to do with it: the +# name reads the same on this tree as it did at 2f57ff21, measured. +# The walk stops before the trailing roman numeral assign reads as +# the suffix, asked over the pieces as they stand and again over the +# view the take would leave -- #424's reading, stated at rules.md#M2. +# maiden 'Smith V' -> maiden 'Smith', suffix 'V', with the report +# #516 added at that slot. +# +# A rule of its own rather than an alternative in 'fix(#424/#445) the +# maiden walk stops before the trailing numeral, and the lone name +# word is the family' above: that rule's second half is the #445 move +# and this name has none -- family 'Doe' stands either way. What grew +# is the CORPUS: this change's rows put the numeral's clause shape in +# front of the gate for the first time at a baseline that can see it. +name_regex = "^Jane Doe nee Smith V$" +fields = ["_ambiguities", "maiden", "suffix"] diff --git a/tools/differential/expected_since_2.1.0.toml b/tools/differential/expected_since_2.1.0.toml index 2f6962de..c76311e1 100644 --- a/tools/differential/expected_since_2.1.0.toml +++ b/tools/differential/expected_since_2.1.0.toml @@ -78,7 +78,15 @@ # never claim. [[change]] issue = "fix(#436/#437) a space-separated post-nominal run renders with spaces, not commas" -name_regex = "^(?:JOHN DOE PHD MD|John Doe MD PhD|John Smith MD PhD|John Smith Mc V|Kenneth Clarke QC MP|Smith, John PhD I\\.|The Rt Hon Kenneth Clarke QC MP, HMG|Washington Jr\\. MD, Franklin|abdul Smith Jr Ma|abdul Smith Jr V)$" +# 2026-09-19, #533: one name joins the alternation and the rule +# does not change. 'Jane Doe nee Smith PhD MA' is a two-word +# credential run this baseline renders 'PhD, MA' and the tree +# renders 'PhD MA' -- the whole of its diff at this baseline, and +# the same diff at 2f57ff21, measured. #533 moves nothing on it: +# the unambiguous PhD already ended the walk, so the clause gave +# the MA up before this change as it does after it. What grew is +# the CORPUS, and the run rendering is what the diff is about. +name_regex = "^(?:JOHN DOE PHD MD|Jane Doe nee Smith PhD MA|John Doe MD PhD|John Smith MD PhD|John Smith Mc V|Kenneth Clarke QC MP|Smith, John PhD I\\.|The Rt Hon Kenneth Clarke QC MP, HMG|Washington Jr\\. MD, Franklin|abdul Smith Jr Ma|abdul Smith Jr V)$" fields = ["suffix"] # The six #449 rules go SECOND, not first: the rule above @@ -1550,8 +1558,16 @@ issue = "fix(#445) a maiden marker makes the lone name word the family" # rule of its own: this baseline already reads it maiden 'Jones Smith # Ma', so the acronym is no part of the diff. At 1.4.0, where v1 read # suffix 'Ma', it is the one name of this class that needs one. +# 2026-09-19, #533: `_ambiguities` joins the two, and no name +# leaves or enters. The roles are unchanged by that change -- +# 'John née Jones Smith Ma' has its Title-cased member declined +# by its own writing, exactly as the count declined it before -- +# but the fork is now CONSULTED at the end of the clause and +# REPORTED, and the gate compares `_ambiguities` as a field. The +# other five names carry no member and contribute nothing to the +# union, which is why this widening is one field and not a rule. name_regex = "(?i)^(?:Janey n[ée]e Jones|Jane n[ée]e Jones J\\. V|Jane n[ée]e Jones Smith|John n[ée]e Jones Smith Ma|Smith n[ée]e Jones|Smith n[ée]e Jones PhD)$" -fields = ["given", "family"] +fields = ["given", "family", "_ambiguities"] [[change]] issue = "fix(#445) the lone name word beside a marker a connective join no longer absorbs" @@ -2475,7 +2491,17 @@ issue = "fix(#531) a credential ending the given part of a family-comma listing # and nothing reports. _MUST_NOT_MATCH carries both, with the `do` # spellings P6 keeps and the third-comma-part name #144's restriction # still owns. -name_regex = "^(?:DOE, JOHN MA|DOE, MARY JO MA|Doe, Dr\\. John MA|Doe, J\\. MA|Doe, John BA|Doe, John MA|Doe, John MA JD|Doe, John MA Jr|Doe, John MA PhD|Doe, John PhD MA|Doe, John Q\\. MA|Doe, John X\\.Y\\.Z\\.|doe, john ma)$" +# 2026-09-19, #533: TWO names join the alternation and the rule does +# not change. 'Doe, J. ba' and 'Smith nee Jones, Jane MA' are this +# slot read exactly as the thirteen above are -- a lower-case member +# ending the given part after an initial, and the same member behind +# a maiden clause that sits in the FAMILY segment, where no trailing +# rule reads the clause's own tail and the given part's slot is +# reached as usual. Both read the same way at 2f57ff21, before #533, +# measured name by name: what grew is the CORPUS, not the rule, and +# a `fix(#533)` rule for them would attribute a #531 reading to the +# wrong change. +name_regex = "^(?:DOE, JOHN MA|DOE, MARY JO MA|Doe, Dr\\. John MA|Doe, J\\. MA|Doe, J\\. ba|Doe, John BA|Doe, John MA|Doe, John MA JD|Doe, John MA Jr|Doe, John MA PhD|Doe, John PhD MA|Doe, John Q\\. MA|Doe, John X\\.Y\\.Z\\.|Smith nee Jones, Jane MA|doe, john ma)$" fields = ["_ambiguities", "middle", "suffix"] orders = ["DEFAULT"] @@ -2593,3 +2619,213 @@ issue = "fix(#380) a trailing mc after a family comma is the tussenvoegsel, not name_regex = "^Berg, Jan mc$" fields = ["_ambiguities", "family", "suffix"] orders = ["DEFAULT"] + +[[change]] +issue = "fix(#533) a credential ending a maiden clause reads as a credential" +# The words a maiden marker takes now end where a trailing credential +# begins, where the rule that reads the name left standing reads the +# word as one -- both as the name is written and as the take would +# leave it (rules.md#M2, decisions.md#M2). `suffix` fills as `maiden` +# shortens, and the fork is reported: the end of a maiden clause was +# the last trailing position in the library where a member of the +# ambiguous credential class was read in silence. +# +# Fourteen names here, three fewer than the same rule carries at +# 2.2.0 and 2.3.0, and the three are a fact about THIS baseline +# rather than about the change. 'Maria Kowalska z domu Nowak MA' and +# 'John née Jones Smith MA' move roles this baseline has never read, +# so each needs a rule that explains the whole of its diff; 'Doe, J. +# nee MA ba' moves no role from here at all and is with the reports +# below. Width is baseline-relative, and so is membership. +# +# What is left is the class read from every side the corpora reach +# it: the bare member ('Jane Doe nee Smith MA', 'Doe, Jane nee Smith +# MA'); its one-case spellings, where the lean is inert and the +# POSITIONAL reading is not ('JANE DOE NEE SMITH MA', 'JANE DOE NEE +# YO-YO MA', 'jane doe nee smith ma', 'Doe, Jane nee Smith ma'); the +# German abbreviation ('Jane Doe geb. Smith MA'); a member beside +# another credential, where `suffix` GROWS rather than filling ('Jane +# Doe nee Smith MA JD', 'Jane Doe nee Smith MA PhD', 'Jane Doe nee +# Smith Ma JD' -- whose Title-cased Ma stays in the clause while the +# JD behind it leaves); past a trailing title, which the peel steps +# over ('Jane Doe nee Smith Prof. MA'); the slot reached past a +# middle initial ('Doe, Jane Q. nee Smith MA'); and the `do` +# collision, where the capitals decide ('Doe, Jane nee Smith DO' +# beside 'Jane Doe nee Smith do'). +# +# Literal-anchored for the reason fix(#531)'s rules give: the subject +# is a SLOT, so a regex for it would claim the names whose WRITING +# declines the word as readily as the ones it takes. +name_regex = "^(?:Doe, Jane Q\\. nee Smith MA|Doe, Jane nee Smith DO|Doe, Jane nee Smith MA|Doe, Jane nee Smith ma|JANE DOE NEE SMITH MA|JANE DOE NEE YO-YO MA|Jane Doe geb\\. Smith MA|Jane Doe nee Smith MA|Jane Doe nee Smith MA JD|Jane Doe nee Smith MA PhD|Jane Doe nee Smith Ma JD|Jane Doe nee Smith Prof\\. MA|Jane Doe nee Smith do|jane doe nee smith ma)$" +fields = ["_ambiguities", "maiden", "suffix"] + +[[change]] +issue = "fix(#533) the maiden clause reports the credential it keeps" +# The other direction of the same fork, and the whole of what these +# five names do at this baseline: no role moves and the decision +# starts being REPORTED. A Title-cased member in a mixed-case name is +# written the way a name is written, so the clause keeps it ('Jane +# Doe nee Smith Ma', 'Jane Doe nee Yo-Yo Ma', 'Doe, Jane nee Smith +# Ma'); a member that is the ONLY word after the marker stays the +# maiden name whatever its writing says ('Jane Doe nee MA'); and +# 'Doe, J. nee MA ba' is the CLAMP, whose roles this baseline already +# reads the way the tree does -- maiden 'MA', suffix 'ba' -- so from +# here the clamp is visible only as the report it raises. +# +# Reporting a DECLINED fork is #530's stated rule -- the report +# tracks the fork consulted, not the lean -- and it is why this is a +# rule of its own rather than a widening of the one above: `fields` +# is `_ambiguities` alone, so it cannot absorb a role diff on any of +# the five. +name_regex = "^(?:Doe, J\\. nee MA ba|Doe, Jane nee Smith Ma|Jane Doe nee MA|Jane Doe nee Smith Ma|Jane Doe nee Yo-Yo Ma)$" +fields = ["_ambiguities"] + +[[change]] +issue = "fix(#533) accepted: an unlisted dotted acronym ending a maiden clause is read by position" +# 'Jane Doe nee Smith X.Y.Z.'. A by-shape member carries no writing +# convention to read -- `listed_lean` returns None where the switch +# made the token credential-SHAPED and the vocabulary did not list it +# -- so the positional reading decides alone and the clause gives the +# word up. The same trade #516 recorded at its own slot and #531 at +# the given part's, taken to agree with the comma-less 'John Doe +# X.Y.Z.' and rules.md#S3's shape rule rather than to restore +# anything. +# +# Its own rule rather than one of the movers above, because the +# ARGUMENT is different: every name up there has a writing the lean +# could have read, and this one has none. +name_regex = "^Jane Doe nee Smith X\\.Y\\.Z\\.$" +fields = ["_ambiguities", "maiden", "suffix"] + +[[change]] +issue = "fix(#533) accepted: the released credential lands where the segment it is released into puts it" +# 'Doe, Dr. nee Smith MA', and it is SILENT, which is why it is not +# with the movers above: a rule declaring `_ambiguities` over a name +# that gains no report is the over-declaration #452's check names. +# Once 'Smith' leaves with the marker, segment 1 is 'Dr. MA' -- a +# no-name segment the credential-run gate reads whole (rules.md#C1) +# -- so the member is a suffix and the given part's own emitter never +# sees it. +# +# 'Berg, abdul nee Jones MA' is the other name of this shape and is +# NOT here: from this baseline its diff is P5's, and fix(#411) above +# carries it. It joins this rule at 2.2.0 and 2.3.0, where the +# reserve's own change has already shipped. +name_regex = "^Doe, Dr\\. nee Smith MA$" +fields = ["maiden", "suffix"] + +[[change]] +issue = "fix(#533) restores the suffix reading a 2.4 retag had moved into the maiden name" +# 'John Smith nee Jones R.A.I.', and it is a RESTORATION rather than +# a change: this baseline reads suffix 'R.A.I.' with maiden 'Jones', +# and so does the tree, so only the report diffs here. What moved in +# between was this unreleased 2.4 cycle. At 2.3.0 the token carried +# `vocab:suffix` and the maiden walk's suffix-piece test stopped at +# it; #516 retags it `shape:acronym` plus `vocab:suffix-ambiguous`, +# it stops being a suffix piece, and the walk took it -- maiden +# 'Jones R.A.I.' on this tree at 2f57ff21, measured. The name was in +# no corpus file, so no gate ever compared it and no 2.4.0 release +# bullet records the move. +name_regex = "^John Smith nee Jones R\\.A\\.I\\.$" +fields = ["_ambiguities"] + +[[change]] +issue = "fix(#445/#533) the clause gives the credential up, and the one name word it leaves is the family" +# 'John née Jones Smith MA', whose diff has TWO causes from here and +# one rule has to explain the whole of it -- the shape the +# fix(#424/#445) rules above already record for the numeral and for +# the Title-cased acronym. #445 moves the one name word the clause +# leaves into the family (given 'John' -> family 'John'), which this +# baseline has never read, and #533 takes the MA out of the maiden +# name and into `suffix`. Attributing either to the other would be +# false, and a rule declaring only one pair of fields would explain +# nothing while the name went UNEXPLAINED. +# +# Case-SENSITIVE, unlike the fix(#445) rule above, and measured: the +# capitals are what decide here, so 'John née Jones Smith Ma' keeps +# its member in the clause and 'John née Jones Smith MA' gives it up. +# An (?i) anchor would stand ready to explain either name's diff with +# the other's reasoning. +name_regex = "^John née Jones Smith MA$" +fields = ["_ambiguities", "family", "given", "maiden", "suffix"] + +[[change]] +issue = "fix(#445/#533) a one-case clause keeps the credential, and the one name word it leaves is the family" +# 'JOHN NEE JONES SMITH MA PHD'. One case, so no lean; a suffix word +# BEHIND the member ends the peel before it can reach the member, so +# the clause KEEPS the MA and only the clause's own emitter speaks. +# The role move is #445's -- the lone name word the clause leaves +# becomes the family -- and this baseline has not read it, so the +# diff is that move plus the new report. +# +# Its own rule rather than an alternative in the fix(#445) rule +# above, and case-sensitivity is the reason: that rule is (?i), and +# the mixed-case spelling of these words reads the other way. 'John +# nee Jones Smith MA PHD' gives the MA up and reads suffix 'MA PHD', +# measured 2026-09-19, so an (?i) alternative here would stand ready +# to explain a regression on a name whose reading is the opposite of +# this one's. +name_regex = "^JOHN NEE JONES SMITH MA PHD$" +fields = ["_ambiguities", "family", "given"] + +[[change]] +issue = "fix(#434/#533) a marker PHRASE takes the maiden name, and its clause ends at the credential" +# 'Maria Kowalska z domu Nowak MA'. The marker PHRASE is what moves +# the roles from here -- this baseline had no `z domu` entry, so it +# read middle 'Kowalska z domu', family 'Nowak' -- and the tree reads +# given 'Maria', family 'Kowalska', maiden 'Nowak'. #533's own effect +# on the name is invisible at this baseline: the trailing MA is a +# suffix on both sides, and both sides report it. What #533 changed +# is which words the maiden name holds, and `maiden` is already in +# the diff for the phrase's sake. +# +# Named for both changes because the maiden TEXT is the compound: at +# 2f57ff21 the same three fields moved and `maiden` read 'Nowak MA'. +# A rule naming only #434 would attribute this change's stop to the +# vocabulary addition; one naming only #533 would attribute the +# phrase to it. +name_regex = "^Maria Kowalska z domu Nowak MA$" +fields = ["family", "maiden", "middle"] + +[[change]] +issue = "fix(#424) the maiden walk stops before the trailing numeral, with the family name left standing" +# 'Jane Doe nee Smith V', and #533 has nothing to do with it: the +# name reads the same on this tree as it did at 2f57ff21, measured. +# The walk stops before the trailing roman numeral assign reads as +# the suffix, asked over the pieces as they stand and again over the +# view the take would leave -- #424's reading, stated at rules.md#M2. +# maiden 'Smith V' -> maiden 'Smith', suffix 'V', with the report +# #516 added at that slot. +# +# A rule of its own rather than an alternative in 'fix(#424/#445) the +# maiden walk stops before the trailing numeral, and the lone name +# word is the family' above: that rule's second half is the #445 move +# and this name has none -- family 'Doe' stands either way. What grew +# is the CORPUS: this change's rows put the numeral's clause shape in +# front of the gate for the first time at a baseline that can see it. +name_regex = "^Jane Doe nee Smith V$" +fields = ["_ambiguities", "maiden", "suffix"] + +[[change]] +issue = "fix(#533) the maiden clause ends at the credential in a native-script name too" +# '田中 太郎 旧姓 佐藤 MA': maiden '佐藤 MA' -> maiden '佐藤', suffix +# 'MA', one report. The walk reads the same rule whatever script the +# clause is written in -- the marker is 旧姓, the member is Latin, and +# neither fact reaches the stop -- so this is the movers' rule above +# with a CJK marker in front of it. +# +# A COMPOSED form, so the name is a `tolerated` row on the radar tier +# (decisions.md#cjk-comma-demotion) and its diff prints under +# UNCLASSIFIED (radar) rather than blocking. It is classified all the +# same, on fix(#289/#516)'s stated ground and the one #531's caseless +# rule follows: a radar diff an arc INTENDED is one a release note +# may be written from, and an unclassified one is not. +# +# Its own rule rather than an alternative in the movers' regex: a CJK +# member in an alternation is claimed by the honorific pin, which +# would demand that alternation be a copy of GLUED_HONORIFICS. A lone +# literal has no alternation to pin. At 1.4.0 and 2.0.0 the marker's +# own rule carries this name instead, the 旧姓 consumption being the +# larger diff from those baselines. +name_regex = "^田中 太郎 旧姓 佐藤 MA$" +fields = ["_ambiguities", "maiden", "suffix"] diff --git a/tools/differential/expected_since_2.2.0.toml b/tools/differential/expected_since_2.2.0.toml index 783eada8..8b399531 100644 --- a/tools/differential/expected_since_2.2.0.toml +++ b/tools/differential/expected_since_2.2.0.toml @@ -67,7 +67,15 @@ # never claim. [[change]] issue = "fix(#436/#437) a space-separated post-nominal run renders with spaces, not commas" -name_regex = "^(?:JOHN DOE PHD MD|John Doe MD PhD|John Smith MD PhD|John Smith Mc V|Kenneth Clarke QC MP|Smith, John PhD I\\.|The Rt Hon Kenneth Clarke QC MP, HMG|Washington Jr\\. MD, Franklin|abdul Smith Jr Ma|abdul Smith Jr V)$" +# 2026-09-19, #533: one name joins the alternation and the rule +# does not change. 'Jane Doe nee Smith PhD MA' is a two-word +# credential run this baseline renders 'PhD, MA' and the tree +# renders 'PhD MA' -- the whole of its diff at this baseline, and +# the same diff at 2f57ff21, measured. #533 moves nothing on it: +# the unambiguous PhD already ended the walk, so the clause gave +# the MA up before this change as it does after it. What grew is +# the CORPUS, and the run rendering is what the diff is about. +name_regex = "^(?:JOHN DOE PHD MD|Jane Doe nee Smith PhD MA|John Doe MD PhD|John Smith MD PhD|John Smith Mc V|Kenneth Clarke QC MP|Smith, John PhD I\\.|The Rt Hon Kenneth Clarke QC MP, HMG|Washington Jr\\. MD, Franklin|abdul Smith Jr Ma|abdul Smith Jr V)$" fields = ["suffix"] # The six #449 rules go SECOND, not first: the rule above @@ -1089,7 +1097,17 @@ issue = "fix(#531) a credential ending the given part of a family-comma listing # and nothing reports. _MUST_NOT_MATCH carries both, with the `do` # spellings P6 keeps and the third-comma-part name #144's restriction # still owns. -name_regex = "^(?:DOE, JOHN MA|DOE, MARY JO MA|Doe, Dr\\. John MA|Doe, J\\. MA|Doe, John BA|Doe, John MA|Doe, John MA JD|Doe, John MA Jr|Doe, John MA PhD|Doe, John PhD MA|Doe, John Q\\. MA|Doe, John X\\.Y\\.Z\\.|doe, john ma)$" +# 2026-09-19, #533: TWO names join the alternation and the rule does +# not change. 'Doe, J. ba' and 'Smith nee Jones, Jane MA' are this +# slot read exactly as the thirteen above are -- a lower-case member +# ending the given part after an initial, and the same member behind +# a maiden clause that sits in the FAMILY segment, where no trailing +# rule reads the clause's own tail and the given part's slot is +# reached as usual. Both read the same way at 2f57ff21, before #533, +# measured name by name: what grew is the CORPUS, not the rule, and +# a `fix(#533)` rule for them would attribute a #531 reading to the +# wrong change. +name_regex = "^(?:DOE, JOHN MA|DOE, MARY JO MA|Doe, Dr\\. John MA|Doe, J\\. MA|Doe, J\\. ba|Doe, John BA|Doe, John MA|Doe, John MA JD|Doe, John MA Jr|Doe, John MA PhD|Doe, John PhD MA|Doe, John Q\\. MA|Doe, John X\\.Y\\.Z\\.|Smith nee Jones, Jane MA|doe, john ma)$" fields = ["_ambiguities", "middle", "suffix"] orders = ["DEFAULT"] @@ -1181,3 +1199,152 @@ issue = "fix(#531) the trailing slot's positional reading reaches a caseless scr name_regex = "^(?:田中, 太郎 MA|김, 민준 MA)$" fields = ["_ambiguities", "middle", "suffix"] orders = ["DEFAULT"] + +[[change]] +issue = "fix(#533) a credential ending a maiden clause reads as a credential" +# The words a maiden marker takes now end where a trailing credential +# begins, where the rule that reads the name left standing reads the +# word as one -- both as the name is written and as the take would +# leave it (rules.md#M2, decisions.md#M2). `suffix` fills as `maiden` +# shortens, and the fork is reported: the end of a maiden clause was +# the last trailing position in the library where a member of the +# ambiguous credential class was read in silence. +# +# Seventeen names, and they are the class read from every side the +# corpora reach it: the bare member ('Jane Doe nee Smith MA', +# 'Doe, Jane nee Smith MA'); its one-case spellings, where the lean is +# inert and the POSITIONAL reading is not ('JANE DOE NEE SMITH MA', +# 'JANE DOE NEE YO-YO MA', 'jane doe nee smith ma', 'Doe, Jane nee +# Smith ma'); the other markers, the German abbreviation and the +# Polish PHRASE the walk reads no differently for its being three +# words ('Jane Doe geb. Smith MA', 'Maria Kowalska z domu Nowak MA'); +# a member beside another credential, where `suffix` GROWS rather than +# filling ('Jane Doe nee Smith MA JD', 'Jane Doe nee Smith MA PhD', +# 'Jane Doe nee Smith Ma JD' -- whose Title-cased Ma stays in the +# clause while the JD behind it leaves); past a trailing title, which +# the peel steps over ('Jane Doe nee Smith Prof. MA'); the slot +# reached past a middle initial ('Doe, Jane Q. nee Smith MA'); the +# `do` collision, where the capitals decide ('Doe, Jane nee Smith DO' +# beside 'Jane Doe nee Smith do', whose comma-less spelling P6 never +# reaches); the clause that leaves ONE name word, which rules.md#M4 +# makes the family ('John née Jones Smith MA'); and the CLAMP, where +# the member is the only word the marker would otherwise leave and +# only what stands BEHIND it is given up ('Doe, J. nee MA ba' keeps +# maiden 'MA' and reads suffix 'ba'). +# +# Literal-anchored for the reason fix(#531)'s rules give, and it is +# the same reason here: the subject is a SLOT, so a regex for it +# would claim the names whose WRITING declines the word -- which keep +# their maiden reading and have the rule below -- as readily as the +# ones it takes. _MUST_NOT_MATCH carries both directions. +name_regex = "^(?:Doe, J\\. nee MA ba|Doe, Jane Q\\. nee Smith MA|Doe, Jane nee Smith DO|Doe, Jane nee Smith MA|Doe, Jane nee Smith ma|JANE DOE NEE SMITH MA|JANE DOE NEE YO-YO MA|Jane Doe geb\\. Smith MA|Jane Doe nee Smith MA|Jane Doe nee Smith MA JD|Jane Doe nee Smith MA PhD|Jane Doe nee Smith Ma JD|Jane Doe nee Smith Prof\\. MA|Jane Doe nee Smith do|John née Jones Smith MA|Maria Kowalska z domu Nowak MA|jane doe nee smith ma)$" +fields = ["_ambiguities", "maiden", "suffix"] + +[[change]] +issue = "fix(#533) the maiden clause reports the credential it keeps" +# The other direction of the same fork, and the whole of what these +# eight names do at this baseline: no role moves and the decision +# starts being REPORTED. A Title-cased member in a mixed-case name is +# written the way a name is written, so the clause keeps it ('Jane +# Doe nee Smith Ma', 'Jane Doe nee Yo-Yo Ma', 'John née Jones Smith +# Ma', 'Doe, Jane nee Smith Ma'); a member that is the ONLY word +# after the marker stays the maiden name whatever its writing says, +# the marker having announced a name ('Jane Doe nee MA'); a suffix +# word BEHIND the member ends the peel before it reaches the member, +# so the clause keeps it and the clause's own emitter is what raises +# the fork ('JOHN NEE JONES SMITH MA PHD'); and P6's attachment keeps +# every `do` spelling the capitals do not take ('Doe, Jane nee Smith +# do', 'Doe, Jane nee Smith MA do'). +# +# Reporting a DECLINED fork is #530's stated rule -- the report +# tracks the fork consulted, not the lean -- and it is why this is a +# rule of its own rather than a widening of the one above: `fields` +# is `_ambiguities` alone, so it cannot absorb a role diff on any of +# the eight. +# +# Literal-anchored: the class is the slot's declining half, and a +# regex for it would claim the seventeen movers above. +name_regex = "^(?:Doe, Jane nee Smith MA do|Doe, Jane nee Smith Ma|Doe, Jane nee Smith do|JOHN NEE JONES SMITH MA PHD|Jane Doe nee MA|Jane Doe nee Smith Ma|Jane Doe nee Yo-Yo Ma|John née Jones Smith Ma)$" +fields = ["_ambiguities"] + +[[change]] +issue = "fix(#533) accepted: an unlisted dotted acronym ending a maiden clause is read by position" +# 'Jane Doe nee Smith X.Y.Z.'. A by-shape member carries no writing +# convention to read -- `listed_lean` returns None where the switch +# made the token credential-SHAPED and the vocabulary did not list it +# -- so the positional reading decides alone and the clause gives the +# word up. The same trade #516 recorded at its own slot and #531 at +# the given part's, taken to agree with the comma-less 'John Doe +# X.Y.Z.' and rules.md#S3's shape rule rather than to restore +# anything. +# +# Its own rule rather than one of the movers above, because the +# ARGUMENT is different: every name up there has a writing the lean +# could have read, and this one has none. +name_regex = "^Jane Doe nee Smith X\\.Y\\.Z\\.$" +fields = ["_ambiguities", "maiden", "suffix"] + +[[change]] +issue = "fix(#533) accepted: the released credential lands where the segment it is released into puts it" +# Two names where the member leaves the clause and something other +# than the trailing peel catches it. Both are SILENT, which is why +# they are not with the movers above: a rule declaring `_ambiguities` +# over a name that gains no report is the over-declaration #452's +# check names. +# +# 'Doe, Dr. nee Smith MA': once 'Smith' leaves with the marker, +# segment 1 is 'Dr. MA' -- a no-name segment the credential-run gate +# reads whole (rules.md#C1) -- so the member is a suffix and the +# given part's own emitter never sees it. 'Berg, abdul nee Jones MA': +# P5's lenient post-comma join takes the released member into the +# bound-given pair before assign can read it, exactly as it does in +# the clause-less 'Berg, abdul MA', so the clause form now AGREES +# with the bare form. That agreement is what makes each a consequence +# of the change rather than a defect in it. +name_regex = "^(?:Berg, abdul nee Jones MA|Doe, Dr\\. nee Smith MA)$" +fields = ["given", "maiden", "suffix"] + +[[change]] +issue = "fix(#533) restores the suffix reading a 2.4 retag had moved into the maiden name" +# 'John Smith nee Jones R.A.I.', and it is a RESTORATION rather than +# a change: this baseline reads suffix 'R.A.I.' with maiden 'Jones', +# and so does the tree, so only the report diffs here. What moved in +# between was this unreleased 2.4 cycle. At 2.3.0 the token carried +# `vocab:suffix` and the maiden walk's suffix-piece test stopped at +# it; #516 retags it `shape:acronym` plus `vocab:suffix-ambiguous`, +# it stops being a suffix piece, and the walk took it -- maiden +# 'Jones R.A.I.' on this tree at 2f57ff21, measured. The name was in +# no corpus file, so no gate ever compared it and no 2.4.0 release +# bullet records the move; this change's rows admit it, which turns +# an unrecorded regression into a recorded restoration. +# +# `fields` is `_ambiguities` alone at this baseline and says so +# deliberately: the restoration is invisible from here, and the rule +# that could absorb a role move on this name is the one this file +# must not have. +name_regex = "^John Smith nee Jones R\\.A\\.I\\.$" +fields = ["_ambiguities"] + +[[change]] +issue = "fix(#533) the maiden clause ends at the credential in a native-script name too" +# '田中 太郎 旧姓 佐藤 MA': maiden '佐藤 MA' -> maiden '佐藤', suffix +# 'MA', one report. The walk reads the same rule whatever script the +# clause is written in -- the marker is 旧姓, the member is Latin, and +# neither fact reaches the stop -- so this is the movers' rule above +# with a CJK marker in front of it. +# +# A COMPOSED form, so the name is a `tolerated` row on the radar tier +# (decisions.md#cjk-comma-demotion) and its diff prints under +# UNCLASSIFIED (radar) rather than blocking. It is classified all the +# same, on fix(#289/#516)'s stated ground and the one #531's caseless +# rule follows: a radar diff an arc INTENDED is one a release note +# may be written from, and an unclassified one is not. +# +# Its own rule rather than an alternative in the movers' regex: a CJK +# member in an alternation is claimed by the honorific pin, which +# would demand that alternation be a copy of GLUED_HONORIFICS. A lone +# literal has no alternation to pin. At 1.4.0 and 2.0.0 the marker's +# own rule carries this name instead, the 旧姓 consumption being the +# larger diff from those baselines. +name_regex = "^田中 太郎 旧姓 佐藤 MA$" +fields = ["_ambiguities", "maiden", "suffix"] diff --git a/tools/differential/expected_since_2.3.0.toml b/tools/differential/expected_since_2.3.0.toml index 73e6f21e..a937d30e 100644 --- a/tools/differential/expected_since_2.3.0.toml +++ b/tools/differential/expected_since_2.3.0.toml @@ -421,7 +421,17 @@ issue = "fix(#531) a credential ending the given part of a family-comma listing # and nothing reports. _MUST_NOT_MATCH carries both, with the `do` # spellings P6 keeps and the third-comma-part name #144's restriction # still owns. -name_regex = "^(?:DOE, JOHN MA|DOE, MARY JO MA|Doe, Dr\\. John MA|Doe, J\\. MA|Doe, John BA|Doe, John MA|Doe, John MA JD|Doe, John MA Jr|Doe, John MA PhD|Doe, John PhD MA|Doe, John Q\\. MA|Doe, John X\\.Y\\.Z\\.|doe, john ma)$" +# 2026-09-19, #533: TWO names join the alternation and the rule does +# not change. 'Doe, J. ba' and 'Smith nee Jones, Jane MA' are this +# slot read exactly as the thirteen above are -- a lower-case member +# ending the given part after an initial, and the same member behind +# a maiden clause that sits in the FAMILY segment, where no trailing +# rule reads the clause's own tail and the given part's slot is +# reached as usual. Both read the same way at 2f57ff21, before #533, +# measured name by name: what grew is the CORPUS, not the rule, and +# a `fix(#533)` rule for them would attribute a #531 reading to the +# wrong change. +name_regex = "^(?:DOE, JOHN MA|DOE, MARY JO MA|Doe, Dr\\. John MA|Doe, J\\. MA|Doe, J\\. ba|Doe, John BA|Doe, John MA|Doe, John MA JD|Doe, John MA Jr|Doe, John MA PhD|Doe, John PhD MA|Doe, John Q\\. MA|Doe, John X\\.Y\\.Z\\.|Smith nee Jones, Jane MA|doe, john ma)$" fields = ["_ambiguities", "middle", "suffix"] orders = ["DEFAULT"] @@ -513,3 +523,152 @@ issue = "fix(#531) the trailing slot's positional reading reaches a caseless scr name_regex = "^(?:田中, 太郎 MA|김, 민준 MA)$" fields = ["_ambiguities", "middle", "suffix"] orders = ["DEFAULT"] + +[[change]] +issue = "fix(#533) a credential ending a maiden clause reads as a credential" +# The words a maiden marker takes now end where a trailing credential +# begins, where the rule that reads the name left standing reads the +# word as one -- both as the name is written and as the take would +# leave it (rules.md#M2, decisions.md#M2). `suffix` fills as `maiden` +# shortens, and the fork is reported: the end of a maiden clause was +# the last trailing position in the library where a member of the +# ambiguous credential class was read in silence. +# +# Seventeen names, and they are the class read from every side the +# corpora reach it: the bare member ('Jane Doe nee Smith MA', +# 'Doe, Jane nee Smith MA'); its one-case spellings, where the lean is +# inert and the POSITIONAL reading is not ('JANE DOE NEE SMITH MA', +# 'JANE DOE NEE YO-YO MA', 'jane doe nee smith ma', 'Doe, Jane nee +# Smith ma'); the other markers, the German abbreviation and the +# Polish PHRASE the walk reads no differently for its being three +# words ('Jane Doe geb. Smith MA', 'Maria Kowalska z domu Nowak MA'); +# a member beside another credential, where `suffix` GROWS rather than +# filling ('Jane Doe nee Smith MA JD', 'Jane Doe nee Smith MA PhD', +# 'Jane Doe nee Smith Ma JD' -- whose Title-cased Ma stays in the +# clause while the JD behind it leaves); past a trailing title, which +# the peel steps over ('Jane Doe nee Smith Prof. MA'); the slot +# reached past a middle initial ('Doe, Jane Q. nee Smith MA'); the +# `do` collision, where the capitals decide ('Doe, Jane nee Smith DO' +# beside 'Jane Doe nee Smith do', whose comma-less spelling P6 never +# reaches); the clause that leaves ONE name word, which rules.md#M4 +# makes the family ('John née Jones Smith MA'); and the CLAMP, where +# the member is the only word the marker would otherwise leave and +# only what stands BEHIND it is given up ('Doe, J. nee MA ba' keeps +# maiden 'MA' and reads suffix 'ba'). +# +# Literal-anchored for the reason fix(#531)'s rules give, and it is +# the same reason here: the subject is a SLOT, so a regex for it +# would claim the names whose WRITING declines the word -- which keep +# their maiden reading and have the rule below -- as readily as the +# ones it takes. _MUST_NOT_MATCH carries both directions. +name_regex = "^(?:Doe, J\\. nee MA ba|Doe, Jane Q\\. nee Smith MA|Doe, Jane nee Smith DO|Doe, Jane nee Smith MA|Doe, Jane nee Smith ma|JANE DOE NEE SMITH MA|JANE DOE NEE YO-YO MA|Jane Doe geb\\. Smith MA|Jane Doe nee Smith MA|Jane Doe nee Smith MA JD|Jane Doe nee Smith MA PhD|Jane Doe nee Smith Ma JD|Jane Doe nee Smith Prof\\. MA|Jane Doe nee Smith do|John née Jones Smith MA|Maria Kowalska z domu Nowak MA|jane doe nee smith ma)$" +fields = ["_ambiguities", "maiden", "suffix"] + +[[change]] +issue = "fix(#533) the maiden clause reports the credential it keeps" +# The other direction of the same fork, and the whole of what these +# eight names do at this baseline: no role moves and the decision +# starts being REPORTED. A Title-cased member in a mixed-case name is +# written the way a name is written, so the clause keeps it ('Jane +# Doe nee Smith Ma', 'Jane Doe nee Yo-Yo Ma', 'John née Jones Smith +# Ma', 'Doe, Jane nee Smith Ma'); a member that is the ONLY word +# after the marker stays the maiden name whatever its writing says, +# the marker having announced a name ('Jane Doe nee MA'); a suffix +# word BEHIND the member ends the peel before it reaches the member, +# so the clause keeps it and the clause's own emitter is what raises +# the fork ('JOHN NEE JONES SMITH MA PHD'); and P6's attachment keeps +# every `do` spelling the capitals do not take ('Doe, Jane nee Smith +# do', 'Doe, Jane nee Smith MA do'). +# +# Reporting a DECLINED fork is #530's stated rule -- the report +# tracks the fork consulted, not the lean -- and it is why this is a +# rule of its own rather than a widening of the one above: `fields` +# is `_ambiguities` alone, so it cannot absorb a role diff on any of +# the eight. +# +# Literal-anchored: the class is the slot's declining half, and a +# regex for it would claim the seventeen movers above. +name_regex = "^(?:Doe, Jane nee Smith MA do|Doe, Jane nee Smith Ma|Doe, Jane nee Smith do|JOHN NEE JONES SMITH MA PHD|Jane Doe nee MA|Jane Doe nee Smith Ma|Jane Doe nee Yo-Yo Ma|John née Jones Smith Ma)$" +fields = ["_ambiguities"] + +[[change]] +issue = "fix(#533) accepted: an unlisted dotted acronym ending a maiden clause is read by position" +# 'Jane Doe nee Smith X.Y.Z.'. A by-shape member carries no writing +# convention to read -- `listed_lean` returns None where the switch +# made the token credential-SHAPED and the vocabulary did not list it +# -- so the positional reading decides alone and the clause gives the +# word up. The same trade #516 recorded at its own slot and #531 at +# the given part's, taken to agree with the comma-less 'John Doe +# X.Y.Z.' and rules.md#S3's shape rule rather than to restore +# anything. +# +# Its own rule rather than one of the movers above, because the +# ARGUMENT is different: every name up there has a writing the lean +# could have read, and this one has none. +name_regex = "^Jane Doe nee Smith X\\.Y\\.Z\\.$" +fields = ["_ambiguities", "maiden", "suffix"] + +[[change]] +issue = "fix(#533) accepted: the released credential lands where the segment it is released into puts it" +# Two names where the member leaves the clause and something other +# than the trailing peel catches it. Both are SILENT, which is why +# they are not with the movers above: a rule declaring `_ambiguities` +# over a name that gains no report is the over-declaration #452's +# check names. +# +# 'Doe, Dr. nee Smith MA': once 'Smith' leaves with the marker, +# segment 1 is 'Dr. MA' -- a no-name segment the credential-run gate +# reads whole (rules.md#C1) -- so the member is a suffix and the +# given part's own emitter never sees it. 'Berg, abdul nee Jones MA': +# P5's lenient post-comma join takes the released member into the +# bound-given pair before assign can read it, exactly as it does in +# the clause-less 'Berg, abdul MA', so the clause form now AGREES +# with the bare form. That agreement is what makes each a consequence +# of the change rather than a defect in it. +name_regex = "^(?:Berg, abdul nee Jones MA|Doe, Dr\\. nee Smith MA)$" +fields = ["given", "maiden", "suffix"] + +[[change]] +issue = "fix(#533) restores the suffix reading a 2.4 retag had moved into the maiden name" +# 'John Smith nee Jones R.A.I.', and it is a RESTORATION rather than +# a change: this baseline reads suffix 'R.A.I.' with maiden 'Jones', +# and so does the tree, so only the report diffs here. What moved in +# between was this unreleased 2.4 cycle. At 2.3.0 the token carried +# `vocab:suffix` and the maiden walk's suffix-piece test stopped at +# it; #516 retags it `shape:acronym` plus `vocab:suffix-ambiguous`, +# it stops being a suffix piece, and the walk took it -- maiden +# 'Jones R.A.I.' on this tree at 2f57ff21, measured. The name was in +# no corpus file, so no gate ever compared it and no 2.4.0 release +# bullet records the move; this change's rows admit it, which turns +# an unrecorded regression into a recorded restoration. +# +# `fields` is `_ambiguities` alone at this baseline and says so +# deliberately: the restoration is invisible from here, and the rule +# that could absorb a role move on this name is the one this file +# must not have. +name_regex = "^John Smith nee Jones R\\.A\\.I\\.$" +fields = ["_ambiguities"] + +[[change]] +issue = "fix(#533) the maiden clause ends at the credential in a native-script name too" +# '田中 太郎 旧姓 佐藤 MA': maiden '佐藤 MA' -> maiden '佐藤', suffix +# 'MA', one report. The walk reads the same rule whatever script the +# clause is written in -- the marker is 旧姓, the member is Latin, and +# neither fact reaches the stop -- so this is the movers' rule above +# with a CJK marker in front of it. +# +# A COMPOSED form, so the name is a `tolerated` row on the radar tier +# (decisions.md#cjk-comma-demotion) and its diff prints under +# UNCLASSIFIED (radar) rather than blocking. It is classified all the +# same, on fix(#289/#516)'s stated ground and the one #531's caseless +# rule follows: a radar diff an arc INTENDED is one a release note +# may be written from, and an unclassified one is not. +# +# Its own rule rather than an alternative in the movers' regex: a CJK +# member in an alternation is claimed by the honorific pin, which +# would demand that alternation be a copy of GLUED_HONORIFICS. A lone +# literal has no alternation to pin. At 1.4.0 and 2.0.0 the marker's +# own rule carries this name instead, the 旧姓 consumption being the +# larger diff from those baselines. +name_regex = "^田中 太郎 旧姓 佐藤 MA$" +fields = ["_ambiguities", "maiden", "suffix"] From d97d3eb713d3b6f075909b198f56454b4f3d003d Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Sat, 19 Sep 2026 17:54:18 -0700 Subject: [PATCH 3/5] docs(#533): the rule, the decision and the release note MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit rules.md#M2's statement gains the reader and the reason behind the credential stop commit 1 put in it: the double check stated once for both forks -- the count of words to spare includes the very words the marker removes -- and the reader named, the trailing rule where a trailing rule reads the part (no comma, and before a SUFFIX comma, measured to read the same way), the end of the given part after a family comma, nobody before that comma or past a second one. The lone-word floor is stated as the deliberate divergence it is from the `née V` / `née PhD` precedent, with the reason: that class carries no evidence of which it is. Reporting is stated at the boundary the tree actually holds -- a member ENDING a clause some rule reads is reported where the clause KEEPS it, while one it gives up reports only where the position it lands in reports, which `Doe, Dr. nee Smith MA` and `Berg, abdul nee Jones MA` would have falsified as a universal. #S2 and #S3 gain the slot in their enumerations; #P6 needs no change, the `do` reading here being #531's unchanged. A second Accepted row records the trailing-title boundary, where the two spellings disagree. Its two example lines were already corpus names, so corpus_rules.jsonl grows by two and no ledger claim moves. decisions.md gets one bullet under M2 amending the 2026-08-22 #424 entry -- which is left exactly as it landed -- and one under S2. The first records the double check, the reader table, the clamp and the two names a veto got wrong, the `do` pair and the accepted costs; it also re-measures the `one_case` plumbing those sites carry, which the 2026-09-18 entry recorded as inert under the numeral-only reading. The pair is the finding: 0 over the corpus as it stood, 36 on six names over the corpus with this change's own rows, the plumbing live either way. The second records the shared predicate, its frame price, and the one-case-head exception as the M2 instance of #492's deferred question. The release log gains one bullet. One correction in place: the #530 bullet said "the one slot this bullet left silent", which two bullets now close rather than one. docs/customize.rst's two 2.4 switch entries enumerate the positions each switch reaches, so both gain this slot. Three sweep findings beyond the plan's table, each a claim no test reads. The 1.4.0 ledger's fix(#424/#445) prose quoted the superseded Accepted row and said the fork is left to assign; it now says the WRITING declines the word. test_parser.py's "the walk takes the numeral only" is false of the tree, and the row under it is really about the member not being TRAILING, so it says that and gains the mirror case. And the judged-token sentence test_properties.py attributes to mechanisms.md is decisions.md#S2's, while the P3 sentence the same comment leans on says something P3 does not say -- both corrected to what those documents carry. mechanisms.md gains a field note for the trap commit 2 hit: a ledger rule EXPLAINS a diff and nothing checks that its sentence still DESCRIBES it, so a too-wide anchor kept `unexplained: 0` green over a name that had started reading the opposite way. Attribution against the parent found it; the gate could not. Review round: the design-docs review found a MOVER that reports nothing -- `Jane Doe, Jr nee Smith MA`, where the take leaves a post-nominals-only segment that is read whole -- which is the pre-existing no-name-word-in-front silence, written as a title's and reached by a post-nominal too; stated in M2 and S2, widened in _types.py, added to the M2 bullet and the release note, and pinned by a unit test rather than an example line, the name differing from this tree at all five baselines for reasons mostly not this change's. M2 and P6 now cite each other. The two _group.py denominators are re-measured: 36 of 10,752 (1,792 names) and 0 over 21,504, the latter with the probe shown live (site reached 394 times, tag admits 310). Co-Authored-By: Claude Fable 5.1 --- docs/customize.rst | 13 +++-- docs/design/decisions.md | 3 ++ docs/design/mechanisms.md | 1 + docs/design/rules.md | 56 ++++++++++++++++++-- docs/release_log.rst | 4 +- nameparser/_pipeline/_group.py | 10 ++-- nameparser/_types.py | 16 +++++- tests/v2/test_parser.py | 53 +++++++++++++++++- tests/v2/test_properties.py | 7 +-- tools/differential/corpus_rules.jsonl | 2 + tools/differential/expected_since_1.4.0.toml | 11 ++-- 11 files changed, 154 insertions(+), 22 deletions(-) diff --git a/docs/customize.rst b/docs/customize.rst index 7cef3b90..326f53dd 100644 --- a/docs/customize.rst +++ b/docs/customize.rst @@ -454,7 +454,11 @@ listed below. ``"Jack X.Y.Z."`` keeps family ``X.Y.Z.``, and either reading is reported. The family-comma form is one of those positions since 2.4: ``"Doe, John X.Y.Z."`` gives suffix ``X.Y.Z.`` - while ``"Doe, X.Y.Z."`` keeps given ``X.Y.Z.``. + while ``"Doe, X.Y.Z."`` keeps given ``X.Y.Z.``. So is the word + ending a maiden marker's clause, also since 2.4: + ``"Jane Doe nee Smith X.Y.Z."`` gives maiden ``Smith`` with + suffix ``X.Y.Z.``, where ``False`` keeps maiden + ``Smith X.Y.Z.``. Case is irrelevant — the periods are the signal. Whole-token vocabulary still wins (``M.A.``, ``Ph.D.``), and a single trailing period is not this shape @@ -477,8 +481,11 @@ listed below. - Reads an unlisted all-caps word of two or more letters, with no period in it, in a name written in more than one case as a credential where the position allows it: ``"John Smith XYZ"`` - gives suffix ``XYZ``, and since 2.4 so does the family-comma - form ``"Doe, John XYZ"``. Defaults to ``False``, and + gives suffix ``XYZ``, and since 2.4 so do the family-comma + form ``"Doe, John XYZ"`` and the word ending a maiden marker's + clause (``"Jane Doe nee Smith XYZ"`` gives maiden ``Smith`` + with suffix ``XYZ``, where off it keeps maiden + ``Smith XYZ``). Defaults to ``False``, and deliberately: an all-caps surname is a real writing convention that shape cannot separate from a credential, so ``"Jean Pierre DUPONT"`` diff --git a/docs/design/decisions.md b/docs/design/decisions.md index d5f2ecad..8cb7f2a3 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -200,6 +200,8 @@ the fullwidth-colon marker (旧姓：佐藤 arrives as one word; the head-peel q One limit is recorded rather than closed. The contiguity test is role plus comma bucket, and role is one-directional: a role change is always a clause edge, but two ADJACENT clauses of the same role are indistinguishable, so 'Jane (z) (domu) Jones' does tag a run across two separate nickname clauses. Nothing reads it — the piece walk never sees role-bearing tokens and the clause drop is scoped to one span — and the parse is identical with and without the phrase mechanism. The clean fix is a per-token structural UNIT id derived from state.segments and state.extracted, which would replace role-plus-bucket with one comparison and close the adjacent-clause case as a side effect. Deliberately not taken here: it changes a mechanism four correctness reviewers had just cleared, inside a pass whose stated business was not to change behavior. It is the shape to reach for if this test ever needs to grow a third term. +- 2026-09-19 (Derek), #533 — THE CLAUSE ENDS AT A TRAILING CREDENTIAL, AND THE DOUBLE CHECK THE 2026-08-22 BULLET DEFERRED NOW EXISTS FOR THE ACRONYM TOO. That bullet stopped the walk before the trailing numeral and left the bare-acronym fork alone, with its reason stated: the acronym fork COUNTS pieces and the walk removes the very pieces it counted, so a peel read over the pieces as they stand said "credential with words to spare" about `John née Jones Smith Ma` and would have left maiden 'Jones Smith', family 'Ma'. The reason was right and is unchanged; what it argued for was a deferral, and the answer is the one the numeral got — ask the peel TWICE, once over the pieces as written and once over the name the take would leave, and stop only where both read the word as the credential. "Left to assign" meant "read in silence", assign never getting the question at all, the clause's tokens carrying the maiden role before pieces exist for it: the end of a maiden clause was the last trailing position in the library where a member of the ambiguous credential class was read without a report. WHICH RULE reads the name left standing is the part the numeral's version did not have to answer, because a numeral reads the same way everywhere and a credential does not. With no comma it is the trailing peel; after a family comma it is #531's slot, where the comma has already settled the count and the writing decides alone; before that comma the words are the family the comma named and a stop would hand one of them to `family` rather than to `suffix`; past a second comma the segment is read as credentials whole and no trailing rule is consulted. The last two READ AS NONE, which is a statement and not a default — the first prototype reported on `Smith, John, Jr nee Jones MA`, where the walk had asked nothing. A FIFTH CASE, found by the design-docs review of this commit and not by the plan: after a family comma the reader may take the word and nothing may then report it. `Jane Doe, Jr nee Smith MA` is a family comma whose segment 1 is `Jr nee Smith MA`, so the given-slot reader is asked and takes the member — maiden 'Smith', suffix 'Jr MA' — but the take leaves that segment as `Jr MA`, post-nominals only, which the credential-run gate reads whole without ever reaching the emitter. It is a MOVER that reports nothing (2f57ff21 read maiden 'Smith MA'), and it is the pre-existing "no name word in FRONT of the member" silence, the same one `Doe, Dr. nee Smith MA` reaches — written as a TITLE's silence, where a post-nominal reaches it just as well. The reading is right and the silence is the documentation defect: measured over a generated sweep of clause shapes (particles, titles, connectives, numerals, by-shape and caps-on members, comma and no-comma heads) under five policies, 780 silent taken rows on 156 distinct names, every one of them on a `Jane Doe, Jr` / `Jane Doe, Jr.` / `Jane Doe, III` / `Jane Doe, PhD` head, and every one agreeing with its clause-less control on BOTH the class the member lands in and the report — 0 disagreements, which is what makes this wording rather than behavior. The same sweep found no sixth shape. The KEPT direction still reports here, the clause's own emitter being what raises it: `Jane Doe, Jr nee Smith Ma` and `Jane Doe, Jr nee MA` both say so. Pinned by a unit test rather than by a rules.md example line: the name differs from the tree at all five baselines and most of the diff is not this change's (1.4.0 reads title 'Jr', given 'nee', family 'Jane Doe'; 2.0.0 and 2.1.0 title 'Jr', family 'Jane Doe'; 2.2.0 and 2.3.0 already read given 'Jane', family 'Doe', suffix 'Jr'), so an example line would have put five ledger rules into this docs commit to attribute four other changes' readings. THE SECOND CHECK ASKS WHETHER THE READER TAKES THAT WORD, not whether it takes something, and the distinction is not academic: the numeral's existing re-ask tests whether the view's trailing run starts at the view's end, exactly right for a fork that reads one piece, and under it `JOHN NEE JONES SMITH MA PHD` leaves `JOHN MA PHD`, whose peel takes 'PHD' and then DECLINES 'MA' for want of words to spare — so the weaker question answers yes while the member becomes the FAMILY name, which is the 2026-08-22 disaster one word further on. Measured both ways on the prototype; the shipped check compares the view's run start against the member's own index in that view, and `JOHN NEE JONES SMITH MA PHD` keeps maiden 'JONES SMITH MA' with suffix 'PHD' and reports. AFTER A FAMILY COMMA THE CHECK NEEDS #531'S FLOOR, NOT ONLY ITS LEAN: `Doe, Jane nee Smith MA do` leaves `Jane MA do`, where 'MA' leans credential but the 'do' behind it does not read as a suffix (P6 keeps it, and `Doe, Jane MA do` reads middle 'MA', family 'do Doe'), so #531's slot would read 'MA' as a MIDDLE name — and a check that asked only about 'MA' released it from the clause into the current name's middle, a word crossing from one person's name to another's. With the floor the clause keeps 'MA do' whole and reports the 'do'. THE FLOOR ON THE FIRST WORD IS A CLAMP, NOT A VETO, and the first draft got two names wrong by making it one: stated as "a member that is the ONLY word after the marker stays the maiden name" and implemented as "cancel the stop when nothing would be left", it cancels too much wherever the peel consumed that first word AND words behind it — `Doe, J. nee MA ba` peels 'ba' and then 'MA', so the first piece the peel took IS the only maiden word, and the veto handed 'ba' back to the clause, giving maiden 'MA ba' where `Doe, J. ba` reads suffix 'ba'. Clamped to the piece after the marker instead, that name keeps maiden 'MA' and reads suffix 'ba', reporting both. The floor itself is deliberate and diverges from the `née V` / `née PhD` precedent, where the marker declines and stays an ordinary word: certain suffix vocabulary declines the marker, while an ambiguous word is kept by the clause it ends, because that class is the one carrying no evidence of which it is — its members are borne surnames as well as credentials, and nobody writes a credential straight after the marker. 1.4.0 read `Jane Doe nee MA` as last 'nee', suffix 'MA' (measured on the wheel), so this half restores nothing and does not try to. THE `do` PAIR is #531's reading, unchanged and now SHARED as a predicate rather than spelled twice (`credential_at_the_given_slot` in `_pipeline/_pieces.py`; the drift would have been silent, each site's own tests going on passing): `Doe, Jane nee Smith do` keeps maiden 'Smith do', `Doe, Jane nee Smith Do` keeps 'Smith Do', `Doe, Jane nee Smith DO` gives maiden 'Smith' with suffix 'DO', and the comma-less `Jane Doe nee Smith do` reads suffix 'do' — right for the reason the whole change is, since `John Doe do` reads suffix 'do' today, P6 does not run without a comma, and a clause must not change how a word outside it reads. ACCEPTED COSTS, each measured on this tree: `JANE DOE NEE YO-YO MA` reads maiden 'YO-YO', suffix 'MA', a two-word birth surname losing its last word in a one-case record — and `JANE YO-YO MA` reads suffix 'MA' too, so the cost is the one-case reading's rather than this rule's, and in mixed case the writing saves it (`Jane Doe nee Yo-Yo Ma` keeps maiden 'Yo-Yo Ma' and reports); `Jane Doe nee Smith X.Y.Z.` reads suffix where 1.4.0 read last 'X.Y.Z.', the same trade #531 took at its own slot and for agreement with `John Doe X.Y.Z.` rather than for parity; `Doe, Dr. nee Smith MA` MOVES IN SILENCE to title 'Dr.', maiden 'Smith', suffix 'MA', the take leaving a segment with no name word which the credential-run gate reads whole; `Berg, abdul nee Jones MA` gives given 'abdul MA', P5's lenient post-comma join taking the released member into the bound-given pair exactly as it does in the clause-less `Berg, abdul MA`, so the clause form now AGREES with the bare form; `Jane Smith nee Jones and MA` reads maiden 'Jones and' with suffix 'MA', the take running before the joins (#420) so the connective is a piece of its own when the walk stops; and a report on a declined word adds noise without moving a field, which is #530's stated rule and the trade #531 took at its slot. NOT TRANSPARENT HERE: a trailing title. `Jane Doe nee Smith MA Prof.` is unchanged and silent while `Jane Doe nee Smith Prof. MA` gives maiden 'Smith Prof.' with suffix 'MA' and reports, because the walk reads the trailing credential run and the H5 title chain is read by the rule the walk does not run. Accepted for #533, pinned as a pair of boundary example lines under M2; moving the walk onto the title-aware reading would change the numeral half too, which the 2026-08-22 bullet settled with its own measurements, and the pair belongs to one decision about what "trailing" means inside a clause — a follow-up is drafted for Derek rather than filed here. THE `one_case` PLUMBING AT THESE SITES IS NOW LIVE, and the 2026-09-18 bullet under `### S2` headed THE PREDICTION THAT DID NOT SURVIVE ITS OWN MEASUREMENT stands as what was true then: `numeral_only` answered off the peeled numeral and the numeral fork is decided before the peel reads a lean, so the fact reached only the bare-acronym fork, which that reading discarded. The acronym fork is asked now. Re-measured 2026-09-19 with the same wrapper shape over the same population that entry's recipe names — AND THE PAIR IS THE FINDING: over the corpus as it stood the day before this change, dropping the argument inside the walk moved 0 parses; over the corpus WITH this change's own rows it moves 36 of 10,752 (1,792 names), on six distinct names (`Doe, Jane nee Smith DO`, `Doe, Jane nee Smith Ma`, `Jane Doe nee Smith Ma`, `Jane Doe nee Smith Ma JD`, `Jane Doe nee Yo-Yo Ma`, `John née Jones Smith MA`). The population, the six policies and the denominator are recorded at the call site in `_pipeline/_group.py`, as the 2026-09-18 entry records its own; both were re-run on 2026-09-19 after the review found the figures first written there (1,790 names, 10,740 and 21,480 parses) did not reproduce from the recipe as stated — they had been taken two corpus names earlier in the same day's work. The FINDING was unharmed, 36 parses on the same six names. The plumbing was live either way and the corpus could not show it, which is the blindness mechanisms.md's corpus field note asks to be measured before any "N names move" is written down. + ### N3 — the lone-word nickname rule - 2026-07 (v2 core, PR #288; recorded plan deviation #2 of the core plan) — v1's rule counted pieces before grouping; the v2 port fires only when the nickname accompanies exactly ONE piece in total — a title counts against it, so "'Smitty' Dr. Jones" reaches H1 with a title and one name word left standing — through 2.1 that meant given="Jones" with the family empty, and since #410 (2026-08-25) H1 names the family, so it reads family="Jones". The count is unchanged; what moved is what happens after it declines. The rule lives in assignment because that is where the piece count is settled. (An earlier wording here said "one non-title piece", predicting the opposite output; the coherence review measured the truth.) @@ -602,6 +604,7 @@ for n in ('Smith, John','Smith, XYZ'): print(n, calls_for(off.parse, n), calls_f - 2026-09-18, VERIFICATION ROUND — THE REPORT INVENTORY AN UPGRADER SEES, 2.3.0 → TREE, AND ITS RECIPE. docs/release_log.rst's comma-report bullet claimed TWO reports go away and named `Steven Hardman, MD, DO, DDS` and `John Smith, MD, R.A.I.`. The second is false and was never measured against the wheel: `John Smith, MD, R.A.I.` reports NOTHING at 2.3.0, having carried a `comma-structure` flag only inside this arc's own development, where rules.md#S3's narrow retirement raised it and the C2 amendment quieted it again. Over the whole corpus exactly ONE report is lost, and one more is SWAPPED. Measured: LOST — `comma-structure` ×1 (`Steven Hardman, MD, DO, DDS`), `given-or-family` ×2 (`Jack X.Y.I.`, and `e and e`, which is PR #527's conjunction change and not this arc's). GAINED — `suffix-or-name` ×37, `conjunction-or-initial` ×9 (#527's again), `given-or-family` ×2 (`Jack MA`, `Jack MA.`). So `Jack X.Y.I.` is the only name whose report CHANGED KIND on this arc's account, `given-or-family` → `suffix-or-name`, and the bullet now names it as the swap rather than counting it among the losses. Recompute: collect the corpus glob through compare.py's `_load_entries` deduped by name, parse each name on the tree and on the 2.3.0 wheel (a PEP 723 `uv run --no-project` script pinning `nameparser==2.3.0`, per AGENTS.md's warning that a same-tree comparison reports parity), and take the per-name multiset difference of the `ambiguities` kind values in both directions. The kind counts are multisets, not name counts — `e and e` contributes two `conjunction-or-initial` gains on its own — which is why a bullet quantified from the gate's per-rule name lists would not answer this question. - 2026-09-18 (Derek), #531 — CAPITALS DECIDE FOR `do`, AND THE PARTICLE RULE KEEPS EVERY OTHER SPELLING. `do` is the one member of the ambiguous credential class (`ba`, `do`, `ed`, `jd`, `ma` — recompute with `sorted(Lexicon.default().suffix_acronyms_ambiguous)`) that is also particle vocabulary, so the trailing slot of the given part and P6's attachment want the same word. Where the writing carries a positive credential lean — an all-caps member in a name written in more than one case — the credential reading wins and P6 stands down: `Doe, John DO` reads suffix `DO` and reports `suffix-or-name`. Every other spelling attaches exactly as it did before, with P6's own `particle-or-given` and no second report: `Doe, John do`, `Doe, John Do`, `DOE, JOHN DO`, `doe, john do` all read family `do Doe` in their own case. THE PAIRING IS THE ARGUMENT, and the accepted cost and the reason are the two halves of it. In ONE CASE the rule cannot tell `NASCIMENTO, EDSON ARANTES DO` from `SMITH, JOHN DO` and reads both as the particle — right about the Brazilian record, wrong about the osteopath, whose `DO` 1.4.0 read as a suffix; both report `particle-or-given`, which is how a caller finds the second. In MIXED CASE it can, and does: `Nascimento, Edson Arantes do` keeps family `do Nascimento` while `Doe, John DO` gives suffix `DO`. Accepted rather than repaired: the one-case reading is the commoner of the two collisions, and the Nascimento record is the name a wrong answer would damage. The other four members are not particles and are unaffected. This is the exception rules.md#P6's "this attachment outranks the suffix reading (S2)" clause now names. Implementing it takes TWO carve-outs rather than one, which is worth recording because the second is in the REPORT and not the reading: the reading carve-out takes a particle-tagged member only on a POSITIVE credential lean, since `Doe, John do` leans nothing and the positional reading would otherwise take it — the wrong ANSWER, not merely a stray report; and the emitter beside it skips a particle-tagged member it did not itself take, since an emitter gated on tag membership alone reported both kinds on every `do` row. Where P6 owns the fork, P6 reports it, which is mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE read strictly. ONE SHAPE THE PAIRING DOES NOT REACH — a shape, not a spelling, and the difference is the review finding: ANY particle or particle chain standing in front of the caps `DO` takes it out of this slot, because grouping's prefix chain merges the two into ONE particle piece before the slot exists and there is then no lone member for the capitals to lean on. It is a fact about this WORD being particle vocabulary itself, not about particles generally — the caps `MA` in `Doe, John van MA` is not swallowed, reads as the credential, and leaves `van` to attach behind it. `Doe, John van DO` reads family `van DO Doe`, and so do the `de`, `von`, `del`, `la` and `van der` spellings of it (measured 2026-09-19 — parse them and read `family`). Pre-existing and unmoved by #531; `Doe, John van DO` is the case row and rules.md#P6's example line, and P6's statement carries the carve-out. The mirror shape is silent rather than merely unreached and is recorded on `AmbiguityKind.SUFFIX_OR_NAME`'s fourth boundary: a particle BEHIND the member ends the given part where this walk reads it, so `Doe, John MA do` keeps middle `MA` beside family `do Doe` though the capitals would otherwise have taken it, the attachment that moves `do` running a stage later. - 2026-09-18 (Derek), #531 — THE COMMA REPORT'S REACH NOW INCLUDES THE GIVEN SEGMENT'S TRAILING SLOT, AND THE OPEN FOLLOW-UP IS CLOSED. The 2026-09-18 bullet above headed THE COMMA REPORT'S REACH IS THE FIRST POST-COMMA PIECE recorded this slot as an open maintainer decision and named the two questions it turned on — whether a middle initial's neighbourhood should start reporting (a noise judgement, rules.md#A1) and whether the silence was a 1.4 parity gap. Both are answered here, and that bullet stands as it landed. The slot now reads and reports: `Doe, John MA` gives suffix `MA` and `Doe, John Ma` keeps middle `Ma`, each saying which way it went. Derek chose to restore the ROLE and report both ways — one rule for both spellings — over a report-only change and over a capitals-only one, because the issue exists in the first place because two spellings of one name disagree. The noise question was settled by MEASURING THE DISAGREEMENT rather than by argument: over a generated sweep of 78 pairs — five listed members and two by-shape tokens in three cased spellings each, plus five controls in one spelling apiece, so 26 words against three name shapes — 48 pairs disagreed about whether the word was a credential or a name, and every one of the 48 disagreed in the same direction, the comma form declining what the comma-less form took. After this change 3 disagree and all three are the lower-case `do` rows P6 owns. The sweep and its allowlist are `tests/v2/test_properties.py`; a slot that answers differently from the same name written without a comma is not a quiet slot, it is an inconsistent one. Recorded under `3-0-reevaluations`' standing rule because v1 parity is LOAD-BEARING for one half of this and explicitly NOT for the other: `Doe, John MA` reads suffix `MA` on the 1.4.0 wheel, so the bare-acronym half RESTORES v1's role and the report is all that is new there — its 1.4.0 ledger rule neither retired nor narrowed to `_ambiguities` (nothing below baseline 2.0 can diff on that pseudo-field) but was RE-POINTED to the four names where the writing declines the credential, which keep the 2.0-era middle name against v1. `Doe, John X.Y.Z.` reads middle `X.Y.Z.` at 1.4.0 too, so the dotted half LEAVES v1 and carries a 1.4.0 ledger rule of its own; it moves to match the comma-less `John Doe X.Y.Z.` and rules.md#S3's shape rule, not to restore anything. BLAST RADIUS, stated the way this log's own rule asks: twenty corpus names move, and only TWO of them were in any corpus before this branch (`Doe, John MA` and `Doe, John X.Y.Z.`, both admitted by #530's own arc) — the other eighteen are this change's own case rows, so what the differential measures on pre-existing data is two names, and the population the rule reaches is a SHAPE (every family-comma listing whose given part ends in a member of this class) that the corpora barely sample. ACCEPTED COSTS, all measured on the differential corpora: `SMITH, JOHN DO` keeps family `DO SMITH` where 1.4.0 read suffix `DO`, paired with the Nascimento record in the bullet above; one-case and caseless names take the credential with no case evidence at all, so `DOE, MARY JO MA`, `doe, john ma`, `田中, 太郎 MA` and `김, 민준 MA` all read a suffix, which is what 1.4.0 read for each of the four — "caseless is inert" holds for the LEAN and not for the outcome, since `is_one_case` answers True for a script with no case and the positional reading then decides; `Doe, John van MA` loses its middle to the family, reading family `van Doe`, suffix `MA` with two reports where it read middle `van MA` in silence, which is `Berg, Jan van Jr.`'s reading arriving through a shape it could not reach before (Derek accepted it 2026-09-18 as P6 working correctly rather than as a cascade to carve out); `Doe, John Prof. MA` gains a TITLE role `Prof.` never had, H5's transparency reaching it once `MA` leaves the walk, landing it on the same answer as `Doe, John MA Prof.`; `Smith, LEED AP` moves under the default-off caps switch alone, to given `LEED`, family `Smith`, suffix `AP` with two reports, so no default reading is at stake; and a genuine middle name that is also a class member is now a credential wherever it ends the given part and is not Title-cased — `DOE, JOHN ED` reads suffix `ED` — which is the same cost the comma-less form has carried since 2.0. NOT REPAIRED HERE and left open: the maiden walk claims `Smith MA` whole in `Doe, Jane nee Smith MA` before this slot exists, so the rule cannot reach it and the name stays silent with maiden `Smith MA` — the same gap #530's close-out saw from the other side with `John Smith nee Jones R.A.I.`, and it is out of scope for #531. +- 2026-09-19 (Derek), #533 — THE CLASS'S LAST SILENT TRAILING POSITION IS CLOSED, AND #531'S READING NOW LIVES IN ONE PLACE. The slot list in rules.md#S2 gains the trailing slot of a maiden marker's clause, and #S3's enumeration gains it with the by-shape spellings; the rule that governs what the clause does with those words is M2's, and the entry under `### M2` above is where its reasoning lives. The gap the bullet above left open as out of scope for #531 is the one this closes, from both sides: `Doe, Jane nee Smith MA` now gives maiden 'Smith' with suffix 'MA' and reports, and `John Smith nee Jones R.A.I.` gives suffix 'R.A.I.' again — a RESTORATION rather than a change, since 2.3.0 read it that way (measured on the wheel) and this unreleased cycle moved it into the maiden name when #516 took dotted tokens out of the certain-suffix class, where no corpus file held the name and no gate could see it. Two things belong here rather than under M2. FIRST, `credential_at_the_given_slot` in `_pipeline/_pieces.py` is now where #531's reading of a class member ending the given part is written, with two callers — assign's walk over that part, and the maiden walk's second check over the name a take would leave. Spelling it twice is precisely the "condition written to match it" mechanisms.md#ONE-PREDICATE-PER-QUESTION names, and the drift would have been silent, since each site's own tests would have gone on passing. It is a text-and-tags question, which is what puts it in `_pieces` rather than beside either caller — the destination follows the LAYER, not the topic. It costs one frame PER MEMBER asked at that slot: measured 2026-09-19 against 2f57ff21 per `Parser.parse`, `Doe, John MA` goes 310 → 311 and `Doe, John MA Ma MA`, which asks four times, 439 → 443, while a name with no member there never reaches it and pays nothing. The reference band does not move (412/449 on `uv run python tools/perf/call_count.py`), no test pins 310, and Derek took the trade rather than keep two conditions in step across two stages with no test that asks them both. The maiden path got one frame CHEAPER in the same pass, the numeral reading calling the peel pair it had wrapped rather than the wrapper: `Jane Doe nee Smith` 249 → 248. SECOND, THE ONE-CASE HEAD IS AN ACCEPTED EXCEPTION AND IT IS THE M2 INSTANCE OF #492'S DEFERRED QUESTION about whether a cased suffix token counts as case evidence. `DOE, JANE nee Smith Ma` reads suffix 'Ma' where the clause-less `DOE, JANE Ma` keeps middle 'Ma', because the own-words span stops at the marker — rules.md#P3 puts "a maiden marker's run and every word after it" outside the name's own words from the moment the marker is tagged — so the member's own Title-casing is not in the span `one_case` is computed over and the clause HIDES the contrast. That is THE PREDICATE KEEPS THE JUDGED TOKEN IN THE SPAN, the 2026-09-14 #289/#516 entry at the head of this section, failing structurally rather than by oversight: at this slot the judged token is never in the span, and that entry's own answer — include it — cannot be had here. Widening the span for this one question would change `one_case` for the whole name, and three sites read it, so the exception is accepted instead. Measured 2026-09-19: 114 of 2016 generated pairs — every listed member and both by-shape spellings, in three cased spellings, against sixteen heads and six clause bodies of NAME WORDS ONLY — against 186 allowlisted and 984 disagreeing outside the class before the change, and 0 outside it after. `tests/v2/test_properties.py` carries the sweep with the class defined STRUCTURALLY (`one_case` true of the clause form and false of the clause-less one) rather than as a name list, plus the recorded count asserted beside it, because a structural allowlist cannot notice its own growth. ### indic-honorifics — the renunciate class and the Indic honorific vocabulary (2026-09-06, #346/#344/#343) diff --git a/docs/design/mechanisms.md b/docs/design/mechanisms.md index 67db3320..fafad208 100644 --- a/docs/design/mechanisms.md +++ b/docs/design/mechanisms.md @@ -191,6 +191,7 @@ Problem shape. A test pins an ordering, a sort, a dedup or a partition, and its - Run all the gates, not the ones you remember: ruff runs before mypy and pytest in CI, and each has caught what the others passed. - Purge __pycache__ between same-length source mutations; stale bytecode makes a changed file measure as unchanged. - After NARROWING a rule, check the receiver: the names a narrowed rule sheds land on a neighbour, and nothing guarantees the neighbour's prose describes what it inherited — #375 fixed an over-claiming rule and relocated the bug onto its neighbour. Ask also what the old behavior was CONCEALING: #379's attachment removed the input a test used to build an all-particle middle name (#402), and #400's reserve fix exposed the dual-membership count shape #397 names -- twice in one session a fix's real yield was a defect it stopped hiding. +- A ledger rule EXPLAINS a diff; nothing checks that its own sentence still DESCRIBES it, so a rule can go on covering a name it has stopped being true of and the gate stays green. Measured on #533: a `(?i)` anchor written for `John née Jones Smith Ma` — the clause keeping the credential — also reached the capitals spelling, which that change made read the opposite way, the clause giving the credential up. The rule still matched the name and still covered its fields, so `unexplained: 0` was silent about it. What found it was ATTRIBUTION rather than the gate: measuring every corpus name the change touches against the PARENT commit, the EXPLAINED ones as well as the unexplained, and asking of each which change actually moves it. Do that before writing a ledger rule's prose, not only when a name goes unexplained — the failure mode is a rule that reads as authoritative and argues for the reading it lost. - A skip is indistinguishable from "correctly declined": pytest turns an empty parametrize into a skip, and a filter that widens its own skip set cannot fail. After changing any selection shape, verify the guard still REACHES the code it watches — assert the selected set is non-empty, or force-a-decision on its size. - A differential corpus cannot evidence behavior keyed to OUT-of-vocabulary shapes: it holds only names someone wrote down, and an unrecognized word is by definition outside the vocabulary — a green run over the corpus proves nothing about such a rule. - The corpus can be near-blind to a WRITING CONVENTION even where the vocabulary is well covered, and a small honest count then reads as a small blast radius. Measured for rules.md#P6: of 782 corpus names, 245 carry a comma, TWO of those end in a particle, and ONE of the two clears the words-to-spare guard and actually moves — so the Dutch trailing-tussenvoegsel listing the rule exists for is essentially unsampled. Before reporting "N names move", report the size of the population that COULD move; when that is ~1, the number is evidence about the corpus. This note's first wording said "exactly ONE ends in a particle", conflating the population with the movers — the very error it exists to prevent, committed inside the correction. Caught by re-running the count against AGENTS.md's axis 1 rather than by re-reading it. diff --git a/docs/design/rules.md b/docs/design/rules.md index 5aff882a..46d697ca 100644 --- a/docs/design/rules.md +++ b/docs/design/rules.md @@ -891,7 +891,7 @@ P6. Rationale: a particle ending the name has nothing to link negative-control sweep pinning the disagreeing set the precedence bullet above names. A change that breaks one side of that pair should expect that test, not this file, to say so first. - history: decisions.md#P6 · interacts: A1, C1, P1, S2, P5 · implemented: nameparser/_pipeline/_post_rules.py + history: decisions.md#P6 · interacts: A1, C1, P1, S2, P5, M2 · implemented: nameparser/_pipeline/_post_rules.py ## Suffixes: generational & credentials (S) @@ -932,7 +932,12 @@ S2. Rationale: generational suffixes and credentials are recognized name — and at the slots that report, either reading carries the ambiguity flag. Those slots are the trailing slot of a name, the first slot after a family comma, the trailing slot of the GIVEN - part after that comma, and the segments beyond it. + part after that comma, the trailing slot of a maiden marker's + clause (M2), and the segments beyond it. A word this document + says is READ at one of those slots is not always a word one of + them reports: where a reading moves a word out of the slot that + asked about it, what reports is the slot it lands in, and that + may be none — M2 states the case where a clause does it. After a family comma, a word of this class ending the GIVEN part is read as the comma-less spelling reads a word ending the name, and the count is not what decides it there. The comma has already @@ -1089,7 +1094,8 @@ S3. Rationale: credentials are often written run together with to spare, a name word where it does not, either reading reported at the slots S2 reports at, and the same at a comma — which means the FIRST piece after a family comma, the word - trailing the given part after one, and the part before a SUFFIX + trailing the given part after one, the word ending a maiden + marker's clause (M2), and the part before a SUFFIX comma. The part before a FAMILY comma never reports, the comma having already named it the family. Case says nothing here — the periods are the evidence — and three shapes are outside it: a @@ -1231,7 +1237,40 @@ M2. Rationale: a maiden marker announces that what follows it is the reading the name left standing reads the word as the credential, and never the first word after the marker — as the maiden name, and - the marker itself is dropped. A marker + the marker itself is dropped. + Those last two stops are each asked TWICE for one reason: the + count of words to spare includes the very words the marker + removes, so a reading taken over the name as written can be + wrong about the name the take would leave. WHICH rule does the + reading depends on where the clause stands. Where the clause is + in the part a trailing rule reads — a name with no comma, and + the part before a SUFFIX comma, which that rule reads the same + way — that rule is the reader. After a family comma it is the + reading the end of the given part takes, where the comma has + already settled the count and the writing decides alone. Before + a family comma, and in a part after a second one, no trailing + rule reads those words at all: the clause keeps them and says + nothing about them. + That the credential stop spares the first word after the marker + is a deliberate divergence from what certain suffix vocabulary + gets in the same position, where the marker declines and stays + an ordinary word. The marker announces a name, and this class is + the one carrying no evidence of which it is: its members are + borne surnames as well as credentials, and nobody writes a + credential straight after the marker, so a lone member reads as + the name it was announced to be. + A member ENDING a clause that some rule reads is reported where + the clause KEEPS it (S2); one the clause gives up is read like + any other word in the position it lands in, and reports where + that reading reports — which is sometimes nowhere. What the take + leaves behind decides that, not the clause: where the words in + front of the given-up member are all post-nominals or titles, + the part it lands in has no name word left in it, and a part of + nothing but credentials is read whole and asked nothing. So a + clause standing after a family comma in a part whose only other + words are post-nominals moves its member in SILENCE, agreeing + with the same name written without the clause. + A marker with nothing after it, or nothing before it, is just a word. A marker may be more than one word, and is then recognized only whole and only where its words stand together: its own first word @@ -1288,7 +1327,14 @@ M2. Rationale: a maiden marker announces that what follows it is the a reading needs is taken over the name the take would leave rather than over the words as they stand. "John née Jones Smith Ma" → maiden="Jones Smith Ma" - history: decisions.md#M2 · interacts: P2, P3, P5, R2, M1, S2, H1, H5 · implemented: nameparser/_pipeline/_group.py + Accepted: a trailing title is not transparent inside a clause, + and the two spellings disagree — the walk reads the trailing + credential run and not the title chain behind it, so a title + AFTER a member of that class hides it and a title before it + does not. + "Jane Doe nee Smith MA Prof." → maiden="Smith MA Prof." · boundary + "Jane Doe nee Smith Prof. MA" → maiden="Smith Prof." · boundary + history: decisions.md#M2 · interacts: P2, P3, P5, P6, R2, M1, S2, H1, H5 · implemented: nameparser/_pipeline/_group.py M3. Rationale: an enclosure says nothing about whether it means maiden, but a recognized marker word inside it does — the clause diff --git a/docs/release_log.rst b/docs/release_log.rst index cdbb1818..d727d4c2 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -16,10 +16,12 @@ Release Log - **New Policy field unlisted_caps_suffixes, off by default: an opt-in reading for an unlisted all-caps credential.** It reaches the core parser only -- ``Parser(policy=Policy(unlisted_caps_suffixes=True))`` -- since the field has no v1 ``Constants`` manager. With it on, ``.parse("John Smith XYZ")`` gives given ``John``, last ``Smith``, suffix ``XYZ``, and ``.parse("John Smith, XYZ")`` gives the same three fields. It is off by default because an all-caps surname is a real writing convention that shape cannot separate from a credential: ``Jean DUPONT``, ``Minjun KIM`` and ``Jean Pierre DUPONT`` are surnames in French and Korean records, and the last of those gives given ``Jean``, last ``Pierre``, suffix ``DUPONT`` with the switch on. Off, nothing changes and nothing is reported -- 1.4.0's reading for that whole class. Neither of the two new fields reaches the v1 ``Constants`` API, as ``lenient_comma_suffixes`` does not: a ``HumanName`` tracks the parser's own DEFAULTS, so the dotted reading above (default on) reaches it while this one (default off) cannot be turned on from there. See the ``S2`` entry of ``docs/design/decisions.md`` (closes #516) - - **The comma's own decision about an ambiguous credential is now reported.** ``parse("Smith, MA").ambiguities`` names ``suffix-or-name``, and so does every other decision at the ambiguous credential class -- before or after a comma, in either direction, with no new ``AmbiguityKind`` (the family-comma attachment fork already reported this way, e.g. ``parse("Berg, Jan vd")``). One report per decision: ``Smith, Ma`` reports that the word was kept as the given name just as ``Smith, MA`` reports that it was taken as a credential. The reading a SURNAME PARTICLE swallows is reported too, which no release before this one did: ``John van der Berg Ma`` gives last ``van der Berg Ma`` and names ``suffix-or-name``, where the chain took a word the credential reading had considered. ONE report goes away, because a comma segment the parser reads as a credential run is no longer called unrecognized: ``Steven Hardman, MD, DO, DDS`` no longer reports ``comma-structure``, on its written case. That is the whole of the losses over the differential corpora -- ``John Smith, MD, R.A.I.`` is quieted on its shape by the same change, but it never reported at 2.3.0 either, having only carried the flag inside this release's own development. The other movement an upgrader sees is a SWAP rather than a loss: ``Jack X.Y.I.`` reported ``given-or-family`` at 2.3.0 and reports ``suffix-or-name`` here, the dotted retirement above having handed it to the ambiguous class. Everything else at this class is a GAIN, which is what the rest of this bullet describes. The one slot this bullet left silent no longer is: the bullet below closes it, and a credential trailing the GIVEN part of a family-comma listing now reads as a credential and reports either way. See the ``S2`` and ``C1`` entries of ``docs/design/decisions.md`` + - **The comma's own decision about an ambiguous credential is now reported.** ``parse("Smith, MA").ambiguities`` names ``suffix-or-name``, and so does every other decision at the ambiguous credential class -- before or after a comma, in either direction, with no new ``AmbiguityKind`` (the family-comma attachment fork already reported this way, e.g. ``parse("Berg, Jan vd")``). One report per decision: ``Smith, Ma`` reports that the word was kept as the given name just as ``Smith, MA`` reports that it was taken as a credential. The reading a SURNAME PARTICLE swallows is reported too, which no release before this one did: ``John van der Berg Ma`` gives last ``van der Berg Ma`` and names ``suffix-or-name``, where the chain took a word the credential reading had considered. ONE report goes away, because a comma segment the parser reads as a credential run is no longer called unrecognized: ``Steven Hardman, MD, DO, DDS`` no longer reports ``comma-structure``, on its written case. That is the whole of the losses over the differential corpora -- ``John Smith, MD, R.A.I.`` is quieted on its shape by the same change, but it never reported at 2.3.0 either, having only carried the flag inside this release's own development. The other movement an upgrader sees is a SWAP rather than a loss: ``Jack X.Y.I.`` reported ``given-or-family`` at 2.3.0 and reports ``suffix-or-name`` here, the dotted retirement above having handed it to the ambiguous class. Everything else at this class is a GAIN, which is what the rest of this bullet describes. Two slots this bullet left silent no longer are, and the two bullets below close them: a credential trailing the GIVEN part of a family-comma listing now reads as a credential and reports either way, and so does one ending a maiden marker's clause. See the ``S2`` and ``C1`` entries of ``docs/design/decisions.md`` - **Fix a credential ending the given part of a family-comma listing being read as a middle name in silence.** ``HumanName("Doe, John MA")`` gives first ``John``, last ``Doe``, suffix ``MA``, where 2.0 through 2.3 gave middle ``MA`` -- and 1.4.0 gave the suffix, so this restores v1's reading for that half. The comma has already named the family and the first word after it is the given name, so the words-to-spare count that governs the comma-less form is satisfied by construction and the writing decides alone: ``Doe, John Ma`` keeps middle ``Ma``, written the way a name is written, and ``Doe, John Ed`` keeps middle ``Ed``. Either reading is now reported, and the report belongs to the SPELLING rather than to the fields -- a declined name re-rendered without its comma, ``John Ma Doe``, re-parses to those same three fields and reports nothing, the word no longer standing where the question is asked. A name word behind the credential still ends its reach and stays silent -- ``Doe, John MA Smith`` gives middle ``MA Smith`` and reports nothing -- while a credential run or a trailing title is transparent to it: ``Doe, John MA PhD`` gives suffix ``MA PhD`` and ``Doe, John MA Prof.`` gives title ``Prof.`` with suffix ``MA``. Two second-order movements an upgrader may see, both consequences of the word leaving the given part rather than of this rule reaching further: ``Doe, John Prof. MA`` now gives title ``Prof.`` where it gave middle ``Prof. MA``, the trailing-title chain reaching a word the credential used to hide; and ``Doe, John van MA`` gives last ``van Doe`` with suffix ``MA`` where it gave middle ``van MA``, the surname-particle rule reaching a particle the same way. A name written wholly in one case says nothing either way and takes the credential, which is what 1.4.0 read: ``DOE, MARY JO MA``, ``doe, john ma``, ``田中, 太郎 MA`` and ``김, 민준 MA`` all give a suffix. The unlisted dotted spelling moves with them without the parity claim -- ``Doe, John X.Y.Z.`` gives suffix ``X.Y.Z.`` where 1.4.0 and 2.3.0 both gave a middle name -- to match the comma-less ``John Doe X.Y.Z.``. One word is carved out: ``do`` is the only member of this class that is also a surname particle, so capitals decide it and the particle reading keeps every other spelling. ``Doe, John DO`` gives suffix ``DO``, while ``Doe, John do``, ``Doe, John Do``, ``DOE, JOHN DO`` and ``doe, john do`` are unchanged and keep the particle-or-given report they already had. In a name written wholly in one case the two cannot be told apart, so ``SMITH, JOHN DO`` keeps last ``DO SMITH`` as ``NASCIMENTO, EDSON ARANTES DO`` does -- right about the Portuguese record, wrong about the osteopath, and the report is how a caller finds the second. See the ``S2`` and ``P6`` entries of ``docs/design/decisions.md`` (closes #531) + - **Fix a maiden marker's clause swallowing a trailing credential in silence.** ``HumanName("Jane Doe nee Smith MA")`` gives maiden ``Smith`` with suffix ``MA``, where 2.0 through 2.3 gave maiden ``Smith MA`` and said nothing; 1.4.0 read the ``MA`` as a suffix too. ``Doe, Jane nee Smith MA`` moves with it, and so do the one-case spellings ``JANE DOE NEE SMITH MA`` and ``jane doe nee smith ma``. The words a marker takes now end where a trailing credential begins, which is what the marker's other two stops -- a suffix word, a trailing roman numeral -- have always done. Until this release it was the last trailing position in the library where a word of the ambiguous credential class was read without a report, and it was order-sensitive besides: ``Jane Doe nee Smith MA PhD`` gave maiden ``Smith MA`` while ``Jane Doe nee Smith PhD MA`` gave maiden ``Smith``, so whether the word was read at all depended on which side of the unambiguous credential the writer put it. Both now give maiden ``Smith``, with suffix ``MA PhD`` and ``PhD MA``. The writing still decides, exactly as it does for the same word ending a name with no clause: ``Jane Doe nee Smith Ma`` keeps maiden ``Smith Ma``, and ``Jane Doe nee Yo-Yo Ma`` keeps a two-word birth surname whole. The one member of this class that is also a surname particle keeps the carve-out it has outside a clause -- ``Doe, Jane nee Smith DO`` gives suffix ``DO`` while ``Doe, Jane nee Smith do`` and ``Doe, Jane nee Smith Do`` keep maiden ``Smith do`` and ``Smith Do``, and the comma-less ``Jane Doe nee Smith do`` gives suffix ``do`` as ``John Doe do`` does. Either reading is now reported, except where the reading moves the word into a part that reports nothing -- see the silent movements below. A name word behind the credential ends its reach and stays silent -- ``Jane Doe nee MA Smith`` gives maiden ``MA Smith`` and reports nothing -- and this stop never takes the first word after the marker, whatever its writing says: ``Jane Doe nee MA`` keeps maiden ``MA`` and reports, the marker having announced a name where there would otherwise be none, and ``Jane Doe nee MA PhD`` keeps it too. That differs on purpose from what a certain post-nominal gets there, ``Jane Smith nee PhD`` and ``Jane Smith nee V`` leaving the marker standing as an ordinary word as before. Where no trailing rule reads the clause's tail nothing is decided and the clause keeps every word: ``Smith nee Jones MA, Jane`` and ``Smith, John, Jr nee Jones MA`` both keep maiden ``Jones MA``, unchanged and with no ``suffix-or-name`` report. A trailing title is not transparent here and the two spellings disagree -- ``Jane Doe nee Smith MA Prof.`` is unchanged and silent while ``Jane Doe nee Smith Prof. MA`` gives maiden ``Smith Prof.`` with suffix ``MA`` -- which is recorded as a boundary rather than fixed. Two second-order movements an upgrader may see: ``Doe, Dr. nee Smith MA`` gives title ``Dr.`` with suffix ``MA`` and moves in SILENCE, the take leaving a segment with no name word for the credential-run gate to read whole -- a post-nominal does that as readily as a title, so ``Jane Doe, Jr nee Smith MA`` gives maiden ``Smith`` with suffix ``Jr MA`` and is silent too, agreeing with the clause-less ``Jane Doe, Jr MA``, while the kept direction ``Jane Doe, Jr nee Smith Ma`` still reports; and ``Berg, abdul nee Jones MA`` gives first ``abdul MA``, the bound-given join taking the released word exactly as it does in ``Berg, abdul MA``. One name is a restoration rather than a change: ``John Smith nee Jones R.A.I.`` gives suffix ``R.A.I.`` again, as 2.3.0 read it, this unreleased cycle having moved it into the maiden name when the unlisted-dotted reading above took the word out of the certain-suffix class. See the ``M2`` and ``S2`` entries of ``docs/design/decisions.md`` (closes #533) + **Additions** - **Add Lexicon.conjunctions_ambiguous, the one-letter connectives that read as initials.** A subset of ``conjunctions`` holding ``e`` by default; it is the knob for the change above rather than a switch. Portuguese data, where ``e`` links surnames the way ``y`` does in Spanish, takes it out: ``Lexicon.default().remove(conjunctions_ambiguous={"e"})`` restores the joining reading. Dutch data, where a bare single letter is an initial and never a connective, adds the other one: ``Lexicon.default().add(conjunctions_ambiguous={"y"})``. A v1 ``Constants`` has no manager of its own for it -- deleting the word from ``conjunctions`` is what turns the marking off, the same rule the glued-honorific tails follow. See ``docs/customize.rst`` (#383, #479) diff --git a/nameparser/_pipeline/_group.py b/nameparser/_pipeline/_group.py index 7bdf1ddc..d62b2bc0 100644 --- a/nameparser/_pipeline/_group.py +++ b/nameparser/_pipeline/_group.py @@ -292,7 +292,7 @@ def _maiden_take(pieces: Sequence[Sequence[int]], # flipped). THE PAIR IS THE FINDING: # over the corpus as it stood the day before this change it moved # 0 parses, and over the corpus WITH this change's own rows it - # moves 36 of 10,740 (1,790 names), on 6 distinct names ('Doe, + # moves 36 of 10,752 (1,792 names), on 6 distinct names ('Doe, # Jane nee Smith DO', 'Doe, Jane nee Smith Ma', 'Jane Doe nee # Smith Ma', 'Jane Doe nee Smith Ma JD', 'Jane Doe nee Yo-Yo Ma', # 'John née Jones Smith MA'). The plumbing was live either way; @@ -373,9 +373,13 @@ def _maiden_take(pieces: Sequence[Sequence[int]], # rather than asserted. A probe that fires wherever the tag # test admits a head this length test then DECLINES -- the # only sites where dropping it could matter -- recorded 0 over - # 21,480 parses: decisions.md#S2's population (1,790 names; + # 21,504 parses: decisions.md#S2's population (1,792 names; # the recipe is spelled out above) under six policies and two - # lexicons, the default and one listing `ph` ambiguous. Nor is + # lexicons, the default and one listing `ph` ambiguous. That 0 + # is a measured absence rather than an unreached probe: over + # the same sweep this site is reached 394 times and the TAG + # test admits the head in 310 of them, so the probe can fire + # and does not (re-measured 2026-09-19). Nor is # there a # price: with `ph` listed, dropping the test leaves 'Jane Doe # nee Smith Ph. D.' at 372 frames, 'Doe, Jane nee Smith diff --git a/nameparser/_types.py b/nameparser/_types.py index 27cca660..08517bf4 100644 --- a/nameparser/_types.py +++ b/nameparser/_types.py @@ -456,7 +456,9 @@ class AmbiguityKind(StrEnum): #: than moving it -- a member with no name word IN FRONT of it is #: not at this #: slot either, because the slot is the end of a given part and - #: there is none: a title took that position. "Doe, Dr. MA" gives + #: there is none: a TITLE or a POST-NOMINAL took that position -- + #: either one leaves the segment with no name word for the slot to + #: be the end of. "Doe, Dr. MA" gives #: suffix ``MA`` and "Doe, Mr. MA PhD" suffix ``MA PhD``, the #: credential-run gate reading those segments whole; "Doe, Dr. Ma" #: reaches the walk instead and makes ``Ma`` the given name @@ -472,6 +474,18 @@ class AmbiguityKind(StrEnum): #: credential-run gate reads whole. (The other direction still #: reports, the clause's own emitter being what raises it: "Doe, #: Dr. nee Smith Ma" keeps maiden ``Smith Ma`` and says so.) + #: What stands in front need not be a title, and a clause is the + #: commonest way to reach the POST-NOMINAL spelling: "Jane Doe, Jr + #: nee Smith MA" gives maiden ``Smith`` with suffix ``Jr MA`` and + #: moves in silence, the take leaving segment 1 as ``Jr MA`` -- + #: post-nominals only, so the same gate reads it whole. ``III`` + #: and ``PhD`` head it the same way, and each agrees with its + #: clause-less control ("Jane Doe, Jr MA" is silent too). The + #: clause's own emitter still covers the kept direction here: + #: "Jane Doe, Jr nee Smith Ma" and "Jane Doe, Jr nee MA" both + #: report. Found by the #533 design-docs review, which is the + #: measurement worth keeping: this silence was written as a + #: title's and a post-nominal reaches it too. #: And fourth -- pre-existing and untouched by 2.4 as well -- #: a JOIN beside the member can #: take it out of this slot, from either side. Where a chain has diff --git a/tests/v2/test_parser.py b/tests/v2/test_parser.py index 3cb2dd06..7a09863d 100644 --- a/tests/v2/test_parser.py +++ b/tests/v2/test_parser.py @@ -671,10 +671,59 @@ def test_the_chain_and_the_walk_stop_where_the_peel_begins() -> None: assert (n.given, n.family, n.suffix) == ("", "von Richthofen", "V") n = parse("Dr. Smith V") assert (n.given, n.family, n.suffix) == ("", "Smith", "V") - # the walk takes the numeral only: an acronym between the maiden - # name and the numeral is maiden text + # both stops read the TRAILING word, so an acronym with the + # numeral behind it is not the word either fork asks about and + # stays maiden text -- and so does a numeral with an acronym + # behind it ('Jane Doe nee Smith V MA' keeps maiden 'Smith V') n = parse("Jane Smith née Jones Ma V") assert (n.maiden, n.suffix) == ("Jones Ma", "V") + n = parse("Jane Doe nee Smith V MA") + assert (n.maiden, n.suffix) == ("Smith V", "MA") + + +def test_a_post_nominal_head_makes_the_clause_move_the_member_in_silence( +) -> None: + """#533 design-docs review: a MOVER that reports nothing. + + After a family comma the given-slot reader takes the member, but + the take can leave that segment holding post-nominals only -- no + name word for the slot to be the end of -- and a part of nothing + but credentials is read whole and asked nothing. This is the + pre-existing "no name word in FRONT of the member" silence + (``AmbiguityKind.SUFFIX_OR_NAME``'s third position), which was + written as a TITLE's and which a post-nominal reaches just as + well; `Doe, Dr. nee Smith MA` is the titled spelling. + + No rules.md example line pins it, deliberately: the name differs + from this tree at all five differential baselines and most of the + diff belongs to other changes (1.4.0 reads title 'Jr', given + 'nee', family 'Jane Doe'), so an example line would have put five + ledger rules into a docs commit. This test is the pin instead. + """ + n = parse("Jane Doe, Jr nee Smith MA") + assert (n.given, n.family, n.maiden, n.suffix) == ( + "Jane", "Doe", "Smith", "Jr MA") + assert n.ambiguities == () + # it is a MOVER: 2f57ff21 read maiden 'Smith MA' + assert n.maiden == "Smith" + # and it agrees with the same name written without the clause, + # which is why the silence is right rather than a lost report + control = parse("Jane Doe, Jr MA") + assert control.suffix == "Jr MA" + assert control.ambiguities == () + # a post-nominal, not only a generational word, heads it the same + for head in ("III", "PhD"): + n = parse(f"Jane Doe, {head} nee Smith MA") + assert (n.maiden, n.suffix) == ("Smith", f"{head} MA") + assert n.ambiguities == () + # the KEPT direction still reports -- the clause's own emitter is + # what raises it, and it never needed the given slot + for text, maiden in (("Jane Doe, Jr nee Smith Ma", "Smith Ma"), + ("Jane Doe, Jr nee MA", "MA")): + n = parse(text) + assert n.maiden == maiden + assert [a.kind for a in n.ambiguities] == [ + AmbiguityKind.SUFFIX_OR_NAME] def test_the_numeral_fork_fires_on_the_last_piece_only() -> None: diff --git a/tests/v2/test_properties.py b/tests/v2/test_properties.py index bac10e7a..000f1a42 100644 --- a/tests/v2/test_properties.py +++ b/tests/v2/test_properties.py @@ -249,9 +249,10 @@ def test_the_comma_agreement_exceptions_are_all_still_exceptions( #: contrast in the string, so the clause hides it. `own_words` stops #: at the maiden marker (rules.md#P3), so a member inside the clause #: cannot contribute the case contrast `one_case` is computed from -- -#: which is mechanisms.md's "the predicate keeps the judged token in -#: the span" failing structurally, the judged token never being in -#: the span at this slot. Accepted by Derek 2026-09-19 and recorded in +#: which is decisions.md#S2's own "THE PREDICATE KEEPS THE JUDGED +#: TOKEN IN THE SPAN" (the 2026-09-14 #289/#516 entry) failing +#: structurally, the judged token never being in the span at this +#: slot. Accepted by Derek 2026-09-19 and recorded in #: decisions.md#S2 as the M2 instance of #492's deferred question. #: The COUNT is asserted beside the class because a structural #: allowlist cannot notice a 115th member of it: 114 of 2016 pairs on diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index e07a24cf..9bab66b5 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -88,7 +88,9 @@ "Jane Doe nee MA" "Jane Doe nee MA Smith" "Jane Doe nee Smith MA" +"Jane Doe nee Smith MA Prof." "Jane Doe nee Smith Ma" +"Jane Doe nee Smith Prof. MA" "Jane Doe nee Smith X.Y.Z." "Jane Smith (Nee)" "Jane Smith (Nee) (Jones)" diff --git a/tools/differential/expected_since_1.4.0.toml b/tools/differential/expected_since_1.4.0.toml index 53aa9b99..e2c3de99 100644 --- a/tools/differential/expected_since_1.4.0.toml +++ b/tools/differential/expected_since_1.4.0.toml @@ -1940,12 +1940,15 @@ fields = ["family", "suffix"] [[change]] issue = "fix(#424/#445) accepted: the maiden walk keeps a bare acronym, and the lone name word is the family" -# 'John née Jones Smith Ma': rules.md#M2 -- "a bare acronym the peel -# would take with words to spare is maiden text all the same". middle +# 'John née Jones Smith Ma': rules.md#M2 -- "a bare acronym the +# reading declines is maiden text all the same". middle # 'née Jones', family 'Smith', suffix 'Ma' -> maiden 'Jones Smith Ma': # v1 had no maiden support; 2.0 has read the name so since #274, and -# the walk deliberately leaves the acronym fork to assign (the count -# it needs includes the words the marker removes). The fix(#274) rule +# since #533 the walk asks the acronym fork rather than leaving it: +# what declines this word is the WRITING, Title case saying name +# where capitals would say credential, and the count such a reading +# needs is taken over the name the take would leave rather than over +# the words as they stand. The fix(#274) rule # cannot carry it: its fields omit the suffix v1 read. A rules.md # Accepted example, first witnessed here. # From b45707afe08d5934848caac65a445715c36b9332 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Sat, 19 Sep 2026 20:19:10 -0700 Subject: [PATCH 4/5] fix(#533): review round -- a released word lands in suffix or the clause keeps it A. The clause could give a word up "as a credential" and the word then landed in `given`, `middle` or `family` instead -- 1,446 violating parses of 855,108 at d97d3eb7, 0 at the parent 2f57ff21, 0 here. The stop now requires the release to survive: GIVEN_SLOT needs a name word ahead of the member in the view it would leave, and either reader withdraws where a join below the marker pass would take the word (P2's chain, P5's bound-given join). `Doe, Prof. nee Smith A.B.`, `Jane Doe, Jr nee Smith MA`, `Doe, Dr. nee Smith MA`, `Berg, abdul nee Jones MA`, `Berg, Jane van der nee Smith DO` and `Jane Doe nee Smith DO DO` read as 2f57ff21 read them, and report. Pinned by a new property over an 8,466-parse grid (790 failures at d97d3eb7). Silent movers vs the parent: 4,100 tokens in 59 shapes before, 0 now. B. Delimited clauses keep the whole span and report nothing, unchanged -- documented in rules.md#M2, _types.py and three case rows. C. assert_never on the reader (with a test that reaches it and pins len(TailReader) == 3 and group()'s mapping), required keyword-only `reader`/`maiden_ambiguities`, `at = left.index(stop)`, a frame-free membership assert in `credential_at_the_given_slot`, and MaidenIndices named beside MaidenTake. D. `len(head) == 1` fires 1,440 times in a population that holds the Ph. D. shape (the recorded 0 was a population artefact) and is kept: byte-identical over 905,796 parses but +15 frames on `BERG, ABDUL Z DOMU MA PH. D.`. `stop < trailing` and the emitter's `len(last) == 1` are REMOVED -- structural, 0 hits over 1,760,904 parses, byte-identical, frame-neutral. Comment fixes: the sibling site is `segment_suffix_reading`, `peeled.names` indexes `rest`, OPTION 1 is THE FIRST-WORD FLOOR, the floor's example now bears a clause. E. Agreement test widened to three markers x three policies (1,026 of 18,144, exactly 9x, digest-pinned as a SET); the span test gains two-member tails so the maiden emitter and assign's peel actually pair (1,944 parses, 0 before); a `maiden_clause` shape guards M2's view (4.05-4.12 clean, 12.2-12.6 on a per-piece rebuild); four missing rows added. The bare delimiter core ending a clause predates this change and is recorded in decisions.md rather than fixed. Suite 9351/324/4, mypy and ruff clean, call_count 412.00/449.00, `Doe, John MA` 311 and marker-less names +0, coverage 100% on the three modules touched, and all five differential gates end 0 unexplained. Co-Authored-By: Claude Fable 5.1 --- AGENTS.md | 2 +- docs/design/decisions.md | 8 +- docs/design/rules.md | 41 ++- docs/release_log.rst | 2 +- nameparser/_pipeline/_group.py | 358 +++++++++++++------ nameparser/_pipeline/_pieces.py | 15 + nameparser/_types.py | 51 +-- tests/v2/cases.py | 190 ++++++++-- tests/v2/pipeline/test_group.py | 127 ++++++- tests/v2/test_benchmark.py | 17 + tests/v2/test_ledger_guards.py | 172 +++++---- tests/v2/test_parser.py | 71 ++-- tests/v2/test_properties.py | 272 ++++++++++++-- tools/differential/corpus_rules.jsonl | 6 + tools/differential/corpus_shapes.jsonl | 10 + tools/differential/expected_since_1.4.0.toml | 123 ++++++- tools/differential/expected_since_2.0.0.toml | 103 +++++- tools/differential/expected_since_2.1.0.toml | 103 +++++- tools/differential/expected_since_2.2.0.toml | 45 +-- tools/differential/expected_since_2.3.0.toml | 45 +-- 20 files changed, 1338 insertions(+), 423 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 95f6b211..14f83214 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -380,7 +380,7 @@ Add a dedicated `copy.deepcopy()` round-trip test for it too (see `test_regexes_ **`_normalize` must reach a fixed point** — storage and match-time share the one fold, and `Lexicon.__setstate__` re-validates, so a value that changes on re-normalization changes under its owner. `strip().strip(".")` alone is not idempotent (`'. a .'` → `' a '` → `'a'`). The loop is the fix; keep any new stripping inside it. **Anything built on `_normalize` must converge too** — `_fold_words` runs `_normalize` per word and DROPS the words that fold away (`_title_key` is that list space-joined, and `_run_addresses_by_given` reads the list itself, so its last-word arm is the last word of the FOLDED key by construction); keeping the empty slot stored `'lt .'` as `'lt '`, a key match-time can never rebuild (so the entry is silently inert) and `__setstate__` rejects on the next round-trip as "not written by this version". -**Perf regressions are caught by the scaling test, not the absolute-time ones** — `tests/v2/test_benchmark.py::test_parse_cost_grows_no_worse_than_linearly` times a repeated unit at n vs 4n over eleven shapes (one per pipeline inner loop) and bounds the ratio; the `_thousand_names` tests use constant-size, delimiter-free input and are structurally blind to a complexity regression. Two rules when touching it: calibrate `_MAX_RATIO` against the WEAKEST quadratic's signal (a mixed quadratic surfaces far below the textbook 16×, so the operating point `_BASE` matters more than the bound), and confirm a planted regression fails it across REPEATED runs — one failure is a coin-flip on a timing test. The eleven shapes cover different dimensions (segment count only via `commas`, intra-piece accumulation only via `particles`/`conjunctions`, non-ASCII input only via `honorifics` — the other ten are pure ASCII, so `script_segment` returns at its bail and the CJK stages go unmeasured); measure before pruning one. **A shape the CLOCK cannot reach needs a FRAME-count guard instead**, which is the second scaling test in that file (`test_a_trailing_credential_run_does_not_cost_exponentially`, #531): where the defect is an exponential rather than a quadratic, the input length that separates the curves on a timing test does not finish, so the guard counts frames over 8 units against 16 and bounds THAT ratio. One pair does not see every curve, and the fix round for #531 measured why: at 2× the input the per-member LINEAR work swamps a quadratic (2.08× for a genuine one against 1.73× clean), so that pair guards the exponential alone and a second, longer pair — 16 against 64, where the same quadratic reads 7.42× against 3.53× clean — is what can see one. Assert them in that order: an exponential never returns from the longer run, so the cheap pair has to have failed first. Frame counts do not move under load, so this shape needs no repeated-run calibration — but it does need the same reachability assertion `_POLICY_SHAPES` rows carry, since the walk under measurement runs only while every unit still reads as a credential. A stage gated on an opt-in `Policy` field needs a `_POLICY_SHAPES` entry instead, since bare `parse()` never enters it — and that table's rows carry a **reachability probe** run before the measurement, because a precedence change can quietly stop the shape reaching the stage and leave a green test measuring a no-op (`_POLICY_SHAPES` is also asserted non-empty: an empty `parametrize` is a skip, not a failure, so deleting its last row would retire the guard silently). +**Perf regressions are caught by the scaling test, not the absolute-time ones** — `tests/v2/test_benchmark.py::test_parse_cost_grows_no_worse_than_linearly` times a repeated unit at n vs 4n over twelve shapes (one per pipeline inner loop) and bounds the ratio; the `_thousand_names` tests use constant-size, delimiter-free input and are structurally blind to a complexity regression. Two rules when touching it: calibrate `_MAX_RATIO` against the WEAKEST quadratic's signal (a mixed quadratic surfaces far below the textbook 16×, so the operating point `_BASE` matters more than the bound), and confirm a planted regression fails it across REPEATED runs — one failure is a coin-flip on a timing test. The twelve shapes cover different dimensions (segment count only via `commas`, intra-piece accumulation only via `particles`/`conjunctions`, non-ASCII input only via `honorifics` — the other eleven are pure ASCII, so `script_segment` returns at its bail and the CJK stages go unmeasured, and M2's clause view only via `maiden_clause`, whose unit has to END on a class member: `MA nee ` holds the same two words, the peel stops at the trailing marker, and the shape reaches nothing); measure before pruning one. **A shape the CLOCK cannot reach needs a FRAME-count guard instead**, which is the second scaling test in that file (`test_a_trailing_credential_run_does_not_cost_exponentially`, #531): where the defect is an exponential rather than a quadratic, the input length that separates the curves on a timing test does not finish, so the guard counts frames over 8 units against 16 and bounds THAT ratio. One pair does not see every curve, and the fix round for #531 measured why: at 2× the input the per-member LINEAR work swamps a quadratic (2.08× for a genuine one against 1.73× clean), so that pair guards the exponential alone and a second, longer pair — 16 against 64, where the same quadratic reads 7.42× against 3.53× clean — is what can see one. Assert them in that order: an exponential never returns from the longer run, so the cheap pair has to have failed first. Frame counts do not move under load, so this shape needs no repeated-run calibration — but it does need the same reachability assertion `_POLICY_SHAPES` rows carry, since the walk under measurement runs only while every unit still reads as a credential. A stage gated on an opt-in `Policy` field needs a `_POLICY_SHAPES` entry instead, since bare `parse()` never enters it — and that table's rows carry a **reachability probe** run before the measurement, because a precedence change can quietly stop the shape reaching the stage and leave a green test measuring a no-op (`_POLICY_SHAPES` is also asserted non-empty: an empty `parametrize` is a skip, not a failure, so deleting its last row would retire the guard silently). **Expected-failure tests use `@pytest.mark.xfail`** — the conftest parametrized fixture breaks `@unittest.expectedFailure`; always use `@pytest.mark.xfail` instead. diff --git a/docs/design/decisions.md b/docs/design/decisions.md index 8cb7f2a3..2357ac48 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -200,7 +200,11 @@ the fullwidth-colon marker (旧姓：佐藤 arrives as one word; the head-peel q One limit is recorded rather than closed. The contiguity test is role plus comma bucket, and role is one-directional: a role change is always a clause edge, but two ADJACENT clauses of the same role are indistinguishable, so 'Jane (z) (domu) Jones' does tag a run across two separate nickname clauses. Nothing reads it — the piece walk never sees role-bearing tokens and the clause drop is scoped to one span — and the parse is identical with and without the phrase mechanism. The clean fix is a per-token structural UNIT id derived from state.segments and state.extracted, which would replace role-plus-bucket with one comparison and close the adjacent-clause case as a side effect. Deliberately not taken here: it changes a mechanism four correctness reviewers had just cleared, inside a pass whose stated business was not to change behavior. It is the shape to reach for if this test ever needs to grow a third term. -- 2026-09-19 (Derek), #533 — THE CLAUSE ENDS AT A TRAILING CREDENTIAL, AND THE DOUBLE CHECK THE 2026-08-22 BULLET DEFERRED NOW EXISTS FOR THE ACRONYM TOO. That bullet stopped the walk before the trailing numeral and left the bare-acronym fork alone, with its reason stated: the acronym fork COUNTS pieces and the walk removes the very pieces it counted, so a peel read over the pieces as they stand said "credential with words to spare" about `John née Jones Smith Ma` and would have left maiden 'Jones Smith', family 'Ma'. The reason was right and is unchanged; what it argued for was a deferral, and the answer is the one the numeral got — ask the peel TWICE, once over the pieces as written and once over the name the take would leave, and stop only where both read the word as the credential. "Left to assign" meant "read in silence", assign never getting the question at all, the clause's tokens carrying the maiden role before pieces exist for it: the end of a maiden clause was the last trailing position in the library where a member of the ambiguous credential class was read without a report. WHICH RULE reads the name left standing is the part the numeral's version did not have to answer, because a numeral reads the same way everywhere and a credential does not. With no comma it is the trailing peel; after a family comma it is #531's slot, where the comma has already settled the count and the writing decides alone; before that comma the words are the family the comma named and a stop would hand one of them to `family` rather than to `suffix`; past a second comma the segment is read as credentials whole and no trailing rule is consulted. The last two READ AS NONE, which is a statement and not a default — the first prototype reported on `Smith, John, Jr nee Jones MA`, where the walk had asked nothing. A FIFTH CASE, found by the design-docs review of this commit and not by the plan: after a family comma the reader may take the word and nothing may then report it. `Jane Doe, Jr nee Smith MA` is a family comma whose segment 1 is `Jr nee Smith MA`, so the given-slot reader is asked and takes the member — maiden 'Smith', suffix 'Jr MA' — but the take leaves that segment as `Jr MA`, post-nominals only, which the credential-run gate reads whole without ever reaching the emitter. It is a MOVER that reports nothing (2f57ff21 read maiden 'Smith MA'), and it is the pre-existing "no name word in FRONT of the member" silence, the same one `Doe, Dr. nee Smith MA` reaches — written as a TITLE's silence, where a post-nominal reaches it just as well. The reading is right and the silence is the documentation defect: measured over a generated sweep of clause shapes (particles, titles, connectives, numerals, by-shape and caps-on members, comma and no-comma heads) under five policies, 780 silent taken rows on 156 distinct names, every one of them on a `Jane Doe, Jr` / `Jane Doe, Jr.` / `Jane Doe, III` / `Jane Doe, PhD` head, and every one agreeing with its clause-less control on BOTH the class the member lands in and the report — 0 disagreements, which is what makes this wording rather than behavior. The same sweep found no sixth shape. The KEPT direction still reports here, the clause's own emitter being what raises it: `Jane Doe, Jr nee Smith Ma` and `Jane Doe, Jr nee MA` both say so. Pinned by a unit test rather than by a rules.md example line: the name differs from the tree at all five baselines and most of the diff is not this change's (1.4.0 reads title 'Jr', given 'nee', family 'Jane Doe'; 2.0.0 and 2.1.0 title 'Jr', family 'Jane Doe'; 2.2.0 and 2.3.0 already read given 'Jane', family 'Doe', suffix 'Jr'), so an example line would have put five ledger rules into this docs commit to attribute four other changes' readings. THE SECOND CHECK ASKS WHETHER THE READER TAKES THAT WORD, not whether it takes something, and the distinction is not academic: the numeral's existing re-ask tests whether the view's trailing run starts at the view's end, exactly right for a fork that reads one piece, and under it `JOHN NEE JONES SMITH MA PHD` leaves `JOHN MA PHD`, whose peel takes 'PHD' and then DECLINES 'MA' for want of words to spare — so the weaker question answers yes while the member becomes the FAMILY name, which is the 2026-08-22 disaster one word further on. Measured both ways on the prototype; the shipped check compares the view's run start against the member's own index in that view, and `JOHN NEE JONES SMITH MA PHD` keeps maiden 'JONES SMITH MA' with suffix 'PHD' and reports. AFTER A FAMILY COMMA THE CHECK NEEDS #531'S FLOOR, NOT ONLY ITS LEAN: `Doe, Jane nee Smith MA do` leaves `Jane MA do`, where 'MA' leans credential but the 'do' behind it does not read as a suffix (P6 keeps it, and `Doe, Jane MA do` reads middle 'MA', family 'do Doe'), so #531's slot would read 'MA' as a MIDDLE name — and a check that asked only about 'MA' released it from the clause into the current name's middle, a word crossing from one person's name to another's. With the floor the clause keeps 'MA do' whole and reports the 'do'. THE FLOOR ON THE FIRST WORD IS A CLAMP, NOT A VETO, and the first draft got two names wrong by making it one: stated as "a member that is the ONLY word after the marker stays the maiden name" and implemented as "cancel the stop when nothing would be left", it cancels too much wherever the peel consumed that first word AND words behind it — `Doe, J. nee MA ba` peels 'ba' and then 'MA', so the first piece the peel took IS the only maiden word, and the veto handed 'ba' back to the clause, giving maiden 'MA ba' where `Doe, J. ba` reads suffix 'ba'. Clamped to the piece after the marker instead, that name keeps maiden 'MA' and reads suffix 'ba', reporting both. The floor itself is deliberate and diverges from the `née V` / `née PhD` precedent, where the marker declines and stays an ordinary word: certain suffix vocabulary declines the marker, while an ambiguous word is kept by the clause it ends, because that class is the one carrying no evidence of which it is — its members are borne surnames as well as credentials, and nobody writes a credential straight after the marker. 1.4.0 read `Jane Doe nee MA` as last 'nee', suffix 'MA' (measured on the wheel), so this half restores nothing and does not try to. THE `do` PAIR is #531's reading, unchanged and now SHARED as a predicate rather than spelled twice (`credential_at_the_given_slot` in `_pipeline/_pieces.py`; the drift would have been silent, each site's own tests going on passing): `Doe, Jane nee Smith do` keeps maiden 'Smith do', `Doe, Jane nee Smith Do` keeps 'Smith Do', `Doe, Jane nee Smith DO` gives maiden 'Smith' with suffix 'DO', and the comma-less `Jane Doe nee Smith do` reads suffix 'do' — right for the reason the whole change is, since `John Doe do` reads suffix 'do' today, P6 does not run without a comma, and a clause must not change how a word outside it reads. ACCEPTED COSTS, each measured on this tree: `JANE DOE NEE YO-YO MA` reads maiden 'YO-YO', suffix 'MA', a two-word birth surname losing its last word in a one-case record — and `JANE YO-YO MA` reads suffix 'MA' too, so the cost is the one-case reading's rather than this rule's, and in mixed case the writing saves it (`Jane Doe nee Yo-Yo Ma` keeps maiden 'Yo-Yo Ma' and reports); `Jane Doe nee Smith X.Y.Z.` reads suffix where 1.4.0 read last 'X.Y.Z.', the same trade #531 took at its own slot and for agreement with `John Doe X.Y.Z.` rather than for parity; `Doe, Dr. nee Smith MA` MOVES IN SILENCE to title 'Dr.', maiden 'Smith', suffix 'MA', the take leaving a segment with no name word which the credential-run gate reads whole; `Berg, abdul nee Jones MA` gives given 'abdul MA', P5's lenient post-comma join taking the released member into the bound-given pair exactly as it does in the clause-less `Berg, abdul MA`, so the clause form now AGREES with the bare form; `Jane Smith nee Jones and MA` reads maiden 'Jones and' with suffix 'MA', the take running before the joins (#420) so the connective is a piece of its own when the walk stops; and a report on a declined word adds noise without moving a field, which is #530's stated rule and the trade #531 took at its slot. NOT TRANSPARENT HERE: a trailing title. `Jane Doe nee Smith MA Prof.` is unchanged and silent while `Jane Doe nee Smith Prof. MA` gives maiden 'Smith Prof.' with suffix 'MA' and reports, because the walk reads the trailing credential run and the H5 title chain is read by the rule the walk does not run. Accepted for #533, pinned as a pair of boundary example lines under M2; moving the walk onto the title-aware reading would change the numeral half too, which the 2026-08-22 bullet settled with its own measurements, and the pair belongs to one decision about what "trailing" means inside a clause — a follow-up is drafted for Derek rather than filed here. THE `one_case` PLUMBING AT THESE SITES IS NOW LIVE, and the 2026-09-18 bullet under `### S2` headed THE PREDICTION THAT DID NOT SURVIVE ITS OWN MEASUREMENT stands as what was true then: `numeral_only` answered off the peeled numeral and the numeral fork is decided before the peel reads a lean, so the fact reached only the bare-acronym fork, which that reading discarded. The acronym fork is asked now. Re-measured 2026-09-19 with the same wrapper shape over the same population that entry's recipe names — AND THE PAIR IS THE FINDING: over the corpus as it stood the day before this change, dropping the argument inside the walk moved 0 parses; over the corpus WITH this change's own rows it moves 36 of 10,752 (1,792 names), on six distinct names (`Doe, Jane nee Smith DO`, `Doe, Jane nee Smith Ma`, `Jane Doe nee Smith Ma`, `Jane Doe nee Smith Ma JD`, `Jane Doe nee Yo-Yo Ma`, `John née Jones Smith MA`). The population, the six policies and the denominator are recorded at the call site in `_pipeline/_group.py`, as the 2026-09-18 entry records its own; both were re-run on 2026-09-19 after the review found the figures first written there (1,790 names, 10,740 and 21,480 parses) did not reproduce from the recipe as stated — they had been taken two corpus names earlier in the same day's work. The FINDING was unharmed, 36 parses on the same six names. The plumbing was live either way and the corpus could not show it, which is the blindness mechanisms.md's corpus field note asks to be measured before any "N names move" is written down. +- 2026-09-19 (Derek), #533 — THE CLAUSE ENDS AT A TRAILING CREDENTIAL, AND THE DOUBLE CHECK THE 2026-08-22 BULLET DEFERRED NOW EXISTS FOR THE ACRONYM TOO. That bullet stopped the walk before the trailing numeral and left the bare-acronym fork alone, with its reason stated: the acronym fork COUNTS pieces and the walk removes the very pieces it counted, so a peel read over the pieces as they stand said "credential with words to spare" about `John née Jones Smith Ma` and would have left maiden 'Jones Smith', family 'Ma'. The reason was right and is unchanged; what it argued for was a deferral, and the answer is the one the numeral got — ask the peel TWICE, once over the pieces as written and once over the name the take would leave, and stop only where both read the word as the credential. "Left to assign" meant "read in silence", assign never getting the question at all, the clause's tokens carrying the maiden role before pieces exist for it: the end of a maiden clause was the last trailing position in the library where a member of the ambiguous credential class was read without a report. WHICH RULE reads the name left standing is the part the numeral's version did not have to answer, because a numeral reads the same way everywhere and a credential does not. With no comma it is the trailing peel; after a family comma it is #531's slot, where the comma has already settled the count and the writing decides alone; before that comma the words are the family the comma named and a stop would hand one of them to `family` rather than to `suffix`; past a second comma the segment is read as credentials whole and no trailing rule is consulted. The last two READ AS NONE, which is a statement and not a default — the first prototype reported on `Smith, John, Jr nee Jones MA`, where the walk had asked nothing. A FIFTH CASE, found by the design-docs review of this commit and not by the plan, and THE PR REVIEW THEN SETTLED IT THE OTHER WAY — see the review bullet below, which is where the reasoning now lives. The shape: after a family comma the reader may take the word and nothing may then report it. `Jane Doe, Jr nee Smith MA` is a family comma whose segment 1 is `Jr nee Smith MA`, so the given-slot reader is asked and would take the member, but the take leaves that segment as `Jr MA`, post-nominals only, which the credential-run gate reads whole without ever reaching the emitter — and the released word therefore lands in `given`, not in `suffix`. This commit read it as maiden 'Smith', suffix 'Jr MA', silently; the review round declines the stop instead, so the clause keeps 'Smith MA' and reports, which is 2f57ff21's reading. The KEPT direction always reported here, the clause's own emitter being what raises it: `Jane Doe, Jr nee Smith Ma` and `Jane Doe, Jr nee MA` both say so, and now so does `Jane Doe, Jr nee Smith MA`. Pinned by a unit test rather than by a rules.md example line: the name differs from the tree at all five baselines and most of the diff is not this change's (1.4.0 reads title 'Jr', given 'nee', family 'Jane Doe'; 2.0.0 and 2.1.0 title 'Jr', family 'Jane Doe'; 2.2.0 and 2.3.0 already read given 'Jane', family 'Doe', suffix 'Jr'), so an example line would have put five ledger rules into this docs commit to attribute four other changes' readings. THE SECOND CHECK ASKS WHETHER THE READER TAKES THAT WORD, not whether it takes something, and the distinction is not academic: the numeral's existing re-ask tests whether the view's trailing run starts at the view's end, exactly right for a fork that reads one piece, and under it `JOHN NEE JONES SMITH MA PHD` leaves `JOHN MA PHD`, whose peel takes 'PHD' and then DECLINES 'MA' for want of words to spare — so the weaker question answers yes while the member becomes the FAMILY name, which is the 2026-08-22 disaster one word further on. Measured both ways on the prototype; the shipped check compares the view's run start against the member's own index in that view, and `JOHN NEE JONES SMITH MA PHD` keeps maiden 'JONES SMITH MA' with suffix 'PHD' and reports. AFTER A FAMILY COMMA THE CHECK NEEDS #531'S FLOOR, NOT ONLY ITS LEAN: `Doe, Jane nee Smith MA do` leaves `Jane MA do`, where 'MA' leans credential but the 'do' behind it does not read as a suffix (P6 keeps it, and `Doe, Jane MA do` reads middle 'MA', family 'do Doe'), so #531's slot would read 'MA' as a MIDDLE name — and a check that asked only about 'MA' released it from the clause into the current name's middle, a word crossing from one person's name to another's. With the floor the clause keeps 'MA do' whole and reports the 'do'. THE FLOOR ON THE FIRST WORD IS A CLAMP, NOT A VETO, and the first draft got two names wrong by making it one: stated as "a member that is the ONLY word after the marker stays the maiden name" and implemented as "cancel the stop when nothing would be left", it cancels too much wherever the peel consumed that first word AND words behind it — `Doe, J. nee MA ba` peels 'ba' and then 'MA', so the first piece the peel took IS the only maiden word, and the veto handed 'ba' back to the clause, giving maiden 'MA ba' where `Doe, J. ba` reads suffix 'ba'. Clamped to the piece after the marker instead, that name keeps maiden 'MA' and reads suffix 'ba', reporting both. The floor itself is deliberate and diverges from the `née V` / `née PhD` precedent, where the marker declines and stays an ordinary word: certain suffix vocabulary declines the marker, while an ambiguous word is kept by the clause it ends, because that class is the one carrying no evidence of which it is — its members are borne surnames as well as credentials, and nobody writes a credential straight after the marker. 1.4.0 read `Jane Doe nee MA` as last 'nee', suffix 'MA' (measured on the wheel), so this half restores nothing and does not try to. THE `do` PAIR is #531's reading, unchanged and now SHARED as a predicate rather than spelled twice (`credential_at_the_given_slot` in `_pipeline/_pieces.py`; the drift would have been silent, each site's own tests going on passing): `Doe, Jane nee Smith do` keeps maiden 'Smith do', `Doe, Jane nee Smith Do` keeps 'Smith Do', `Doe, Jane nee Smith DO` gives maiden 'Smith' with suffix 'DO', and the comma-less `Jane Doe nee Smith do` reads suffix 'do' — right for the reason the whole change is, since `John Doe do` reads suffix 'do' today, P6 does not run without a comma, and a clause must not change how a word outside it reads. ACCEPTED COSTS, each measured on this tree: `JANE DOE NEE YO-YO MA` reads maiden 'YO-YO', suffix 'MA', a two-word birth surname losing its last word in a one-case record — and `JANE YO-YO MA` reads suffix 'MA' too, so the cost is the one-case reading's rather than this rule's, and in mixed case the writing saves it (`Jane Doe nee Yo-Yo Ma` keeps maiden 'Yo-Yo Ma' and reports); `Jane Doe nee Smith X.Y.Z.` reads suffix where 1.4.0 read last 'X.Y.Z.', the same trade #531 took at its own slot and for agreement with `John Doe X.Y.Z.` rather than for parity; (two costs listed here were WITHDRAWN by the PR review and are no longer costs at all — `Doe, Dr. nee Smith MA` and `Berg, abdul nee Jones MA` both keep their maiden words and report; the bullet below carries the reason); `Jane Smith nee Jones and MA` reads maiden 'Jones and' with suffix 'MA', the take running before the joins (#420) so the connective is a piece of its own when the walk stops; and a report on a declined word adds noise without moving a field, which is #530's stated rule and the trade #531 took at its slot. NOT TRANSPARENT HERE: a trailing title. `Jane Doe nee Smith MA Prof.` is unchanged and silent while `Jane Doe nee Smith Prof. MA` gives maiden 'Smith Prof.' with suffix 'MA' and reports, because the walk reads the trailing credential run and the H5 title chain is read by the rule the walk does not run. Accepted for #533, pinned as a pair of boundary example lines under M2; moving the walk onto the title-aware reading would change the numeral half too, which the 2026-08-22 bullet settled with its own measurements, and the pair belongs to one decision about what "trailing" means inside a clause — a follow-up is drafted for Derek rather than filed here. THE `one_case` PLUMBING AT THESE SITES IS NOW LIVE, and the 2026-09-18 bullet under `### S2` headed THE PREDICTION THAT DID NOT SURVIVE ITS OWN MEASUREMENT stands as what was true then: `numeral_only` answered off the peeled numeral and the numeral fork is decided before the peel reads a lean, so the fact reached only the bare-acronym fork, which that reading discarded. The acronym fork is asked now. Re-measured 2026-09-19 with the same wrapper shape over the same population that entry's recipe names — AND THE PAIR IS THE FINDING: over the corpus as it stood the day before this change, dropping the argument inside the walk moved 0 parses; over the corpus WITH this change's own rows it moves 36 of 10,752 (1,792 names), on six distinct names (`Doe, Jane nee Smith DO`, `Doe, Jane nee Smith Ma`, `Jane Doe nee Smith Ma`, `Jane Doe nee Smith Ma JD`, `Jane Doe nee Yo-Yo Ma`, `John née Jones Smith MA`). The population, the six policies and the denominator are recorded at the call site in `_pipeline/_group.py`, as the 2026-09-18 entry records its own; both were re-run on 2026-09-19 after the review found the figures first written there (1,790 names, 10,740 and 21,480 parses) did not reproduce from the recipe as stated — they had been taken two corpus names earlier in the same day's work. The FINDING was unharmed, 36 parses on the same six names. The plumbing was live either way and the corpus could not show it, which is the blindness mechanisms.md's corpus field note asks to be measured before any "N names move" is written down. + +- 2026-09-19 (Derek), #533 review — A WORD THE CLAUSE GIVES UP READS AS A POST-NOMINAL OR THE CLAUSE KEEPS IT, AND THAT INVARIANT REPLACES THE ACCEPTED SILENT MOVERS ABOVE. The PR review measured what the released words actually do, and three of them do not land in `suffix`: `Doe, Prof. nee Smith A.B.` read given 'A.B.' (66 distinct names in the review's own sweep, the same shape reaching `ba`, `X.Y.Z.`, caps-on `XYZ` and the lower-case spellings); `Berg, Jane van der nee Smith DO` read family 'van der DO Berg' and `Jane Doe nee Smith DO DO` read family 'DO DO', a word crossing from the BIRTH name into the current one, which is #424's failure from the other side; and `Berg, abdul nee Jones MA` read given 'abdul MA'. Two of those had been written up as ACCEPTED COSTS in the bullet above, on the argument that the clause form AGREES with the clause-less one. The agreement is real and it is not the question: the clause-less `Berg, abdul MA` puts the word in the given name because nobody promised otherwise, while a CLAUSE promised that the word was either the birth name or a credential, and a third answer is the promise broken in silence. So the direction is the parent's, conservative: THE STOP IS RIGHT ONLY WHERE THE RELEASED WORD ENDS THE PARSE SUFFIX-ROLED, and where the take cannot promise that, the clause keeps the word (and reports it, the emitter asking whether a trailing rule reads these words at all rather than whether the view check then passed). Two view checks carry it, each modelling a LATER decision rather than predicting one: the given-slot reader requires the view to still hold a name word AHEAD of the member, since a trailing slot needs a part to be the end of; and a release is withdrawn where a join reached below the marker pass would absorb the word — P2's chain, for a particle-tagged member behind a particle piece or beside another released one, and P5's bound-given join, for the word after the bound one. Both over-decline rather than predict, which is the direction the invariant asks for. MEASURED, and the pair of numbers is the argument: over the review's 142,518-name corpus under six policies, 855,108 parses, the invariant holds 0 violations at 2f57ff21 and 0 here, against 1,446 in 40 shapes at the commit this round started from; and a token-level diff against 2f57ff21 finds 128,424 moved parses on 24,958 names with ZERO of the moved tokens unnamed by a `suffix-or-name` report, against 4,100 silent moved tokens in 59 shapes before. `tests/v2/test_properties.py` carries the invariant as a property over a generated grid rich enough to hold every shape above; the grid fails on 790 of its 8,466 parses at d97d3eb7 and passes at 2f57ff21, which is what makes it a test rather than a restatement. TWO ACCEPTED SILENCES REMAIN AND ARE NOT SILENCES OF THIS KIND: a member with a name word BEHIND it was never asked about, and a delimited clause is not a fork at all (below). ONE ROUGH EDGE IS RECORDED RATHER THAN FIXED, and it PREDATES this change: with `extra_suffix_delimiters=('-',)`, `Jane Doe nee Smith - MA` gives maiden 'Smith -' — a clause ending on a bare delimiter token. 2f57ff21 does the same thing one word later (`Jane Doe nee Smith - PhD` gives maiden 'Smith -' there too), so what #533 changes is only which words reach the edge. Walking the stop back over it is not the two-line change it looks like: the delimiter-core set the walk already steps over is EMPTY off a tail segment, which is exactly where this name's `-` sits, so the fix would have to widen what counts as a core on every path and would move the parent's reading as well. Left for its own issue. + +- 2026-09-19 (Derek), #533 review — THE DELIMITERS TRUMP ANY OTHER READING, and that is a settled position rather than an un-asked fork. `Jane Doe (nee Smith Ma)` and `Jane Doe (nee Smith MA)` both keep the WHOLE span as the maiden name and report nothing, in every release including this one; the marker INSIDE the delimiters is what tells the parser the span is a maiden clause, even where the pair is not configured in `maiden_delimiters` (M3). The writer drew the boundary, so no fork is called and there is nothing to report — which is a different kind of quiet from the silences M2 and `AmbiguityKind.SUFFIX_OR_NAME` document, where a fork WAS available and the parser declined to ask it. The boundary cuts both ways: `Jane Doe (nee Smith) MA` gives suffix 'MA' and reports, the member standing outside the span being an ordinary trailing credential. Behavior is unchanged from 2f57ff21 and was not touched; what the review added is the three rules.md example lines, the case rows, and the statement in M2 and in the `SUFFIX_OR_NAME` docstring, because an undocumented settled position reads as an oversight to the next person who measures it. ### N3 — the lone-word nickname rule @@ -604,7 +608,7 @@ for n in ('Smith, John','Smith, XYZ'): print(n, calls_for(off.parse, n), calls_f - 2026-09-18, VERIFICATION ROUND — THE REPORT INVENTORY AN UPGRADER SEES, 2.3.0 → TREE, AND ITS RECIPE. docs/release_log.rst's comma-report bullet claimed TWO reports go away and named `Steven Hardman, MD, DO, DDS` and `John Smith, MD, R.A.I.`. The second is false and was never measured against the wheel: `John Smith, MD, R.A.I.` reports NOTHING at 2.3.0, having carried a `comma-structure` flag only inside this arc's own development, where rules.md#S3's narrow retirement raised it and the C2 amendment quieted it again. Over the whole corpus exactly ONE report is lost, and one more is SWAPPED. Measured: LOST — `comma-structure` ×1 (`Steven Hardman, MD, DO, DDS`), `given-or-family` ×2 (`Jack X.Y.I.`, and `e and e`, which is PR #527's conjunction change and not this arc's). GAINED — `suffix-or-name` ×37, `conjunction-or-initial` ×9 (#527's again), `given-or-family` ×2 (`Jack MA`, `Jack MA.`). So `Jack X.Y.I.` is the only name whose report CHANGED KIND on this arc's account, `given-or-family` → `suffix-or-name`, and the bullet now names it as the swap rather than counting it among the losses. Recompute: collect the corpus glob through compare.py's `_load_entries` deduped by name, parse each name on the tree and on the 2.3.0 wheel (a PEP 723 `uv run --no-project` script pinning `nameparser==2.3.0`, per AGENTS.md's warning that a same-tree comparison reports parity), and take the per-name multiset difference of the `ambiguities` kind values in both directions. The kind counts are multisets, not name counts — `e and e` contributes two `conjunction-or-initial` gains on its own — which is why a bullet quantified from the gate's per-rule name lists would not answer this question. - 2026-09-18 (Derek), #531 — CAPITALS DECIDE FOR `do`, AND THE PARTICLE RULE KEEPS EVERY OTHER SPELLING. `do` is the one member of the ambiguous credential class (`ba`, `do`, `ed`, `jd`, `ma` — recompute with `sorted(Lexicon.default().suffix_acronyms_ambiguous)`) that is also particle vocabulary, so the trailing slot of the given part and P6's attachment want the same word. Where the writing carries a positive credential lean — an all-caps member in a name written in more than one case — the credential reading wins and P6 stands down: `Doe, John DO` reads suffix `DO` and reports `suffix-or-name`. Every other spelling attaches exactly as it did before, with P6's own `particle-or-given` and no second report: `Doe, John do`, `Doe, John Do`, `DOE, JOHN DO`, `doe, john do` all read family `do Doe` in their own case. THE PAIRING IS THE ARGUMENT, and the accepted cost and the reason are the two halves of it. In ONE CASE the rule cannot tell `NASCIMENTO, EDSON ARANTES DO` from `SMITH, JOHN DO` and reads both as the particle — right about the Brazilian record, wrong about the osteopath, whose `DO` 1.4.0 read as a suffix; both report `particle-or-given`, which is how a caller finds the second. In MIXED CASE it can, and does: `Nascimento, Edson Arantes do` keeps family `do Nascimento` while `Doe, John DO` gives suffix `DO`. Accepted rather than repaired: the one-case reading is the commoner of the two collisions, and the Nascimento record is the name a wrong answer would damage. The other four members are not particles and are unaffected. This is the exception rules.md#P6's "this attachment outranks the suffix reading (S2)" clause now names. Implementing it takes TWO carve-outs rather than one, which is worth recording because the second is in the REPORT and not the reading: the reading carve-out takes a particle-tagged member only on a POSITIVE credential lean, since `Doe, John do` leans nothing and the positional reading would otherwise take it — the wrong ANSWER, not merely a stray report; and the emitter beside it skips a particle-tagged member it did not itself take, since an emitter gated on tag membership alone reported both kinds on every `do` row. Where P6 owns the fork, P6 reports it, which is mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE read strictly. ONE SHAPE THE PAIRING DOES NOT REACH — a shape, not a spelling, and the difference is the review finding: ANY particle or particle chain standing in front of the caps `DO` takes it out of this slot, because grouping's prefix chain merges the two into ONE particle piece before the slot exists and there is then no lone member for the capitals to lean on. It is a fact about this WORD being particle vocabulary itself, not about particles generally — the caps `MA` in `Doe, John van MA` is not swallowed, reads as the credential, and leaves `van` to attach behind it. `Doe, John van DO` reads family `van DO Doe`, and so do the `de`, `von`, `del`, `la` and `van der` spellings of it (measured 2026-09-19 — parse them and read `family`). Pre-existing and unmoved by #531; `Doe, John van DO` is the case row and rules.md#P6's example line, and P6's statement carries the carve-out. The mirror shape is silent rather than merely unreached and is recorded on `AmbiguityKind.SUFFIX_OR_NAME`'s fourth boundary: a particle BEHIND the member ends the given part where this walk reads it, so `Doe, John MA do` keeps middle `MA` beside family `do Doe` though the capitals would otherwise have taken it, the attachment that moves `do` running a stage later. - 2026-09-18 (Derek), #531 — THE COMMA REPORT'S REACH NOW INCLUDES THE GIVEN SEGMENT'S TRAILING SLOT, AND THE OPEN FOLLOW-UP IS CLOSED. The 2026-09-18 bullet above headed THE COMMA REPORT'S REACH IS THE FIRST POST-COMMA PIECE recorded this slot as an open maintainer decision and named the two questions it turned on — whether a middle initial's neighbourhood should start reporting (a noise judgement, rules.md#A1) and whether the silence was a 1.4 parity gap. Both are answered here, and that bullet stands as it landed. The slot now reads and reports: `Doe, John MA` gives suffix `MA` and `Doe, John Ma` keeps middle `Ma`, each saying which way it went. Derek chose to restore the ROLE and report both ways — one rule for both spellings — over a report-only change and over a capitals-only one, because the issue exists in the first place because two spellings of one name disagree. The noise question was settled by MEASURING THE DISAGREEMENT rather than by argument: over a generated sweep of 78 pairs — five listed members and two by-shape tokens in three cased spellings each, plus five controls in one spelling apiece, so 26 words against three name shapes — 48 pairs disagreed about whether the word was a credential or a name, and every one of the 48 disagreed in the same direction, the comma form declining what the comma-less form took. After this change 3 disagree and all three are the lower-case `do` rows P6 owns. The sweep and its allowlist are `tests/v2/test_properties.py`; a slot that answers differently from the same name written without a comma is not a quiet slot, it is an inconsistent one. Recorded under `3-0-reevaluations`' standing rule because v1 parity is LOAD-BEARING for one half of this and explicitly NOT for the other: `Doe, John MA` reads suffix `MA` on the 1.4.0 wheel, so the bare-acronym half RESTORES v1's role and the report is all that is new there — its 1.4.0 ledger rule neither retired nor narrowed to `_ambiguities` (nothing below baseline 2.0 can diff on that pseudo-field) but was RE-POINTED to the four names where the writing declines the credential, which keep the 2.0-era middle name against v1. `Doe, John X.Y.Z.` reads middle `X.Y.Z.` at 1.4.0 too, so the dotted half LEAVES v1 and carries a 1.4.0 ledger rule of its own; it moves to match the comma-less `John Doe X.Y.Z.` and rules.md#S3's shape rule, not to restore anything. BLAST RADIUS, stated the way this log's own rule asks: twenty corpus names move, and only TWO of them were in any corpus before this branch (`Doe, John MA` and `Doe, John X.Y.Z.`, both admitted by #530's own arc) — the other eighteen are this change's own case rows, so what the differential measures on pre-existing data is two names, and the population the rule reaches is a SHAPE (every family-comma listing whose given part ends in a member of this class) that the corpora barely sample. ACCEPTED COSTS, all measured on the differential corpora: `SMITH, JOHN DO` keeps family `DO SMITH` where 1.4.0 read suffix `DO`, paired with the Nascimento record in the bullet above; one-case and caseless names take the credential with no case evidence at all, so `DOE, MARY JO MA`, `doe, john ma`, `田中, 太郎 MA` and `김, 민준 MA` all read a suffix, which is what 1.4.0 read for each of the four — "caseless is inert" holds for the LEAN and not for the outcome, since `is_one_case` answers True for a script with no case and the positional reading then decides; `Doe, John van MA` loses its middle to the family, reading family `van Doe`, suffix `MA` with two reports where it read middle `van MA` in silence, which is `Berg, Jan van Jr.`'s reading arriving through a shape it could not reach before (Derek accepted it 2026-09-18 as P6 working correctly rather than as a cascade to carve out); `Doe, John Prof. MA` gains a TITLE role `Prof.` never had, H5's transparency reaching it once `MA` leaves the walk, landing it on the same answer as `Doe, John MA Prof.`; `Smith, LEED AP` moves under the default-off caps switch alone, to given `LEED`, family `Smith`, suffix `AP` with two reports, so no default reading is at stake; and a genuine middle name that is also a class member is now a credential wherever it ends the given part and is not Title-cased — `DOE, JOHN ED` reads suffix `ED` — which is the same cost the comma-less form has carried since 2.0. NOT REPAIRED HERE and left open: the maiden walk claims `Smith MA` whole in `Doe, Jane nee Smith MA` before this slot exists, so the rule cannot reach it and the name stays silent with maiden `Smith MA` — the same gap #530's close-out saw from the other side with `John Smith nee Jones R.A.I.`, and it is out of scope for #531. -- 2026-09-19 (Derek), #533 — THE CLASS'S LAST SILENT TRAILING POSITION IS CLOSED, AND #531'S READING NOW LIVES IN ONE PLACE. The slot list in rules.md#S2 gains the trailing slot of a maiden marker's clause, and #S3's enumeration gains it with the by-shape spellings; the rule that governs what the clause does with those words is M2's, and the entry under `### M2` above is where its reasoning lives. The gap the bullet above left open as out of scope for #531 is the one this closes, from both sides: `Doe, Jane nee Smith MA` now gives maiden 'Smith' with suffix 'MA' and reports, and `John Smith nee Jones R.A.I.` gives suffix 'R.A.I.' again — a RESTORATION rather than a change, since 2.3.0 read it that way (measured on the wheel) and this unreleased cycle moved it into the maiden name when #516 took dotted tokens out of the certain-suffix class, where no corpus file held the name and no gate could see it. Two things belong here rather than under M2. FIRST, `credential_at_the_given_slot` in `_pipeline/_pieces.py` is now where #531's reading of a class member ending the given part is written, with two callers — assign's walk over that part, and the maiden walk's second check over the name a take would leave. Spelling it twice is precisely the "condition written to match it" mechanisms.md#ONE-PREDICATE-PER-QUESTION names, and the drift would have been silent, since each site's own tests would have gone on passing. It is a text-and-tags question, which is what puts it in `_pieces` rather than beside either caller — the destination follows the LAYER, not the topic. It costs one frame PER MEMBER asked at that slot: measured 2026-09-19 against 2f57ff21 per `Parser.parse`, `Doe, John MA` goes 310 → 311 and `Doe, John MA Ma MA`, which asks four times, 439 → 443, while a name with no member there never reaches it and pays nothing. The reference band does not move (412/449 on `uv run python tools/perf/call_count.py`), no test pins 310, and Derek took the trade rather than keep two conditions in step across two stages with no test that asks them both. The maiden path got one frame CHEAPER in the same pass, the numeral reading calling the peel pair it had wrapped rather than the wrapper: `Jane Doe nee Smith` 249 → 248. SECOND, THE ONE-CASE HEAD IS AN ACCEPTED EXCEPTION AND IT IS THE M2 INSTANCE OF #492'S DEFERRED QUESTION about whether a cased suffix token counts as case evidence. `DOE, JANE nee Smith Ma` reads suffix 'Ma' where the clause-less `DOE, JANE Ma` keeps middle 'Ma', because the own-words span stops at the marker — rules.md#P3 puts "a maiden marker's run and every word after it" outside the name's own words from the moment the marker is tagged — so the member's own Title-casing is not in the span `one_case` is computed over and the clause HIDES the contrast. That is THE PREDICATE KEEPS THE JUDGED TOKEN IN THE SPAN, the 2026-09-14 #289/#516 entry at the head of this section, failing structurally rather than by oversight: at this slot the judged token is never in the span, and that entry's own answer — include it — cannot be had here. Widening the span for this one question would change `one_case` for the whole name, and three sites read it, so the exception is accepted instead. Measured 2026-09-19: 114 of 2016 generated pairs — every listed member and both by-shape spellings, in three cased spellings, against sixteen heads and six clause bodies of NAME WORDS ONLY — against 186 allowlisted and 984 disagreeing outside the class before the change, and 0 outside it after. `tests/v2/test_properties.py` carries the sweep with the class defined STRUCTURALLY (`one_case` true of the clause form and false of the clause-less one) rather than as a name list, plus the recorded count asserted beside it, because a structural allowlist cannot notice its own growth. +- 2026-09-19 (Derek), #533 — THE CLASS'S LAST SILENT TRAILING POSITION IS CLOSED, AND #531'S READING NOW LIVES IN ONE PLACE. The slot list in rules.md#S2 gains the trailing slot of a maiden marker's clause, and #S3's enumeration gains it with the by-shape spellings; the rule that governs what the clause does with those words is M2's, and the entry under `### M2` above is where its reasoning lives. The gap the bullet above left open as out of scope for #531 is the one this closes, from both sides: `Doe, Jane nee Smith MA` now gives maiden 'Smith' with suffix 'MA' and reports, and `John Smith nee Jones R.A.I.` gives suffix 'R.A.I.' again — a RESTORATION rather than a change, since 2.3.0 read it that way (measured on the wheel) and this unreleased cycle moved it into the maiden name when #516 took dotted tokens out of the certain-suffix class, where no corpus file held the name and no gate could see it. Two things belong here rather than under M2. FIRST, `credential_at_the_given_slot` in `_pipeline/_pieces.py` is now where #531's reading of a class member ending the given part is written, with two callers — assign's walk over that part, and the maiden walk's second check over the name a take would leave. Spelling it twice is precisely the "condition written to match it" mechanisms.md#ONE-PREDICATE-PER-QUESTION names, and the drift would have been silent, since each site's own tests would have gone on passing. It is a text-and-tags question, which is what puts it in `_pieces` rather than beside either caller — the destination follows the LAYER, not the topic. It costs one frame PER MEMBER asked at that slot: measured 2026-09-19 against 2f57ff21 per `Parser.parse`, `Doe, John MA` goes 310 → 311 and `Doe, John MA Ma MA`, which asks four times, 439 → 443, while a name with no member there never reaches it and pays nothing. The reference band does not move (412/449 on `uv run python tools/perf/call_count.py`), no test pins 310, and Derek took the trade rather than keep two conditions in step across two stages with no test that asks them both. The maiden path got one frame CHEAPER in the same pass, the numeral reading calling the peel pair it had wrapped rather than the wrapper: `Jane Doe nee Smith` 249 → 248. SECOND, THE ONE-CASE HEAD IS AN ACCEPTED EXCEPTION AND IT IS THE M2 INSTANCE OF #492'S DEFERRED QUESTION about whether a cased suffix token counts as case evidence. `DOE, JANE nee Smith Ma` reads suffix 'Ma' where the clause-less `DOE, JANE Ma` keeps middle 'Ma', because the own-words span stops at the marker — rules.md#P3 puts "a maiden marker's run and every word after it" outside the name's own words from the moment the marker is tagged — so the member's own Title-casing is not in the span `one_case` is computed over and the clause HIDES the contrast. That is THE PREDICATE KEEPS THE JUDGED TOKEN IN THE SPAN, the 2026-09-14 #289/#516 entry at the head of this section, failing structurally rather than by oversight: at this slot the judged token is never in the span, and that entry's own answer — include it — cannot be had here. Widening the span for this one question would change `one_case` for the whole name, and three sites read it, so the exception is accepted instead. Measured 2026-09-19: 114 of 2016 generated pairs — every listed member and both by-shape spellings, in three cased spellings, against sixteen heads and six clause bodies of NAME WORDS ONLY — against 186 allowlisted and 984 disagreeing outside the class before the change, and 0 outside it after. The PR review widened that grid to three markers (`nee`, `née`, `geb.`) and three policies (the default and each 2.4 switch), and the class is INDIFFERENT to both: 1,026 of 18,144, exactly 9x the original in both columns, and still 0 disagreeing outside it. `tests/v2/test_properties.py` carries the sweep with the class defined STRUCTURALLY (`one_case` true of the clause form and false of the clause-less one) rather than as a name list. The recorded control beside it is now the SET and not only its size, as a digest over the members: a count cannot notice a swap, one pair leaving and another arriving, which is the same blindness the count was added to close one level up. ### indic-honorifics — the renunciate class and the Indic honorific vocabulary (2026-09-06, #346/#344/#343) diff --git a/docs/design/rules.md b/docs/design/rules.md index 46d697ca..b17c6769 100644 --- a/docs/design/rules.md +++ b/docs/design/rules.md @@ -1260,16 +1260,31 @@ M2. Rationale: a maiden marker announces that what follows it is the credential straight after the marker, so a lone member reads as the name it was announced to be. A member ENDING a clause that some rule reads is reported where - the clause KEEPS it (S2); one the clause gives up is read like - any other word in the position it lands in, and reports where - that reading reports — which is sometimes nowhere. What the take - leaves behind decides that, not the clause: where the words in - front of the given-up member are all post-nominals or titles, - the part it lands in has no name word left in it, and a part of - nothing but credentials is read whole and asked nothing. So a - clause standing after a family comma in a part whose only other - words are post-nominals moves its member in SILENCE, agreeing - with the same name written without the clause. + the clause KEEPS it (S2); one the clause gives up is reported + where the reading that took it reports, so no word is reported + twice and none goes unreported. + That holds because a word the clause gives up reads as a + post-nominal or the clause keeps it. The stop is right only + where the released word ends the parse in the SUFFIX, so a stop + that would put it in a name part is no stop and the clause keeps + the word. What the take LEAVES BEHIND decides that, not the + clause as written. Two shapes leave nothing that could read the + word as a credential. A part whose other words are all + post-nominals or titles has no name word left for a trailing + slot to be the end of, and a part of nothing but credentials is + read whole and asked nothing. And a join reached below the take + — a particle chain (P2), or a bound given-name join (P5) — can + absorb the released word into a name part before any trailing + rule sees it, which would carry a word of the BIRTH name into + the current one. In both the clause keeps the word, and reports + it as it reports every member it keeps. + Delimiters outrank every reading inside them. Where a recognized + marker stands inside a delimited clause, the whole span is the + maiden name whatever its last word is, and whether or not the + pair is a configured maiden delimiter (M3): the writer drew the + boundary, so no fork is called and nothing is reported. A word + the writer left OUTSIDE the span is outside the clause and reads + as it would anywhere else. A marker with nothing after it, or nothing before it, is just a word. A marker may be more than one word, and is then recognized only @@ -1299,6 +1314,12 @@ M2. Rationale: a maiden marker announces that what follows it is the "Jane Doe nee MA" → maiden="MA" · boundary "Jane Doe nee MA Smith" → maiden="MA Smith" · boundary "John née Jones Smith MA" → maiden="Jones Smith" + "Doe, Dr. nee Smith MA" → maiden="Smith MA" · boundary + "Berg, abdul nee Jones MA" → maiden="Jones MA" · boundary + "Jane Doe nee Smith DO DO" → maiden="Smith DO DO" · boundary + "Jane Doe (nee Smith MA)" → maiden="Smith MA" + "Jane Doe (nee Smith Ma)" → maiden="Smith Ma" + "Jane Doe (nee Smith) MA" → suffix="MA" "Jones née" → family="née" · boundary "née Jones" → family="Jones" · boundary "Jane van der Berg née Jones" → maiden="Jones" diff --git a/docs/release_log.rst b/docs/release_log.rst index d727d4c2..12e9f2ba 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -20,7 +20,7 @@ Release Log - **Fix a credential ending the given part of a family-comma listing being read as a middle name in silence.** ``HumanName("Doe, John MA")`` gives first ``John``, last ``Doe``, suffix ``MA``, where 2.0 through 2.3 gave middle ``MA`` -- and 1.4.0 gave the suffix, so this restores v1's reading for that half. The comma has already named the family and the first word after it is the given name, so the words-to-spare count that governs the comma-less form is satisfied by construction and the writing decides alone: ``Doe, John Ma`` keeps middle ``Ma``, written the way a name is written, and ``Doe, John Ed`` keeps middle ``Ed``. Either reading is now reported, and the report belongs to the SPELLING rather than to the fields -- a declined name re-rendered without its comma, ``John Ma Doe``, re-parses to those same three fields and reports nothing, the word no longer standing where the question is asked. A name word behind the credential still ends its reach and stays silent -- ``Doe, John MA Smith`` gives middle ``MA Smith`` and reports nothing -- while a credential run or a trailing title is transparent to it: ``Doe, John MA PhD`` gives suffix ``MA PhD`` and ``Doe, John MA Prof.`` gives title ``Prof.`` with suffix ``MA``. Two second-order movements an upgrader may see, both consequences of the word leaving the given part rather than of this rule reaching further: ``Doe, John Prof. MA`` now gives title ``Prof.`` where it gave middle ``Prof. MA``, the trailing-title chain reaching a word the credential used to hide; and ``Doe, John van MA`` gives last ``van Doe`` with suffix ``MA`` where it gave middle ``van MA``, the surname-particle rule reaching a particle the same way. A name written wholly in one case says nothing either way and takes the credential, which is what 1.4.0 read: ``DOE, MARY JO MA``, ``doe, john ma``, ``田中, 太郎 MA`` and ``김, 민준 MA`` all give a suffix. The unlisted dotted spelling moves with them without the parity claim -- ``Doe, John X.Y.Z.`` gives suffix ``X.Y.Z.`` where 1.4.0 and 2.3.0 both gave a middle name -- to match the comma-less ``John Doe X.Y.Z.``. One word is carved out: ``do`` is the only member of this class that is also a surname particle, so capitals decide it and the particle reading keeps every other spelling. ``Doe, John DO`` gives suffix ``DO``, while ``Doe, John do``, ``Doe, John Do``, ``DOE, JOHN DO`` and ``doe, john do`` are unchanged and keep the particle-or-given report they already had. In a name written wholly in one case the two cannot be told apart, so ``SMITH, JOHN DO`` keeps last ``DO SMITH`` as ``NASCIMENTO, EDSON ARANTES DO`` does -- right about the Portuguese record, wrong about the osteopath, and the report is how a caller finds the second. See the ``S2`` and ``P6`` entries of ``docs/design/decisions.md`` (closes #531) - - **Fix a maiden marker's clause swallowing a trailing credential in silence.** ``HumanName("Jane Doe nee Smith MA")`` gives maiden ``Smith`` with suffix ``MA``, where 2.0 through 2.3 gave maiden ``Smith MA`` and said nothing; 1.4.0 read the ``MA`` as a suffix too. ``Doe, Jane nee Smith MA`` moves with it, and so do the one-case spellings ``JANE DOE NEE SMITH MA`` and ``jane doe nee smith ma``. The words a marker takes now end where a trailing credential begins, which is what the marker's other two stops -- a suffix word, a trailing roman numeral -- have always done. Until this release it was the last trailing position in the library where a word of the ambiguous credential class was read without a report, and it was order-sensitive besides: ``Jane Doe nee Smith MA PhD`` gave maiden ``Smith MA`` while ``Jane Doe nee Smith PhD MA`` gave maiden ``Smith``, so whether the word was read at all depended on which side of the unambiguous credential the writer put it. Both now give maiden ``Smith``, with suffix ``MA PhD`` and ``PhD MA``. The writing still decides, exactly as it does for the same word ending a name with no clause: ``Jane Doe nee Smith Ma`` keeps maiden ``Smith Ma``, and ``Jane Doe nee Yo-Yo Ma`` keeps a two-word birth surname whole. The one member of this class that is also a surname particle keeps the carve-out it has outside a clause -- ``Doe, Jane nee Smith DO`` gives suffix ``DO`` while ``Doe, Jane nee Smith do`` and ``Doe, Jane nee Smith Do`` keep maiden ``Smith do`` and ``Smith Do``, and the comma-less ``Jane Doe nee Smith do`` gives suffix ``do`` as ``John Doe do`` does. Either reading is now reported, except where the reading moves the word into a part that reports nothing -- see the silent movements below. A name word behind the credential ends its reach and stays silent -- ``Jane Doe nee MA Smith`` gives maiden ``MA Smith`` and reports nothing -- and this stop never takes the first word after the marker, whatever its writing says: ``Jane Doe nee MA`` keeps maiden ``MA`` and reports, the marker having announced a name where there would otherwise be none, and ``Jane Doe nee MA PhD`` keeps it too. That differs on purpose from what a certain post-nominal gets there, ``Jane Smith nee PhD`` and ``Jane Smith nee V`` leaving the marker standing as an ordinary word as before. Where no trailing rule reads the clause's tail nothing is decided and the clause keeps every word: ``Smith nee Jones MA, Jane`` and ``Smith, John, Jr nee Jones MA`` both keep maiden ``Jones MA``, unchanged and with no ``suffix-or-name`` report. A trailing title is not transparent here and the two spellings disagree -- ``Jane Doe nee Smith MA Prof.`` is unchanged and silent while ``Jane Doe nee Smith Prof. MA`` gives maiden ``Smith Prof.`` with suffix ``MA`` -- which is recorded as a boundary rather than fixed. Two second-order movements an upgrader may see: ``Doe, Dr. nee Smith MA`` gives title ``Dr.`` with suffix ``MA`` and moves in SILENCE, the take leaving a segment with no name word for the credential-run gate to read whole -- a post-nominal does that as readily as a title, so ``Jane Doe, Jr nee Smith MA`` gives maiden ``Smith`` with suffix ``Jr MA`` and is silent too, agreeing with the clause-less ``Jane Doe, Jr MA``, while the kept direction ``Jane Doe, Jr nee Smith Ma`` still reports; and ``Berg, abdul nee Jones MA`` gives first ``abdul MA``, the bound-given join taking the released word exactly as it does in ``Berg, abdul MA``. One name is a restoration rather than a change: ``John Smith nee Jones R.A.I.`` gives suffix ``R.A.I.`` again, as 2.3.0 read it, this unreleased cycle having moved it into the maiden name when the unlisted-dotted reading above took the word out of the certain-suffix class. See the ``M2`` and ``S2`` entries of ``docs/design/decisions.md`` (closes #533) + - **Fix a maiden marker's clause swallowing a trailing credential in silence.** ``HumanName("Jane Doe nee Smith MA")`` gives maiden ``Smith`` with suffix ``MA``, where 2.0 through 2.3 gave maiden ``Smith MA`` and said nothing; 1.4.0 read the ``MA`` as a suffix too. ``Doe, Jane nee Smith MA`` moves with it, and so do the one-case spellings ``JANE DOE NEE SMITH MA`` and ``jane doe nee smith ma``. The words a marker takes now end where a trailing credential begins, which is what the marker's other two stops -- a suffix word, a trailing roman numeral -- have always done. Until this release it was the last trailing position in the library where a word of the ambiguous credential class was read without a report, and it was order-sensitive besides: ``Jane Doe nee Smith MA PhD`` gave maiden ``Smith MA`` while ``Jane Doe nee Smith PhD MA`` gave maiden ``Smith``, so whether the word was read at all depended on which side of the unambiguous credential the writer put it. Both now give maiden ``Smith``, with suffix ``MA PhD`` and ``PhD MA``. The writing still decides, exactly as it does for the same word ending a name with no clause: ``Jane Doe nee Smith Ma`` keeps maiden ``Smith Ma``, and ``Jane Doe nee Yo-Yo Ma`` keeps a two-word birth surname whole. The one member of this class that is also a surname particle keeps the carve-out it has outside a clause -- ``Doe, Jane nee Smith DO`` gives suffix ``DO`` while ``Doe, Jane nee Smith do`` and ``Doe, Jane nee Smith Do`` keep maiden ``Smith do`` and ``Smith Do``, and the comma-less ``Jane Doe nee Smith do`` gives suffix ``do`` as ``John Doe do`` does. Either reading is now reported, and there is no third: a word the clause gives up reads as a post-nominal, or the clause keeps it and says so. A name word behind the credential ends its reach and stays silent -- ``Jane Doe nee MA Smith`` gives maiden ``MA Smith`` and reports nothing -- and this stop never takes the first word after the marker, whatever its writing says: ``Jane Doe nee MA`` keeps maiden ``MA`` and reports, the marker having announced a name where there would otherwise be none, and ``Jane Doe nee MA PhD`` keeps it too. That differs on purpose from what a certain post-nominal gets there, ``Jane Smith nee PhD`` and ``Jane Smith nee V`` leaving the marker standing as an ordinary word as before. Where no trailing rule reads the clause's tail nothing is decided and the clause keeps every word: ``Smith nee Jones MA, Jane`` and ``Smith, John, Jr nee Jones MA`` both keep maiden ``Jones MA``, unchanged and with no ``suffix-or-name`` report. A trailing title is not transparent here and the two spellings disagree -- ``Jane Doe nee Smith MA Prof.`` is unchanged and silent while ``Jane Doe nee Smith Prof. MA`` gives maiden ``Smith Prof.`` with suffix ``MA`` -- which is recorded as a boundary rather than fixed. The clause also keeps a word it cannot promise a credential reading for, which is where three shapes that look like they should move do not. Where the part the word would land in holds no name of its own there is nothing to read it as a credential, so ``Doe, Dr. nee Smith MA`` and ``Jane Doe, Jr nee Smith MA`` both keep maiden ``Smith MA`` and report. Where a join would swallow it first the same applies, and it is the birth name that would lose the word: ``Berg, abdul nee Jones MA`` keeps maiden ``Jones MA`` rather than reading first ``abdul MA``, and ``Berg, Jane van der nee Smith DO`` keeps maiden ``Smith DO`` rather than letting the particle chain carry the ``DO`` into last ``van der DO Berg``. Each of those reads as 2.3.0 read it. Delimiters settle the question outright and always did: ``HumanName("Jane Doe (nee Smith MA)")`` keeps the whole span as the maiden name and reports nothing, the writer having drawn the boundary, while ``Jane Doe (nee Smith) MA`` gives suffix ``MA`` for the word left outside it. One name is a restoration rather than a change: ``John Smith nee Jones R.A.I.`` gives suffix ``R.A.I.`` again, as 2.3.0 read it, this unreleased cycle having moved it into the maiden name when the unlisted-dotted reading above took the word out of the certain-suffix class. See the ``M2`` and ``S2`` entries of ``docs/design/decisions.md`` (closes #533) **Additions** diff --git a/nameparser/_pipeline/_group.py b/nameparser/_pipeline/_group.py index d62b2bc0..70e09988 100644 --- a/nameparser/_pipeline/_group.py +++ b/nameparser/_pipeline/_group.py @@ -41,6 +41,7 @@ import dataclasses from collections.abc import Iterable, Sequence, Set from enum import IntEnum +from typing import assert_never from nameparser._lexicon import _run_addresses_by_given from nameparser._pipeline._pieces import ( @@ -64,10 +65,17 @@ # wholly-suffix post-comma run, both of which see the merged reading Piece = list[int] -#: What the marker pass took out of a segment: the marker's tokens (to -#: be dropped -- more than one where the entry is a phrase, 'z domu') -#: and the maiden-name pieces (to take Role.MAIDEN). +#: What the marker pass took out of a segment: the marker's TOKEN +#: indices (more than one where the entry is a phrase, 'z domu') and +#: the maiden-name pieces, themselves token indices, to take +#: Role.MAIDEN. `_group_segment` produces it. MaidenTake = tuple[Piece, list[Piece]] +#: What `_maiden_take` answers with, one index space further out: PIECE +#: indices into its own `pieces` argument -- the marker's pieces, then +#: the maiden name's. `_group_segment` resolves them to the token +#: indices MaidenTake declares, which is the only reason the two are +#: different types rather than one name used twice. +MaidenIndices = tuple[list[int], list[int]] class TailReader(IntEnum): @@ -80,7 +88,17 @@ class TailReader(IntEnum): words are the family the comma already named, and a tail segment is read as credentials whole, so no trailing rule is consulted there and the clause keeps what it has -- a stop would hand a word - to `family` rather than to `suffix`.""" + to `family` rather than to `suffix`. + + A CLOSED set: `_maiden_take` dispatches on it exhaustively + (`typing.assert_never`), so a fourth member is a type error at + every reader until it is given a reading. group() is the one + place (structure, segment index) is mapped onto it, and + tests/v2/pipeline/test_group.py's + `test_the_reader_is_pinned_to_the_structure_it_is_read_from` + pins that mapping -- by watching the call, not by restating it -- + along with this enum's size. Change either and that test says so. + """ NONE = 0 # FAMILY_COMMA segment 0, and every tail segment TRAILING = 1 # the S2 peel: NO_COMMA, SUFFIX_COMMA segment 0 @@ -119,11 +137,10 @@ def _is_prefix_piece(piece: Sequence[int], ptags: Set[str], return len(piece) == 1 and "particle" in tokens[piece[0]].tags -# rules.md#M2: "a recognized maiden marker standing after at least -# one name word takes the words after it" -- up to any suffix word, -# or the trailing numeral assign reads as the suffix, as the maiden -# name, the marker itself dropped -# (history: decisions.md#M2) +# rules.md#M2: "a recognized maiden marker standing after at least one +# name word takes the words after it" -- up to any suffix word, or the +# trailing numeral assign reads as the suffix, as the maiden name, the +# marker itself dropped (history: decisions.md#M2) # # A marker piece is a LONE marker -- M2's own "standing as a word of # its own". The consumer runs before every join but the Ph. D. merge @@ -205,6 +222,69 @@ def _marker_run_pieces(seen: Sequence[int], pieces: Sequence[Sequence[int]], tokens[pieces[seen[k]][0]].tags for k in range(m + 1, len(seen))) +# rules.md#M2: "a word the clause gives up reads as a post-nominal or +# the clause keeps it" -- the two halves of that, asked of the VIEW +# the take would leave. Split out of `_maiden_take` because each +# models a LATER decision about the released word, and the stop is +# only right where that decision goes the way the release assumed +# (#533 review). +def _a_name_word_ahead(view: Sequence[Sequence[int]], + view_tags: Sequence[Set[str]], + tokens: Sequence[WorkToken], + at: int) -> bool: + """Whether the view holds a name word BEFORE the member at `at`. + + #531's reading is the given part's own trailing slot, and a slot + needs a part: the take removes the clause, so where every piece + ahead of the member is a title or a suffix the member is the only + name word left and there is no trailing slot for it to end. + Asked of the view rather than of the segment because the segment + still has the maiden name in it -- that is exactly the word the + release takes away ('Doe, Prof. nee Smith A.B.' left 'Prof. A.B.', + whose A.B. is the GIVEN name and no credential at all).""" + return any(not is_title_piece(view[q], view_tags[q], tokens) + and not is_suffix_piece(view[q], view_tags[q], tokens) + for q in range(at)) + + +def _join_takes_the_member(view: Sequence[Sequence[int]], + view_tags: Sequence[Set[str]], + tokens: Sequence[WorkToken], + at: int) -> bool: + """Whether a join BELOW this pass would absorb the member at `at`. + + A released word only reads as the credential while it is still the + lone piece assign's peel looks at; inside a joined piece it is a + word of the name the join built, and the release has moved it from + the birth name into the CURRENT one -- #424's class of failure, + now from the other side ('Berg, Jane van der nee Smith DO' read + family 'van der DO Berg'). + + Two joins can reach it, and each is modelled by the shape it needs + rather than by running it: P2's chain, where the member is + particle vocabulary standing behind a particle piece or beside + another released one ('Jane Doe nee Smith DO DO' read family + 'DO DO'), and P5's bound-given join, where the member is the word + after the bound one ('Berg, abdul nee Jones MA' read given + 'abdul MA'). Both over-decline rather than predict: a shape that + only MIGHT join keeps its word in the maiden name, which is the + conservative direction M2's invariant asks for.""" + member = tokens[view[at][0]] + if "particle" in member.tags: + if at and _is_prefix_piece(view[at - 1], view_tags[at - 1], tokens): + return True + for q in range(at + 1, len(view)): + if len(view[q]) == 1 and "particle" in tokens[view[q][0]].tags: + return True + # P5 joins the first non-title piece to the one after it, so the + # member is at risk exactly where it IS the one after it. The tag + # pair is tested before `leading_titles` is asked, which is what + # keeps the ordinary credential release from paying that frame. + return (at > 0 and len(view[at - 1]) == 1 + and "vocab:bound-given" in tokens[view[at - 1][0]].tags + and leading_titles(view, view_tags, tokens) == at - 1) + + def _maiden_take(pieces: Sequence[Sequence[int]], ptags: Sequence[Set[str]], tokens: Sequence[WorkToken], @@ -212,11 +292,14 @@ def _maiden_take(pieces: Sequence[Sequence[int]], one_case: bool | None, reader: TailReader, ambiguities: list[PendingAmbiguity], - ) -> tuple[list[int], list[int]] | None: - """The piece indices the marker pass removes, split the way - MaidenTake declares them: the MARKER's pieces (one, or several for - a phrase entry like 'z domu') and the maiden name's. None when the - pass declines. + ) -> MaidenIndices | None: + """The PIECE indices the marker pass removes, split the way + MaidenIndices declares them: the MARKER's pieces (one, or several + for a phrase entry like 'z domu') and the maiden name's. None when + the pass declines. Piece indices into `pieces`, not the TOKEN + indices of the MaidenTake `_group_segment` builds out of them -- + the two are different index spaces and both are named types so a + reader never has to guess which one an annotation means. Split here rather than by the caller because `run` is known here and nowhere else -- it comes off the continuation tags, which are @@ -225,19 +308,23 @@ def _maiden_take(pieces: Sequence[Sequence[int]], Computed before any join (the Ph. D. merge aside), so "up to any trailing suffix" means the first suffix WORD after the marker: a connective beside that suffix cannot un-suffix it first. 'Jane - Smith née Jr y Jones' declines where the joined reading took - 'Jr y Jones' as the maiden name, and 'Jane Smith née Jones Jr y - Smith' takes only 'Jones'. The one reading the order costs; M2's - Accepted row and decisions.md#M2 (#420) record it. + Smith née Jr y Jones' declines where the joined reading took 'Jr y + Jones' as the maiden name, and 'Jane Smith née Jones Jr y Smith' + takes only 'Jones'. The one reading the order costs; M2's Accepted + row and decisions.md#M2 (#420) record it. - "Up to any trailing suffix" also means up to a trailing - CREDENTIAL since #533, where the rule that reads the name left - standing reads the word as one -- the TRAILING peel with no comma, - the given part's own slot after a family comma, and nobody before - that comma or past a second one, which is what `reader` says. A - member standing alone after the marker is never given up: the - marker announces a name, and the rule gives a word up only where - a maiden name is left standing. + "Up to any trailing suffix" also means up to a trailing CREDENTIAL + since #533, where the rule that reads the name left standing reads + the word as one -- the TRAILING peel with no comma, the given + part's own slot after a family comma, and nobody before that comma + or past a second one, which is what `reader` says. A member + standing alone after the marker is never given up: the marker + announces a name, and the rule gives a word up only where a maiden + name is left standing. Nor is one given up to a reader that will + not be there to read it, or to a JOIN that runs before the reader + does: rules.md#M2's invariant is that a released word ends the + parse suffix-roled, and the two view checks below are what makes + the stop conservative enough to hold it (#533 review). A tail segment's delimiter cores (`cores`, empty elsewhere) are structure, not words, and group() drops them after the pass -- @@ -289,26 +376,25 @@ def _maiden_take(pieces: Sequence[Sequence[int]], # this comment counted and why it read 2,429 names), empty # strings dropped, under six policies (the default, both # family-first orders, strict commas, and each 2.4 switch - # flipped). THE PAIR IS THE FINDING: - # over the corpus as it stood the day before this change it moved - # 0 parses, and over the corpus WITH this change's own rows it - # moves 36 of 10,752 (1,792 names), on 6 distinct names ('Doe, - # Jane nee Smith DO', 'Doe, Jane nee Smith Ma', 'Jane Doe nee - # Smith Ma', 'Jane Doe nee Smith Ma JD', 'Jane Doe nee Yo-Yo Ma', - # 'John née Jones Smith MA'). The plumbing was live either way; - # the corpus simply - # held no name that could show it, which is the blindness - # mechanisms.md's corpus field note asks to be measured before any - # "N names move" is written down. The 2026-09-18 record of 0 of - # 9,852 under the numeral-only reading stands as what was true - # then and is superseded here. + # flipped). THE PAIR IS THE FINDING: over the corpus as it stood + # the day before this change it moved 0 parses, and over the + # corpus WITH this change's own rows it moves 36 of 10,752 (1,792 + # names), on 6 distinct names ('Doe, Jane nee Smith DO', 'Doe, + # Jane nee Smith Ma', 'Jane Doe nee Smith Ma', 'Jane Doe nee Smith + # Ma JD', 'Jane Doe nee Yo-Yo Ma', 'John née Jones Smith MA'). The + # plumbing was live either way; the corpus simply held no name + # that could show it, which is the blindness mechanisms.md's + # corpus field note asks to be measured before any "N names move" + # is written down. The 2026-09-18 record of 0 of 9,852 under the + # numeral-only reading stands as what was true then and is + # superseded here. # # The chain-tail measure below (`tail`, and the re-peel after the # chain) is the opposite, and the 2026-09-18 sweep says so: # dropping it moves 18 of that 9,852, on 'John van der Berg Ma', - # 'John de - # Ma' and 'Freiherr von Berg MA' under every one of the six. A - # review round called all three sites inert together; two are. + # 'John de Ma' and 'Freiherr von Berg MA' under every one of the + # six. A review round called all three sites inert together; two + # are. skip = frozenset(range(len(pieces))) - frozenset(seen) rest = peel_walk(seen[m], ptags, skip) peeled = peel_trailing(rest, pieces, ptags, tokens, one_case) @@ -336,61 +422,65 @@ def _maiden_take(pieces: Sequence[Sequence[int]], # #533: the ACRONYM fork, asked the way the numeral is -- the peel # over the pieces as they stand, then again over the name the take # would leave, and a stop only where both read the word as the - # credential. `peeled.names` is the first piece the peel took, so - # the class member it stopped at is that piece and no walk of our - # own is needed. One peel, one view built once per take: O(pieces) - # for the take, not per member, and no re-entrancy -- the - # predicate never calls the walk that calls it. + # credential. `peeled.names` is a COUNT of positions in `rest`, so + # `rest[peeled.names]` is the first piece the peel took and the + # class member it stopped at; no walk of our own is needed. One + # peel, one view built once per take: O(pieces) for the take, not + # per member, and no re-entrancy -- the predicate never calls the + # walk that calls it. if (reader is not TailReader.NONE and peeled.names < len(rest) and m + run + 1 < len(seen)): - # OPTION 1: the stop never takes the FIRST word after the - # marker -- a class member standing alone there stays the - # maiden name. A clamp rather than a veto: where the peel - # consumed that word AND words behind it, only the first stays - # ('Doe, J. nee MA ba' keeps maiden 'MA' and reads suffix - # 'ba'; a veto handed 'ba' back to the clause too). The + # THE FIRST-WORD FLOOR: the stop never takes the FIRST word + # after the marker -- a class member standing alone there + # stays the maiden name. A clamp rather than a veto: where the + # peel consumed that word AND words behind it, only the first + # stays ('Doe, J. nee MA ba' keeps maiden 'MA' and reads + # suffix 'ba'; a veto handed 'ba' back to the clause too). The # clamped piece may then be no member at all, and the test # below declines -- which changes nothing, the walk stopping # at that suffix word of its own accord. stop = max(rest[peeled.names], seen[m + run + 1]) head = pieces[stop] - # Both halves of the membership test below are DEFENSIVE, and - # measured inert on 2026-09-19 -- over 297,381 parses (1,533 - # corpus and case names plus 4,536 generated clause shapes, - # seven policies, seven lexicons) admitting either half moved - # no parse and, at the default vocabulary, no frame. What each - # one guards, and why it stays: - # - # `len(head) == 1` asks a LONE piece's question, and the + # `len(head) == 1` is DEFENSIVE, and measured inert on + # 2026-09-19 -- NOT unreachable, which is what an earlier + # round of this comment claimed on a population that could not + # contain the shape. It asks a LONE piece's question, and the # answer below reads `head[0]` as if the piece were the word. - # The only multi-token piece the marker pass can see is the - # Ph. D. merge above, which carries the `suffix` ptag, so the - # walk stops there of its own accord and the stop would change - # nothing -- structural today, and the default vocabulary does - # not even put the tag on that piece's head: a caller listing - # `ph` ambiguous is what could. - # THE NEGATIVE CONTROL, so the "inert" above is checkable - # rather than asserted. A probe that fires wherever the tag - # test admits a head this length test then DECLINES -- the - # only sites where dropping it could matter -- recorded 0 over - # 21,504 parses: decisions.md#S2's population (1,792 names; - # the recipe is spelled out above) under six policies and two - # lexicons, the default and one listing `ph` ambiguous. That 0 - # is a measured absence rather than an unreached probe: over - # the same sweep this site is reached 394 times and the TAG - # test admits the head in 310 of them, so the probe can fire - # and does not (re-measured 2026-09-19). Nor is - # there a - # price: with `ph` listed, dropping the test leaves 'Jane Doe - # nee Smith Ph. D.' at 372 frames, 'Doe, Jane nee Smith - # Ph. D.' at 399 and 'J. nee Jones Smith Ph. D.' at 336, - # unchanged to the frame. It is the walk, not this test, that - # keeps the merged piece from ever being the stop. Kept - # for the reason `_assign.previous_kept` is: an inert branch - # is cheaper than a question asked of the wrong shape, and the - # four sibling sites (`_pieces.tail_reading`, - # `_pieces.peel_trailing`, the GIVEN_SLOT branch below, the - # emitter at the end of this function) all pair the two. + # + # The multi-token piece it keeps out is the Ph. D. merge + # above. That piece carries the `suffix` ptag, so `peel_walk` + # never returns it and the peel half of `stop` cannot be it -- + # but the FIRST-WORD FLOOR is the other half, and the floor is + # an index rather than a walk, so it reaches the merged piece + # whenever that piece is the second word after the marker. + # 'BERG, ABDUL Z DOMU MA PH. D.' is that shape. + # + # THE NEGATIVE CONTROL, re-measured 2026-09-19 over a + # population built to HOLD the shape -- 4,224 names (twelve + # heads x four markers x 23 bodies, each also with a ', MD' + # tail and in upper and lower case) under six policies and two + # lexicons, the default and one listing `ph` ambiguous, 50,688 + # parses. A probe that fires wherever the tag test admits a + # head this length test then DECLINES -- the only sites where + # dropping it could matter -- fires 1,440 times, against + # 25,920 reaches of this site and 19,296 tag admissions. The + # earlier "0 over 21,504" was the population, not the branch; + # mechanisms.md's corpus field note is about exactly this. + # + # Inert it still is, and now that is a measurement rather than + # a structure: dropping it is byte-identical -- fields, + # ambiguities and every token's role and tags -- over 905,796 + # parses (that population plus the review's 142,518-name + # corpus under the six policies). What it buys is the price, + # and the price is real where the count is not: with `ph` + # listed, dropping it takes 'BERG, ABDUL Z DOMU MA PH. D.' + # from 438 frames to 453. Kept for the reason + # `_assign.previous_kept` is: an inert branch is cheaper than + # a question asked of the wrong shape, and the three sibling + # sites (`_pieces.segment_suffix_reading` -- the `_pieces.py` + # block that pairs the same two conditions -- the GIVEN_SLOT + # branch below, and the emitter at the end of this function) + # each pair a length test with a tag test the same way. # # The tag is the CLASS the rule is stated in terms of, and it # is not redundant with the walk the way the length test is: @@ -407,7 +497,17 @@ def _maiden_take(pieces: Sequence[Sequence[int]], # numbers: `_CALL_BASELINE` is per-interpreter and per entry # point, and a row for one name would have to be guessed for # the four interpreters only CI runs. - if (stop < trailing and len(head) == 1 + # + # A THIRD condition stood here and is gone: `stop < trailing`. + # It guarded nothing, structurally -- `stop` is the larger of + # a walked piece and the floor, and both are bounded by + # `trailing`, so at worst `stop == trailing` and the + # assignment below would set `trailing` to what it already is. + # Measured over the same 1,760,904 parses: `stop > trailing` + # never once, `stop == trailing` 15,696 times, and the tag + # test declined every one of those, so removing it moved no + # parse and no frame. + if (len(head) == 1 and AMBIGUOUS_ACRONYM_TAG in tokens[head[0]].tags): left = [i for i in seen if i < seen[m] or i >= stop] view = [pieces[i] for i in left] @@ -418,32 +518,47 @@ def _maiden_take(pieces: Sequence[Sequence[int]], # whether it takes something -- a suffix word behind the # member answers yes to the weaker question while the # member itself reads as the family name ('JOHN NEE JONES - # SMITH MA PHD' left 'JOHN MA PHD', whose MA is the - # family). - at = len(view) - (len(left) - left.index(stop)) + # SMITH MA PHD' left 'JOHN MA PHD', whose MA is the family). + at = left.index(stop) if reader is TailReader.GIVEN_SLOT: # after a family comma the words to spare are there by # construction, so the reader is #531's -- the member's # own reading, asked through the one predicate that # owns it, and that rule's FLOOR: the member ends the # given part only where every piece behind it reads as - # a suffix too ('Doe, Jane MA do' reads middle 'MA', - # the particle not being a credential, so the clause - # keeps both words rather than handing one of them to - # the current name's middle). - takes = all( + # a suffix too ('Doe, Jane nee Smith MA do' keeps + # maiden 'Smith MA do', the trailing particle not + # being a credential, so the clause keeps both words + # rather than handing one of them to the current + # name's middle -- which is what the clause-less + # 'Doe, Jane MA do' does with them, middle 'MA' and + # family 'do Doe'). And a slot the take would + # leave nobody to read is no slot: `_a_name_word_ahead` + # is that half, asked first because it is the cheaper + # question and because with no name word ahead the + # answer below is about a name that would not exist. + takes = _a_name_word_ahead(view, view_tags, tokens, at) and all( is_suffix_piece(view[q], view_tags[q], tokens) or (len(view[q]) == 1 and AMBIGUOUS_ACRONYM_TAG in tokens[view[q][0]].tags and credential_at_the_given_slot( tokens[view[q][0]], one_case)) for q in range(at, len(view))) - else: + elif reader is TailReader.TRAILING: takes = trailing_start( leading_titles(view, view_tags, tokens), view, view_tags, tokens, one_case=one_case) <= at - if takes: + else: + assert_never(reader) + # rules.md#M2: "a word the clause gives up reads as a + # post-nominal or the clause keeps it". Both readers above + # ask what a TRAILING rule makes of the member, and a join + # below this pass runs first and can take the word out of + # that rule's reach entirely, so the release is withdrawn + # where one would (#533 review). + if takes and not _join_takes_the_member( + view, view_tags, tokens, at): trailing = stop j = m + run while (j < len(seen) and seen[j] < trailing @@ -474,8 +589,21 @@ def _maiden_take(pieces: Sequence[Sequence[int]], # by-shape member reports with the dotted switch off -- classify # writes the shape tag there while the class does not admit it, # which is the one place a declined fork can be recorded. + # + # `last[0]` is the whole word and no `len(last) == 1` guards it, + # unlike the three sibling sites: here the invariant IS structural + # and a test would be inert by construction rather than by + # measurement. The only multi-token piece this pass can see is the + # Ph. D. merge, `merge` gives that piece the `suffix` ptag + # unconditionally, and `is_suffix_piece` answers yes off that ptag + # alone -- so the walk just above ENDS before it, and the last + # maiden piece is never it whatever a caller's vocabulary says. + # Measured too, both ways: a probe on `len(last) != 1` here fired + # 0 times over 1,760,904 parses, and deleting the test is + # byte-identical (fields, ambiguities, token roles and tags) over + # the 905,796-parse oracle at the same frame counts. last = pieces[seen[j - 1]] - if (reader is not TailReader.NONE and len(last) == 1 + if (reader is not TailReader.NONE and not tokens[last[0]].tags.isdisjoint( _AMBIGUOUS_CREDENTIAL_TAGS)): ambiguities.append(PendingAmbiguity( @@ -517,8 +645,8 @@ def _group_segment(seg: tuple[int, ...], additional: int, opens_the_name: bool = False, *, one_case: bool | None, - reader: TailReader = TailReader.TRAILING, - maiden_ambiguities: list[PendingAmbiguity] | None = None, + reader: TailReader, + maiden_ambiguities: list[PendingAmbiguity], ) -> tuple[list[Piece], list[set[str]], MaidenTake | None]: pieces: list[Piece] = [[i] for i in seg] ptags: list[set[str]] = [set() for _ in seg] @@ -527,15 +655,21 @@ def _group_segment(seg: tuple[int, ...], additional: int, # list) suppresses reporting -- see group() for when that applies. if ambiguities is None: ambiguities = [] - # The maiden walk's own channel. group() passes `None` for the - # chain emitter after a family comma -- the comma fixed the - # family, so that fork is settled -- and #533's is not that fork: - # a credential ending a maiden clause is a question the comma - # settles nothing about. A second parameter rather than a widening - # of the first, so neither channel can quietly acquire the other's - # suppression. - if maiden_ambiguities is None: - maiden_ambiguities = ambiguities + # The maiden walk's own channel, and REQUIRED beside `reader` for + # the same reason: the one production caller answers both off the + # segment's structure, and a default would be this module guessing + # what that caller already knows. group() passes `None` on the + # first for the chain emitter after a family comma -- the comma + # fixed the family, so that fork is settled -- and #533's is not + # that fork: a credential ending a maiden clause is a question the + # comma settles nothing about, which is why the two channels are + # two parameters. They are given the SAME list wherever nothing is + # suppressed, which is every segment that is NOT after a family + # comma; what the split buys is the other case, where `None` on + # the first must not reach the second. An earlier spelling + # defaulted the maiden channel to whatever the first was, so a + # caller passing `ambiguities=None` silenced both -- which the + # required argument now makes unsayable (the review's finding). def title(k: int) -> bool: return is_title_piece(pieces[k], ptags[k], tokens) diff --git a/nameparser/_pipeline/_pieces.py b/nameparser/_pipeline/_pieces.py index e6afac6a..d4240e3b 100644 --- a/nameparser/_pipeline/_pieces.py +++ b/nameparser/_pipeline/_pieces.py @@ -486,7 +486,22 @@ def credential_at_the_given_slot(token: WorkToken, (mechanisms.md#ONE-PREDICATE-PER-QUESTION). It is a text-and-tags question, which is what puts it in this module rather than beside either caller. + + The membership half of that contract is CHECKED rather than + trusted, because getting it wrong is silent: all three of + `listed_lean`'s None reasons fall through to the `particle` test + below, so a non-member handed in by mistake is answered True -- + "read it as the credential" -- for a word the class never admitted. + An assert rather than a raise or a branch: it enters no Python + frame (measured -- 'Doe, John MA' stays at 311), it states the + contract where a reader of the function body meets it, and under + -O it is exactly the code that was here before. """ + assert AMBIGUOUS_ACRONYM_TAG in token.tags, ( + f"credential_at_the_given_slot is #531's reading of a LISTED " + f"class member; {token.text!r} carries {sorted(token.tags)} " + f"and is not one. The caller decides membership -- test " + f"AMBIGUOUS_ACRONYM_TAG before calling") lean = listed_lean(token, one_case) return lean == "credential" or (lean is None and "particle" not in token.tags) diff --git a/nameparser/_types.py b/nameparser/_types.py index 08517bf4..567bf079 100644 --- a/nameparser/_types.py +++ b/nameparser/_types.py @@ -452,9 +452,8 @@ class AmbiguityKind(StrEnum): #: clause's peel before it can reach the member at all ("Jane Doe #: nee Smith MA Prof." keeps maiden ``Smith MA Prof.``, where #: "Jane Doe nee Smith Prof. MA" reads suffix ``MA`` and reports). - #: Third -- pre-existing, and 2.4 widening what reaches it rather - #: than moving it -- a member with no name word IN FRONT of it is - #: not at this + #: Third -- pre-existing, and untouched by 2.4 -- a member with no + #: name word IN FRONT of it is not at this #: slot either, because the slot is the end of a given part and #: there is none: a TITLE or a POST-NOMINAL took that position -- #: either one leaves the segment with no name word for the slot to @@ -467,25 +466,16 @@ class AmbiguityKind(StrEnum): #: immediately after the comma and nothing behind that piece: #: "Doe, MA Smith" reports its ``MA``, and "Doe, Dr. MA Smith" -- #: the same member, one title in front of it -- reads given ``MA`` - #: in silence. A maiden clause can put a name in that position: - #: "Doe, Dr. nee Smith MA" gives title ``Dr.``, suffix ``MA`` and - #: maiden ``Smith``, and moves in silence -- once ``Smith`` leaves - #: with the marker, segment 1 is ``Dr. MA``, a no-name segment the - #: credential-run gate reads whole. (The other direction still - #: reports, the clause's own emitter being what raises it: "Doe, - #: Dr. nee Smith Ma" keeps maiden ``Smith Ma`` and says so.) - #: What stands in front need not be a title, and a clause is the - #: commonest way to reach the POST-NOMINAL spelling: "Jane Doe, Jr - #: nee Smith MA" gives maiden ``Smith`` with suffix ``Jr MA`` and - #: moves in silence, the take leaving segment 1 as ``Jr MA`` -- - #: post-nominals only, so the same gate reads it whole. ``III`` - #: and ``PhD`` head it the same way, and each agrees with its - #: clause-less control ("Jane Doe, Jr MA" is silent too). The - #: clause's own emitter still covers the kept direction here: - #: "Jane Doe, Jr nee Smith Ma" and "Jane Doe, Jr nee MA" both - #: report. Found by the #533 design-docs review, which is the - #: measurement worth keeping: this silence was written as a - #: title's and a post-nominal reaches it too. + #: in silence. A maiden CLAUSE does NOT reach this position, and + #: that is a decision rather than an accident: a clause may give + #: a word up only where the word then reads as a post-nominal + #: (rules.md#M2), and here nothing would read it at all, so the + #: clause keeps it and reports instead. "Doe, Dr. nee Smith MA" + #: keeps maiden ``Smith MA`` and says so, as does the + #: post-nominal spelling "Jane Doe, Jr nee Smith MA" -- the take + #: would leave segment 1 as ``Jr MA``, post-nominals only, which + #: the gate reads whole. The same holds of the fourth position + #: below: a clause never hands a word to a join. #: And fourth -- pre-existing and untouched by 2.4 as well -- #: a JOIN beside the member can #: take it out of this slot, from either side. Where a chain has @@ -498,9 +488,9 @@ class AmbiguityKind(StrEnum): #: and reports. The particle chain is the commonest joiner but not #: the only one: the bound-given join takes a member into its pair #: the same way, so "Berg, abdul MA" reads given ``abdul MA`` in - #: silence -- and "Berg, abdul nee Jones MA" agrees with it, the - #: clause giving the member up and the join catching it before - #: this slot is reached. Where a particle the suffix vocabulary + #: silence -- while "Berg, abdul nee Jones MA" keeps maiden + #: ``Jones MA`` and reports, the clause declining to hand the + #: member to a join that would swallow it. Where a particle the suffix vocabulary #: does not also claim stands BEHIND it, the given part ends at #: that particle as this walk reads it, and the attachment that #: moves the particle to the family runs a stage too late to @@ -513,6 +503,17 @@ class AmbiguityKind(StrEnum): #: ``MA`` past a ``vd`` the suffix vocabulary claims outright. #: All four POSITIONS above are silent -- those last two names #: are the boundary each one stops at, not instances of it. + #: A DELIMITED maiden clause is quiet for a different reason and + #: is not one of the four. Where a recognized marker stands + #: inside a delimited span the whole span is the maiden name, + #: whatever its last word is and whether or not the pair is a + #: configured maiden delimiter: "Jane Doe (nee Smith MA)" and + #: "Jane Doe (nee Smith Ma)" both give maiden ``Smith MA`` / + #: ``Smith Ma`` and report nothing. The writer drew the boundary, + #: so no fork was available to decline -- a settled position + #: rather than a fork left un-asked. The boundary cuts both ways: + #: "Jane Doe (nee Smith) MA" gives suffix ``MA`` and reports, the + #: member standing outside the span. SUFFIX_OR_NAME = "suffix-or-name" #: An input the title peel eats down to one last word which is #: itself title vocabulary still has to name somebody, so that diff --git a/tests/v2/cases.py b/tests/v2/cases.py index 13934195..fa422ebc 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -3318,45 +3318,181 @@ def _check_cjk_shape_purity(self) -> None: "middle initial it always was -- the stop reaches the " "clause's trailing word, not the name in front of it", shape=2), - Case("a_no_name_segment_moves_in_silence", + Case("a_no_name_segment_leaves_the_clause_nobody_to_read_it", "Doe, Dr. nee Smith MA", - {"title": "Dr.", "family": "Doe", "suffix": "MA", - "maiden": "Smith"}, - classification="fix(#533)", - ambiguities=(), - notes="an ACCEPTED COST and a recorded silence. Once 'Smith' " + {"title": "Dr.", "family": "Doe", "maiden": "Smith MA"}, + classification="parity", + ambiguities=("suffix-or-name",), + notes="rules.md#M2's invariant, and the row that used to " + "record the opposite: an earlier round of #533 read " + "suffix 'MA' here and said so in silence. Once 'Smith' " "leaves with the marker, segment 1 is 'Dr. MA' -- a " "no-name segment, which the credential-run gate reads " - "whole without ever reaching #531's emitter, and the " - "first-post-comma emitter reads 'Dr.'. That is " - "_types.py's third pre-existing silence (a member with " - "no name word IN FRONT of it), now reachable through a " - "clause. 1.4.0 read given 'nee', middle 'Smith', " + "whole without ever reaching #531's emitter, so the " + "released word would have landed in `given`, not in " + "`suffix`. With no name word ahead of it the member is " + "no trailing word of a given part, the walk declines, " + "and the clause keeps it. The REPORT survives the " + "decline: the emitter asks whether a trailing rule " + "reads these words at all, which after a family comma " + "it does. 1.4.0 read given 'nee', middle 'Smith', " "suffix 'MA'", shape=2), - Case("the_bound_given_join_takes_the_released_member", + Case("the_bound_given_join_would_take_the_released_member", "Berg, abdul nee Jones MA", - {"given": "abdul MA", "family": "Berg", "maiden": "Jones"}, - classification="fix(#533)", - ambiguities=(), - notes="an ACCEPTED COST, and the control beside it says it " - "is not this change's defect: the take releases 'MA' " - "and P5's LENIENT post-comma join swallows it into the " - "bound-given pair before assign can read it -- exactly " - "as it does in 'Berg, abdul MA'. So the clause form " - "now AGREES with the clause-less form. Silent on both, " - "and _types.py already documents that silence (a chain " - "has swallowed the member into one piece, so there is " - "no lone member to ask about)", + {"given": "abdul", "family": "Berg", "maiden": "Jones MA"}, + classification="parity", + ambiguities=("suffix-or-name",), + notes="the other half of M2's invariant: P5's LENIENT " + "post-comma join runs BELOW the marker pass and would " + "swallow the released 'MA' into the bound-given pair " + "before assign could read it -- 'abdul MA' as the " + "given name, which is where the clause-less control " + "below genuinely puts it. A word joined away is a word " + "the clause gave up for nothing, so the walk declines " + "and keeps it. An earlier round of #533 released it " + "and read given 'abdul MA' in silence", shape=2), Case("the_bound_given_joins_control_without_the_clause", "Berg, abdul MA", {"given": "abdul MA", "family": "Berg"}, classification="parity", ambiguities=(), - notes="the recorded control for the row above, and 1.4.0 " - "read it identically (first 'abdul MA', last 'Berg'). " - "Unchanged by this rule", + notes="the recorded control for the row above -- what the " + "released member WOULD have read as, and why releasing " + "it buys nothing. 1.4.0 read it identically (first " + "'abdul MA', last 'Berg'). Unchanged by this rule", + shape=2), + Case("a_particle_chain_would_take_the_released_member", + "Berg, Jane van der nee Smith DO", + {"given": "Jane", "family": "van der Berg", + "maiden": "Smith DO"}, + classification="parity", + ambiguities=("particle-or-given", "suffix-or-name"), + notes="M2's invariant against P2 rather than P5. 'DO' is " + "particle vocabulary standing behind a particle piece, " + "so the chain below this pass would absorb it into the " + "family -- a word crossing from the BIRTH name into " + "the current one, which is #424's failure from the " + "other side. An earlier round of #533 released it and " + "read family 'van der DO Berg' in silence, and the " + "clause-less 'Berg, Jane van der DO' reads that way " + "for its own reasons and is unchanged", + shape=2), + Case("two_released_particle_members_would_chain_each_other", + "Jane Doe nee Smith DO DO", + {"given": "Jane", "family": "Doe", "maiden": "Smith DO DO"}, + classification="parity", + ambiguities=("suffix-or-name",), + notes="the no-comma spelling of the same decline, where what " + "would do the joining is the OTHER released member: " + "the trailing peel reads both 'DO's as credentials, " + "but the moment they are out of the clause the first " + "is a non-leading particle and chains the second into " + "family 'DO DO'. An earlier round of #533 read exactly " + "that, and in silence", + shape=2), + Case("the_default_vocabularys_own_corpus_mover", + "John née Jones Smith Ma", + {"family": "John", "maiden": "Jones Smith Ma"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="the one name in the pre-existing differential corpus " + "this rule reaches at the DEFAULT vocabulary, and it " + "moves by gaining the REPORT rather than a field: " + "'Ma' is Title-case inside a mixed-case name, so the " + "lean declines the credential reading and the clause " + "keeps it -- which is a fork called, and now said out " + "loud. It was only ever pinned under a test lexicon " + "before. Already a corpus_rules.jsonl name, so the " + "shape tag re-witnesses it rather than growing the " + "deduped corpus", + shape=2), + Case("a_member_alone_after_the_marker_keeps_its_credential_behind", + "Jane Doe nee MA PhD", + {"given": "Jane", "family": "Doe", "suffix": "PhD", + "maiden": "MA"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="THE FIRST-WORD FLOOR with an unambiguous credential " + "behind it: the walk may not take the first word after " + "the marker, so 'MA' stays the maiden name whatever " + "the peel read, and the 'PhD' behind it was never this " + "rule's to give -- a suffix WORD ends the clause the " + "way it always did. The report is the clause's own", + shape=2), + Case("a_particle_member_declines_on_its_lean_after_a_comma", + "Doe, Jane nee Smith Do", + {"given": "Jane", "family": "Doe", "maiden": "Smith Do"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="#531's reading at the given slot, reached through a " + "clause: a member that is also particle vocabulary is " + "the credential on a POSITIVE lean alone, and " + "Title-case inside a mixed-case name is not one. So " + "the clause keeps it and says so. The caps spelling " + "'Doe, Jane nee Smith DO' is the other direction", + shape=2), + Case("a_numeral_between_the_clause_and_the_member", + "Jane Doe nee Smith V MA", + {"given": "Jane", "family": "Doe", "suffix": "MA", + "maiden": "Smith V"}, + classification="fix(#533)", + ambiguities=("suffix-or-name",), + notes="both stops read the TRAILING word, so the numeral is " + "not the word either fork asks about: the acronym fork " + "stops at 'MA' and 'V' stays maiden text behind it. " + "The mirror image, 'Jane Smith née Jones Ma V', keeps " + "maiden 'Jones Ma' and reads suffix 'V'", + shape=2), + Case("delimiters_keep_the_whole_span_whatever_the_last_word_is", + "Jane Doe (nee Smith MA)", + {"given": "Jane", "family": "Doe", "maiden": "Smith MA"}, + classification="parity", + ambiguities=(), + notes="rules.md#M2, the delimited clause: the writer drew " + "the boundary and it outranks every reading inside it, " + "so no fork is called and none is reported. The marker " + "inside the pair is what says the span is a maiden " + "clause even where the pair is not in " + "`maiden_delimiters`. Unchanged by #533 and by every " + "release before it -- a settled position, not one of " + "the silences about un-asked forks", + shape=2), + Case("delimiters_keep_the_whole_span_in_title_case_too", + "Jane Doe (nee Smith Ma)", + {"given": "Jane", "family": "Doe", "maiden": "Smith Ma"}, + classification="parity", + ambiguities=(), + notes="the pair for the row above: the two spellings differ " + "in everything the lean reads and the delimiters make " + "the difference immaterial. Undelimited, 'Jane Doe nee " + "Smith MA' reads suffix 'MA' and 'Jane Doe nee Smith " + "Ma' keeps it -- and both report", + shape=2), + Case("a_word_outside_the_delimiters_is_outside_the_clause", + "Jane Doe (nee Smith) MA", + {"given": "Jane", "family": "Doe", "suffix": "MA", + "maiden": "Smith"}, + classification="parity", + ambiguities=("suffix-or-name",), + notes="the boundary cuts both ways: the span is the maiden " + "name whole, and a member the writer left OUTSIDE it " + "is an ordinary trailing credential that assign peels " + "and reports. The control for the two rows above", + shape=2), + Case("two_released_members_each_get_their_own_report", + "Jane Doe nee Smith Ma JD", + {"given": "Jane", "family": "Doe", "suffix": "JD", + "maiden": "Smith Ma"}, + classification="fix(#533)", + ambiguities=("suffix-or-name", "suffix-or-name"), + notes="the shape that puts BOTH suffix-or-name emitters on " + "one parse: the clause keeps 'Ma' on its lean and " + "reports it, assign peels 'JD' and reports that, and " + "the two name different tokens -- which is what " + "test_properties' span test needs to actually " + "exercise its own check", shape=2), Case("no_trailing_rule_reads_the_family_segments_clause", "Smith nee Jones, Jane MA", diff --git a/tests/v2/pipeline/test_group.py b/tests/v2/pipeline/test_group.py index 439762ff..d484d631 100644 --- a/tests/v2/pipeline/test_group.py +++ b/tests/v2/pipeline/test_group.py @@ -1,17 +1,22 @@ import bisect import dataclasses +from collections.abc import Sequence +from typing import cast import pytest from nameparser._lexicon import Lexicon +from nameparser._pipeline import _group as _group_module from nameparser._pipeline._classify import classify from nameparser._pipeline._extract import extract_delimited, _maiden_marked from nameparser._pipeline._group import ( - _group_segment, group, marker_run_length, + TailReader, _group_segment, group, marker_run_length, ) from nameparser._pipeline._script_segment import script_segment from nameparser._pipeline._segment import segment -from nameparser._pipeline._state import ParseState, PendingAmbiguity +from nameparser._pipeline._state import ( + ParseState, PendingAmbiguity, Structure, WorkToken, +) from nameparser._pipeline._tokenize import tokenize from nameparser._pipeline._vocab import maiden_marker_run from nameparser._policy import Policy, Script @@ -1074,20 +1079,118 @@ def test_the_maiden_report_survives_the_family_comma_suppression( assert len(_suffix_forks(out)) == 1 -def test_an_unnamed_maiden_channel_falls_back_to_the_general_one( -) -> None: - """The second channel's DEFAULT, which group() never takes -- it - names both lists at every call. A caller of the segment function - that names only the one gets the one: the maiden fork reports - into it rather than into a list nobody reads, which is what makes - the parameter a routing choice rather than a second switch.""" +def test_the_two_ambiguity_channels_route_independently() -> None: + """Both channels are REQUIRED arguments, and they are two so that + silencing one never silences the other. + + `reader` and `maiden_ambiguities` have no defaults: the one + production caller answers both off the segment's structure, and a + default would be this module guessing what that caller knows. The + routing is what the split buys -- the same list in both slots is + one channel, two lists are two, and #533's review found the + earlier spelling defaulting the maiden channel to whatever the + first was, so `ambiguities=None` silenced both. + """ state = classify(segment(tokenize(extract_delimited(ParseState( original="Jane Doe née Smith Ma", lexicon=_AMBIGUOUS_LEX, policy=Policy()))))) - reported: list[PendingAmbiguity] = [] + general: list[PendingAmbiguity] = [] + maiden: list[PendingAmbiguity] = [] _group_segment(state.segments[0], 0, state.tokens, - ambiguities=reported, one_case=state.one_case) - assert [a.kind for a in reported] == [AmbiguityKind.SUFFIX_OR_NAME] + ambiguities=general, one_case=state.one_case, + reader=TailReader.TRAILING, + maiden_ambiguities=maiden) + assert [a.kind for a in maiden] == [AmbiguityKind.SUFFIX_OR_NAME] + assert general == [] + # the general channel suppressed, the maiden one still speaks -- + # which is exactly what group() does after a family comma + only_maiden: list[PendingAmbiguity] = [] + _group_segment(state.segments[0], 0, state.tokens, + ambiguities=None, one_case=state.one_case, + reader=TailReader.TRAILING, + maiden_ambiguities=only_maiden) + assert [a.kind for a in only_maiden] == [AmbiguityKind.SUFFIX_OR_NAME] + # and NONE is the reader that silences the maiden channel itself, + # because nothing was decided there + silent: list[PendingAmbiguity] = [] + _group_segment(state.segments[0], 0, state.tokens, + ambiguities=None, one_case=state.one_case, + reader=TailReader.NONE, maiden_ambiguities=silent) + assert silent == [] + + +def test_an_unmapped_reader_is_a_loud_failure_rather_than_a_default( +) -> None: + """The exhaustive dispatch, exercised. + + `_maiden_take` ends its reader branch with `assert_never`, which + makes a fourth `TailReader` member a mypy error at this site + rather than a silent fall-through to one of the three readings. + At RUNTIME that line is unreachable by construction, so it is + reached here the only way it can be -- with a value outside the + enum -- both to pin the loudness and to keep the line from being + the one uncovered statement in the module. + """ + state = classify(segment(tokenize(extract_delimited(ParseState( + original="Jane Doe née Smith MA", lexicon=Lexicon.default(), + policy=Policy()))))) + with pytest.raises(AssertionError): + _group_segment(state.segments[0], 0, state.tokens, + ambiguities=[], one_case=state.one_case, + reader=cast(TailReader, 99), + maiden_ambiguities=[]) + + +def test_the_reader_is_pinned_to_the_structure_it_is_read_from( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """`TailReader` is a closed set, and group() maps (structure, + segment index) onto it in one place. Pinned here by WATCHING that + mapping rather than restating it -- a restatement passes when the + code changes under it, which is the shape of vacuous guard + AGENTS.md warns about. `_maiden_take` dispatches on the enum + exhaustively (`assert_never`), so a fourth member with no row + here is a type error rather than a silent default. + """ + assert len(TailReader) == 3 + want = { + # no comma: the whole name, read by the S2 peel + (Structure.NO_COMMA, 0): TailReader.TRAILING, + # suffix comma: segment 0 is the name, the rest is the + # credential run and is read whole + (Structure.SUFFIX_COMMA, 0): TailReader.TRAILING, + (Structure.SUFFIX_COMMA, 1): TailReader.NONE, + (Structure.SUFFIX_COMMA, 2): TailReader.NONE, + # family comma: segment 0 is the family the comma named, + # segment 1 is the given part with its own trailing slot + # (#531), and a third part is a credential run again + (Structure.FAMILY_COMMA, 0): TailReader.NONE, + (Structure.FAMILY_COMMA, 1): TailReader.GIVEN_SLOT, + (Structure.FAMILY_COMMA, 2): TailReader.NONE, + } + texts = ("Jane Doe née Smith MA", + "Jane Doe née Smith, MD, PhD", + "Doe, Jane née Smith MA, MD") + seen: dict[tuple[Structure, int], TailReader] = {} + real = _group_module._group_segment + + def spy(seg: tuple[int, ...], additional: int, + tokens: Sequence[WorkToken], *args: object, + **kwargs: object) -> object: + seen[(state.structure, len(seen_order))] = cast( + TailReader, kwargs["reader"]) + seen_order.append(seg) + return real(seg, additional, tokens, *args, **kwargs) # type: ignore[arg-type] + + monkeypatch.setattr(_group_module, "_group_segment", spy) + for text in texts: + seen_order: list[tuple[int, ...]] = [] + state = classify(segment(tokenize(extract_delimited(ParseState( + original=text, lexicon=Lexicon.default(), + policy=Policy()))))) + group(state) + assert seen == want, ( + f"group() maps the structures to {seen}, pinned as {want}") def test_a_marker_followed_only_by_the_numeral_is_just_a_word() -> None: diff --git a/tests/v2/test_benchmark.py b/tests/v2/test_benchmark.py index c1c0a452..2d0c316a 100644 --- a/tests/v2/test_benchmark.py +++ b/tests/v2/test_benchmark.py @@ -208,6 +208,13 @@ def test_a_thousand_names_still_parse_in_reasonable_time( # first piece is a bound given-name word, # so the reserve's per-piece question is # asked over the whole run (#401) +# M2 clause VIEW maiden_clause ONLY -- the one unit that +# reaches the #533 acronym fork, which +# needs a maiden marker AND a class member +# ending the string. 'MA nee ' has both +# words and reaches nothing: the peel +# stops at the trailing marker, so the +# ORDER inside the unit is the shape _SHAPES = { "delimiter_pairs": "(a) ", # extract: matched pairs -> masked spans "quote_pairs": '"a" ', # extract: the open==close path @@ -220,6 +227,7 @@ def test_a_thousand_names_still_parse_in_reasonable_time( "conjunctions": "and ", # group: merge() accumulating one piece "honorifics": "씨 ", # script_segment: the peel's site scan "bound_given": "abdul ", # group: the P5 reserve over every piece + "maiden_clause": "nee MA ", # group: M2's view over the segment } _BASE = 800 @@ -256,6 +264,15 @@ def test_a_thousand_names_still_parse_in_reasonable_time( # review rather than here, no earlier unit leading with a bound word. # Computed once, the shape reads 4.2, inside the clean column; neither # number moved. +# The twelfth (maiden_clause, #533 review) arrived the same way, with +# its own quadratic in hand. M2's walk builds ONE view per take over +# the segment's own indices, and a rewrite that rebuilt those indices +# per piece -- behavior-identical, and it passed the whole suite -- +# measures 8.3 at base 200, 10.4 at 400 and 12.2-12.6 at 800 against +# a clean 4.0-4.1 at every one of the three, repeated runs. So the +# signal is the strongest of the three quadratics on record and does +# not decide the bound; the shape reads 4.05-4.12 at base 800 across +# repeated runs, inside the clean column, and neither number moved. _MAX_RATIO = 6.0 diff --git a/tests/v2/test_ledger_guards.py b/tests/v2/test_ledger_guards.py index 3f7587e5..06920793 100644 --- a/tests/v2/test_ledger_guards.py +++ b/tests/v2/test_ledger_guards.py @@ -1309,12 +1309,27 @@ def test_case_shape_ids_exist_in_the_inventory() -> None: ("Jane Doe nee Smith MA", "John Doe X.Y.Z.", "Doe, John X.Y.Z.", "Jane Doe nee Smith X.Y.Z", "Jane Doe nee Smith XYZ"), - # The two names where the released member lands elsewhere must - # not reach their own clause-less controls -- which is the pair - # the rule's argument rests on -- nor the spellings that decline. - "fix(#533) accepted: the released credential lands where the segment it is released into puts it": - ("Berg, abdul MA", "Doe, Dr. MA", "Doe, Dr. nee Smith Ma", - "Berg, abdul nee Jones Ma", "Doe, Dr. nee Smith MA PhD"), + # The names whose clause KEEPS the member because no reader or a + # join would have taken it must not reach their own clause-less + # controls -- which is what the rule's argument rests on: 'Berg, + # abdul MA' really does read given 'abdul MA', and that is the + # reading the decline refuses to hand a clause word to. + "fix(#274/#533) the clause keeps the credential a join or a missing reader would have taken": + ("Berg, abdul MA", "Doe, Dr. MA", "Berg, Jane van der DO", + "Doe, Jane MA do", "Jane Doe nee Smith MA"), + # Same, at the 2.x baselines where the decline shows as a report + # rather than as a role move. + "fix(#411/#533) the bound-given pair after a comma": + ("Berg, abdul MA", "Berg, abdul nee Jones Ma", + "Berg, abdul nee Jones"), + "fix(#399/#533) a maiden marker bounds the particle chain, and the clause keeps": + ("Berg, Jane van der DO", "Berg, Jane van der nee Smith Do", + "Berg, Jane van der nee Smith"), + # The delimited rule must not reach the undelimited spellings, + # which is the whole of what it says: the boundary is the writer's. + "fix(#335/#533) a marker-led bracketed clause is the maiden name whatever its last word is": + ("Jane Doe nee Smith MA", "Jane Doe nee Smith Ma", + "Jane Smith (née Jones)"), # The restoration is one name and one spelling: not the same # acronym without the clause, and not a spelling the dotted gate # reads differently. @@ -2059,6 +2074,17 @@ class _LatinCopy(NamedTuple): #: question someone answers in writing, not something to skip past. _NOT_A_VOCABULARY_COPY = frozenset({ frozenset({"^", " "}), # the honorific rule's leading anchor + # The #533 review's two literal-anchored rules, one alternative + # per corpus name. Lists of names, not copies of any wordlist: + # what selects them is the SHAPE the clause's decline turns on -- + # no reader ahead of the member, or a join below the marker pass + # that would take it -- and no vocabulary decides that. The + # delimited trio is the same, keyed on the writer's brackets. + frozenset({"Berg, Jane van der nee Smith DO", "Berg, abdul nee Jones MA", + "Doe, Dr\\. nee Smith MA", "Doe, Jane nee Smith Do", + "Jane Doe nee MA PhD"}), + frozenset({"Jane Doe \\(nee Smith MA\\)", "Jane Doe \\(nee Smith Ma\\)", + "Jane Doe \\(nee Smith\\) MA"}), # fix(#400)'s two openings: start-of-name or just after a family # comma. `abd` joins forward on the given side wherever that side # begins, and the alternation is over ANCHORS, not over words -- @@ -2484,7 +2510,8 @@ class _LatinCopy(NamedTuple): "JANE DOE NEE YO-YO MA", r"Jane Doe geb\. Smith MA", "Jane Doe nee Smith MA", "Jane Doe nee Smith MA JD", "Jane Doe nee Smith MA PhD", "Jane Doe nee Smith Ma JD", - r"Jane Doe nee Smith Prof\. MA", "Jane Doe nee Smith do", + r"Jane Doe nee Smith Prof\. MA", "Jane Doe nee Smith V MA", + "Jane Doe nee Smith do", "John née Jones Smith MA", "Maria Kowalska z domu Nowak MA", "jane doe nee smith ma"}), # and at 2.0.0 and 2.1.0 (fourteen). @@ -2494,13 +2521,17 @@ class _LatinCopy(NamedTuple): r"Jane Doe geb\. Smith MA", "Jane Doe nee Smith MA", "Jane Doe nee Smith MA JD", "Jane Doe nee Smith MA PhD", "Jane Doe nee Smith Ma JD", - r"Jane Doe nee Smith Prof\. MA", "Jane Doe nee Smith do", + r"Jane Doe nee Smith Prof\. MA", "Jane Doe nee Smith V MA", + "Jane Doe nee Smith do", "jane doe nee smith ma"}), # The names the clause KEEPS and now reports, at 2.2.0 and 2.3.0 # (eight), - frozenset({"Doe, Jane nee Smith MA do", "Doe, Jane nee Smith Ma", + frozenset({"Berg, Jane van der nee Smith DO", "Berg, abdul nee Jones MA", + r"Doe, Dr\. nee Smith MA", "Doe, Jane nee Smith Do", + "Doe, Jane nee Smith MA do", "Doe, Jane nee Smith Ma", "Doe, Jane nee Smith do", "JOHN NEE JONES SMITH MA PHD", - "Jane Doe nee MA", "Jane Doe nee Smith Ma", + "Jane Doe nee MA", "Jane Doe nee MA PhD", + "Jane Doe nee Smith DO DO", "Jane Doe nee Smith Ma", "Jane Doe nee Yo-Yo Ma", "John née Jones Smith Ma"}), # and at 2.0.0 and 2.1.0 (five). frozenset({r"Doe, J\. nee MA ba", "Doe, Jane nee Smith Ma", @@ -2510,8 +2541,7 @@ class _LatinCopy(NamedTuple): # than the trailing peel. One set, shared by the 1.4.0 rule and # the 2.2.0/2.3.0 one; at 2.0.0 and 2.1.0 the rule holds one name # and has no alternation to declare. - frozenset({"Berg, abdul nee Jones MA", r"Doe, Dr\. nee Smith MA"}), - # The 1.4.0 pair and the 1.4.0 halves. The clause KEEPS the member + # The 1.4.0 pair and the 1.4.0 halves. The clause KEEPS the member # in six of them, which at that baseline is not a report but the # `suffix` v1 read emptying; it gives the member up in four, whose # runs v1 also wrote with commas. The two-member set is the @@ -2521,6 +2551,10 @@ class _LatinCopy(NamedTuple): frozenset({"Doe, Jane nee Smith MA do", "Doe, Jane nee Smith Ma", "Doe, Jane nee Smith do", "Jane Doe nee MA", "Jane Doe nee Smith Ma", "Jane Doe nee Yo-Yo Ma"}), + frozenset({r"Doe, Dr\. nee Smith MA", r"Doe, J\. nee MA ba", + "Doe, Jane nee Smith Ma", "Jane Doe nee MA", + "Jane Doe nee MA PhD", "Jane Doe nee Smith DO DO", + "Jane Doe nee Smith Ma", "Jane Doe nee Yo-Yo Ma"}), frozenset({r"Doe, J\. nee MA ba", "Jane Doe nee Smith MA JD", "Jane Doe nee Smith MA PhD", "Jane Doe nee Smith Ma JD"}), frozenset({"JOHN NEE JONES SMITH MA PHD", @@ -3004,7 +3038,7 @@ def _claim(rule: dict) -> _Claim: # carrying a marker. Read name by name against the regex; no # role joined the list. "fix(#274) maiden markers consumed": - _Claim(68, ('family', 'maiden', 'middle'), "19914ae9948e", None), + _Claim(73, ('family', 'maiden', 'middle'), 'a2e495d071dd', None), # 2026-09-19, #533: 5 -> 6, the same one new corpus name # '田中 太郎 旧姓 佐藤 MA' as the CJK rule above. "fix(cjk-maiden-marker) maiden marker consumed, compounding with the CJK order flip": @@ -3037,7 +3071,7 @@ def _claim(rule: dict) -> _Claim: # added. Reach, not explanation: all three are the contest # fix(#274) is now declared to outrank. "fix(#379) a tussenvoegsel after a family comma attaches to the family": - _Claim(24, ('family', 'middle'), "0c62afd7f400", None), + _Claim(26, ('family', 'middle'), '22c55d325d9d', None), "fix(#380) a trailing vd after a family comma is the tussenvoegsel, not a post-nominal": _Claim(2, ('family', 'suffix'), "ec0d45289dc1", None), # 279 -> 280 with #371, and the growth is corpus, not behavior: @@ -3097,7 +3131,7 @@ def _claim(rule: dict) -> _Claim: # comma and every family-comma row this change added matches # it. "fix(comma-family) lone post-comma piece routes to suffix/title, not first": - _Claim(356, ('given', 'suffix', 'title'), "16c571069b0c", None), + _Claim(358, ('given', 'suffix', 'title'), '0485bb024e5a', None), "fix(comma-family) a comma followed only by titles keeps the given/family split": _Claim(2, ('family', 'given'), "5bd9c6d96c38", None), "fix(comma-family) a comma followed only by titles keeps the given/family split, the C1 example": @@ -3148,7 +3182,7 @@ def _claim(rule: dict) -> _Claim: # 2026-09-19, #533: 343 -> 356, the same thirteen new comma # names as the rule above and for the same reason. "fix(comma-precomma-family) pre-comma run reads as family, not given": - _Claim(356, ('family', 'given'), "16c571069b0c", None), + _Claim(358, ('family', 'given'), '0485bb024e5a', None), "fix(#397) NOT WANTED: a trailing Catalan/Polish linking 'i' is read as a generation marker and the family is lost": _Claim(1, ('family', 'suffix'), "498602f3cfd0", None), "fix(suffix-delimiter-rendering) no-space delimiter core token kept whole": @@ -3382,7 +3416,7 @@ def _claim(rule: dict) -> _Claim: # 'john van der berg ma' -- rules.md#P2's one-case contrast, # and a particle chain like every other member. "fix(initials-per-word) a particle chain inside a name part initials each word (facade, since 2.0.0)": - _Claim(110, ('_initials',), "9193a1954e3b", ('DEFAULT',)), + _Claim(111, ('_initials',), '3729c1e3152d', ('DEFAULT',)), "fix(initials-per-word) the Ph. D. merge initials each word (facade, since 2.0.0)": _Claim(18, ('_initials',), "f67d8ebddd56", ('DEFAULT',)), # The 2.3 title-run bundle's five rules, last in every @@ -3561,13 +3595,10 @@ def _claim(rule: dict) -> _Claim: # reached a name whose member moved. "fix(#434/#533) a marker PHRASE takes the maiden name, and its clause ends at the credential": _Claim(1, ('family', 'maiden', 'middle'), "bf8359f65c8a", None), - # Two corpus names at 1.4.0 and 2.2.0/2.3.0, one at 2.0.0 and - # 2.1.0 where fix(#411) still carries the bound-given half. - # No `_ambiguities` at any baseline, which is the row's own - # check: both readings are silent, and a report appearing - # here means one of them started speaking. - "fix(#533) accepted: the released credential lands where the segment it is released into puts it": - _Claim(2, ('given', 'maiden', 'middle', 'suffix'), "b3280b888d7c", None), + "fix(#274/#533) the clause keeps the credential a join or a missing reader would have taken": + _Claim(5, ('family', 'given', 'maiden', 'middle', 'suffix'), 'ae9d39617af1', None), + "fix(#335/#533) a marker-led bracketed clause is the maiden name whatever its last word is": + _Claim(3, ('maiden', 'nickname'), 'cc1045ecdc05', None), }, "expected_since_2.0.0.toml": { # #436/#437's Latin alternation, first in every ledger. @@ -3680,7 +3711,7 @@ def _claim(rule: dict) -> _Claim: # 2026-09-19, #533: 21 -> 24, the same three new corpus # names as the 1.4.0 copy -- the do pair this change added # after a family comma. - _Claim(24, ('_ambiguities', 'family', 'middle'), "0c62afd7f400", None), + _Claim(26, ('_ambiguities', 'family', 'middle'), '22c55d325d9d', None), # 2026-09-18: 126 -> 131. Five corpus names arrived with # #289/#516's own case rows -- 'J.씨', 'John Smith 田.中.', # '毛泽东, MA', '田中 太郎, MA', '마틴 킹, MA' -- all of them @@ -3999,25 +4030,18 @@ def _claim(rule: dict) -> _Claim: # so a widening taking any of the three alone would change # the roles here before it reached the gate. "fix(#533) a credential ending a maiden clause reads as a credential": - _Claim(14, ('_ambiguities', 'maiden', 'suffix'), "2147fddf943a", None), + _Claim(15, ('_ambiguities', 'maiden', 'suffix'), '1b2218cb6296', None), # The declining half: eight corpus names at 2.2.0 and 2.3.0, # five at 2.0.0 and 2.1.0. `_ambiguities` alone, so a role # appearing here is this rule reaching a name whose clause # gave the member up. "fix(#533) the maiden clause reports the credential it keeps": - _Claim(5, ('_ambiguities',), "a51ac503d15d", None), + _Claim(8, ('_ambiguities',), '329a6472b344', None), # One corpus name. The by-shape member has no lean to read, # so a growth here is the rule reaching a LISTED member -- # a different reading under this rule's sentence. "fix(#533) accepted: an unlisted dotted acronym ending a maiden clause is read by position": _Claim(1, ('_ambiguities', 'maiden', 'suffix'), "0cc7bd35a9b3", None), - # Two corpus names at 1.4.0 and 2.2.0/2.3.0, one at 2.0.0 and - # 2.1.0 where fix(#411) still carries the bound-given half. - # No `_ambiguities` at any baseline, which is the row's own - # check: both readings are silent, and a report appearing - # here means one of them started speaking. - "fix(#533) accepted: the released credential lands where the segment it is released into puts it": - _Claim(1, ('maiden', 'suffix'), "4cc6318d5d14", None), # One corpus name, `_ambiguities` alone at every 2.x # baseline: the roles are the ones these releases already # read, which is what makes the row a restoration. A role @@ -4047,6 +4071,12 @@ def _claim(rule: dict) -> _Claim: # growth here is the numeral rule reaching an acronym name. "fix(#424) the maiden walk stops before the trailing numeral, with the family name left standing": _Claim(1, ('_ambiguities', 'maiden', 'suffix'), "170a53c37765", None), + "fix(#335/#533) a marker-led bracketed clause is the maiden name whatever its last word is": + _Claim(3, ('_ambiguities', 'maiden', 'nickname'), 'cc1045ecdc05', None), + "fix(#399/#533) a maiden marker bounds the particle chain, and the clause keeps the credential the chain would have taken": + _Claim(1, ('_ambiguities', 'family', 'maiden', 'middle'), 'fa3fe4878b47', None), + "fix(#411/#533) the bound-given pair after a comma, and the clause it keeps its credential in": + _Claim(1, ('_ambiguities', 'given', 'maiden', 'middle'), '431d3dd24c14', None), }, # The 2.3 cycle's first rule, and a facade-only render fix: every # role is identical, so `_initials` alone. Reach and digest as in @@ -4275,26 +4305,19 @@ def _claim(rule: dict) -> _Claim: # so a widening taking any of the three alone would change # the roles here before it reached the gate. "fix(#533) a credential ending a maiden clause reads as a credential": - _Claim(17, ('_ambiguities', 'maiden', 'suffix'), "3051b7cde472", None), + _Claim(18, ('_ambiguities', 'maiden', 'suffix'), '3f7d5fd3cc4b', None), # The declining half: eight corpus names at 2.2.0 and 2.3.0, # five at 2.0.0 and 2.1.0. `_ambiguities` alone, so a role # appearing here is this rule reaching a name whose clause # gave the member up. "fix(#533) the maiden clause reports the credential it keeps": - _Claim(8, ('_ambiguities',), "0b36d3482ba6", None), + _Claim(14, ('_ambiguities',), '24fe6a424eda', None), # One corpus name. The by-shape member has no lean to read, # so a growth here is the rule reaching a LISTED member -- # a different reading under this rule's sentence. "fix(#533) accepted: an unlisted dotted acronym ending a maiden clause is read by position": _Claim(1, ('_ambiguities', 'maiden', 'suffix'), "0cc7bd35a9b3", None), - # Two corpus names at 1.4.0 and 2.2.0/2.3.0, one at 2.0.0 and - # 2.1.0 where fix(#411) still carries the bound-given half. - # No `_ambiguities` at any baseline, which is the row's own - # check: both readings are silent, and a report appearing - # here means one of them started speaking. - "fix(#533) accepted: the released credential lands where the segment it is released into puts it": - _Claim(2, ('given', 'maiden', 'suffix'), "b3280b888d7c", None), - # One corpus name, `_ambiguities` alone at every 2.x +# One corpus name, `_ambiguities` alone at every 2.x # baseline: the roles are the ones these releases already # read, which is what makes the row a restoration. A role # appearing here would mean the restoration had stopped @@ -4427,7 +4450,7 @@ def _claim(rule: dict) -> _Claim: # 2026-09-19, #533: 21 -> 24, the same three new corpus # names as the 1.4.0 copy -- the do pair this change added # after a family comma. - _Claim(24, ('_ambiguities', 'family', 'middle'), "0c62afd7f400", None), + _Claim(26, ('_ambiguities', 'family', 'middle'), '22c55d325d9d', None), "fix(#424) an unlisted abbreviation is as transparent as a listed title to the leading particle": _Claim(1, ('_ambiguities', 'family', 'given'), "ca7b37af6cf8", None), "fix(#367) a title no longer displaces a leading particle out of the leading position": @@ -4703,26 +4726,19 @@ def _claim(rule: dict) -> _Claim: # so a widening taking any of the three alone would change # the roles here before it reached the gate. "fix(#533) a credential ending a maiden clause reads as a credential": - _Claim(14, ('_ambiguities', 'maiden', 'suffix'), "2147fddf943a", None), + _Claim(15, ('_ambiguities', 'maiden', 'suffix'), '1b2218cb6296', None), # The declining half: eight corpus names at 2.2.0 and 2.3.0, # five at 2.0.0 and 2.1.0. `_ambiguities` alone, so a role # appearing here is this rule reaching a name whose clause # gave the member up. "fix(#533) the maiden clause reports the credential it keeps": - _Claim(5, ('_ambiguities',), "a51ac503d15d", None), + _Claim(8, ('_ambiguities',), '329a6472b344', None), # One corpus name. The by-shape member has no lean to read, # so a growth here is the rule reaching a LISTED member -- # a different reading under this rule's sentence. "fix(#533) accepted: an unlisted dotted acronym ending a maiden clause is read by position": _Claim(1, ('_ambiguities', 'maiden', 'suffix'), "0cc7bd35a9b3", None), - # Two corpus names at 1.4.0 and 2.2.0/2.3.0, one at 2.0.0 and - # 2.1.0 where fix(#411) still carries the bound-given half. - # No `_ambiguities` at any baseline, which is the row's own - # check: both readings are silent, and a report appearing - # here means one of them started speaking. - "fix(#533) accepted: the released credential lands where the segment it is released into puts it": - _Claim(1, ('maiden', 'suffix'), "4cc6318d5d14", None), - # One corpus name, `_ambiguities` alone at every 2.x +# One corpus name, `_ambiguities` alone at every 2.x # baseline: the roles are the ones these releases already # read, which is what makes the row a restoration. A role # appearing here would mean the restoration had stopped @@ -4757,6 +4773,12 @@ def _claim(rule: dict) -> _Claim: # through. "fix(#533) the maiden clause ends at the credential in a native-script name too": _Claim(1, ('_ambiguities', 'maiden', 'suffix'), "6bab87214ddf", None), + "fix(#335/#533) a marker-led bracketed clause is the maiden name whatever its last word is": + _Claim(3, ('_ambiguities', 'maiden', 'nickname'), 'cc1045ecdc05', None), + "fix(#399/#533) a maiden marker bounds the particle chain, and the clause keeps the credential the chain would have taken": + _Claim(1, ('_ambiguities', 'family', 'maiden', 'middle'), 'fa3fe4878b47', None), + "fix(#411/#533) the bound-given pair after a comma, and the clause it keeps its credential in": + _Claim(1, ('_ambiguities', 'given', 'maiden', 'middle'), '431d3dd24c14', None), }, "expected_since_2.3.0.toml": { # #383/#479's three rules, the first this ledger carries. The @@ -4854,26 +4876,19 @@ def _claim(rule: dict) -> _Claim: # so a widening taking any of the three alone would change # the roles here before it reached the gate. "fix(#533) a credential ending a maiden clause reads as a credential": - _Claim(17, ('_ambiguities', 'maiden', 'suffix'), "3051b7cde472", None), + _Claim(18, ('_ambiguities', 'maiden', 'suffix'), '3f7d5fd3cc4b', None), # The declining half: eight corpus names at 2.2.0 and 2.3.0, # five at 2.0.0 and 2.1.0. `_ambiguities` alone, so a role # appearing here is this rule reaching a name whose clause # gave the member up. "fix(#533) the maiden clause reports the credential it keeps": - _Claim(8, ('_ambiguities',), "0b36d3482ba6", None), + _Claim(14, ('_ambiguities',), '24fe6a424eda', None), # One corpus name. The by-shape member has no lean to read, # so a growth here is the rule reaching a LISTED member -- # a different reading under this rule's sentence. "fix(#533) accepted: an unlisted dotted acronym ending a maiden clause is read by position": _Claim(1, ('_ambiguities', 'maiden', 'suffix'), "0cc7bd35a9b3", None), - # Two corpus names at 1.4.0 and 2.2.0/2.3.0, one at 2.0.0 and - # 2.1.0 where fix(#411) still carries the bound-given half. - # No `_ambiguities` at any baseline, which is the row's own - # check: both readings are silent, and a report appearing - # here means one of them started speaking. - "fix(#533) accepted: the released credential lands where the segment it is released into puts it": - _Claim(2, ('given', 'maiden', 'suffix'), "b3280b888d7c", None), - # One corpus name, `_ambiguities` alone at every 2.x +# One corpus name, `_ambiguities` alone at every 2.x # baseline: the roles are the ones these releases already # read, which is what makes the row a restoration. A role # appearing here would mean the restoration had stopped @@ -6294,7 +6309,13 @@ class _Excluded(NamedTuple): # costs the entry nothing: 1.4.0 reads the parenthesized pair # as a nickname exactly as the tree does, so there is no diff # to silence. - _Excluded(59, "572f5b8f850a", ()), + # 59 -> 62 on 2026-09-19 for the #533 review's three delimited + # maiden names ('Jane Doe (nee Smith MA)', '... Ma)', '(nee + # Smith) MA'), whose parentheses match this shape. It silences + # none of their diff either: the fix(#335/#533) rule carries + # it at every baseline that has one, and `absorbed_by` stays + # empty, which is the half of this record that matters. + _Excluded(62, "7fdfb86d426e", ()), } @@ -6522,8 +6543,25 @@ def test_a_rule_reaching_no_corpus_name_says_why_it_is_kept() -> None: # earlier rule now carries the [[change.precedes_narrower]] # block saying why it wins; this control records the contest # whether or not it is declared away. + # 3 → 5 on 2026-09-19: the #533 review's two `do`-bearing + # corpus rows ('Berg, Jane van der nee Smith DO', 'Doe, Jane + # nee Smith Do') land inside both regexes. Shared REACH, not + # a widening -- the declared pair is unchanged. ("fix(#274) maiden markers consumed", - "fix(#379) a tussenvoegsel after a family comma attaches to the family", 3), + "fix(#379) a tussenvoegsel after a family comma attaches to the family", 5), + # The #533 review's own rule is wide-first against three + # narrower ones, each declared on it with a + # [[change.precedes_narrower]] block in the 1.4.0 ledger. + # Recorded here for the reason this whole roster exists: the + # declarations say the contests are INTENDED, and these rows + # say how much corpus they cover, so a rule quietly growing + # into a fourth pair fails here. + ("fix(#274/#533) the clause keeps the credential a join or a missing reader would have taken", + "fix(comma-precomma-family) pre-comma run reads as family, not given", 4), + ("fix(#274/#533) the clause keeps the credential a join or a missing reader would have taken", + "fix(#411) the bound-given reserve stops counting words the maiden name takes", 1), + ("fix(#274/#533) the clause keeps the credential a join or a missing reader would have taken", + "fix(#379) a tussenvoegsel after a family comma attaches to the family", 2), ("fix(comma-family) a comma followed only by titles keeps the given/family split, the C1 example", "fix(comma-precomma-family) pre-comma run reads as family, not given", 2), ("fix(#296) a credential-only comma string reads a name and its postnominal", diff --git a/tests/v2/test_parser.py b/tests/v2/test_parser.py index 7a09863d..38d54698 100644 --- a/tests/v2/test_parser.py +++ b/tests/v2/test_parser.py @@ -681,43 +681,54 @@ def test_the_chain_and_the_walk_stop_where_the_peel_begins() -> None: assert (n.maiden, n.suffix) == ("Smith V", "MA") -def test_a_post_nominal_head_makes_the_clause_move_the_member_in_silence( +def test_a_post_nominal_head_leaves_the_clause_nobody_to_read_it( ) -> None: - """#533 design-docs review: a MOVER that reports nothing. - - After a family comma the given-slot reader takes the member, but - the take can leave that segment holding post-nominals only -- no - name word for the slot to be the end of -- and a part of nothing - but credentials is read whole and asked nothing. This is the - pre-existing "no name word in FRONT of the member" silence - (``AmbiguityKind.SUFFIX_OR_NAME``'s third position), which was - written as a TITLE's and which a post-nominal reaches just as - well; `Doe, Dr. nee Smith MA` is the titled spelling. - - No rules.md example line pins it, deliberately: the name differs - from this tree at all five differential baselines and most of the - diff belongs to other changes (1.4.0 reads title 'Jr', given - 'nee', family 'Jane Doe'), so an example line would have put five - ledger rules into a docs commit. This test is the pin instead. + """#533 review: the clause KEEPS the member, and says so. + + After a family comma the given-slot reader takes the member only + where the take leaves a given part for it to end. Where the part + before the marker is nothing but post-nominals, it does not: the + take would leave a segment of credentials, which is read whole + and asked nothing, and the released word would land in `given` + rather than in `suffix`. So the walk declines and the word stays + in the maiden name -- rules.md#M2's invariant, which replaced the + ACCEPTED silent mover an earlier round of this branch shipped + here (it read maiden 'Smith', suffix 'Jr MA'). + + The clause still REPORTS, and that is the measurement this test + exists for: the emitter is gated on there being a trailing rule + at all, not on the view check the rule then fails, so a fork + called the conservative way is still a fork the caller hears + about. """ n = parse("Jane Doe, Jr nee Smith MA") assert (n.given, n.family, n.maiden, n.suffix) == ( - "Jane", "Doe", "Smith", "Jr MA") - assert n.ambiguities == () - # it is a MOVER: 2f57ff21 read maiden 'Smith MA' - assert n.maiden == "Smith" - # and it agrees with the same name written without the clause, - # which is why the silence is right rather than a lost report - control = parse("Jane Doe, Jr MA") - assert control.suffix == "Jr MA" - assert control.ambiguities == () + "Jane", "Doe", "Smith MA", "Jr") + assert [a.kind for a in n.ambiguities] == [ + AmbiguityKind.SUFFIX_OR_NAME] + # 2f57ff21 read it this way too: the review round restored the + # parent reading rather than inventing a third one + assert n.maiden == "Smith MA" # a post-nominal, not only a generational word, heads it the same for head in ("III", "PhD"): n = parse(f"Jane Doe, {head} nee Smith MA") - assert (n.maiden, n.suffix) == ("Smith", f"{head} MA") - assert n.ambiguities == () - # the KEPT direction still reports -- the clause's own emitter is - # what raises it, and it never needed the given slot + assert (n.maiden, n.suffix) == ("Smith MA", head) + assert [a.kind for a in n.ambiguities] == [ + AmbiguityKind.SUFFIX_OR_NAME] + # and a TITLE heads it the same way, which is the spelling + # `AmbiguityKind.SUFFIX_OR_NAME`'s third position was written as + n = parse("Doe, Dr. nee Smith MA") + assert (n.title, n.family, n.maiden) == ("Dr.", "Doe", "Smith MA") + assert [a.kind for a in n.ambiguities] == [ + AmbiguityKind.SUFFIX_OR_NAME] + # the clause-less control is what the member WOULD have read as, + # and the difference is the point: without the clause there is a + # given name in front of the member and the slot exists + control = parse("Doe, Dr. Smith MA") + assert (control.given, control.suffix) == ("Smith", "MA") + # the KEPT direction reported before this round too -- the + # clause's own emitter is what raises it, and it never needed the + # given slot for text, maiden in (("Jane Doe, Jr nee Smith Ma", "Smith Ma"), ("Jane Doe, Jr nee MA", "MA")): n = parse(text) diff --git a/tests/v2/test_properties.py b/tests/v2/test_properties.py index 000f1a42..a2c73350 100644 --- a/tests/v2/test_properties.py +++ b/tests/v2/test_properties.py @@ -7,6 +7,8 @@ against regressions; exploratory fuzzing happened during review. """ import dataclasses +import hashlib +import re import warnings import pytest @@ -254,11 +256,20 @@ def test_the_comma_agreement_exceptions_are_all_still_exceptions( #: structurally, the judged token never being in the span at this #: slot. Accepted by Derek 2026-09-19 and recorded in #: decisions.md#S2 as the M2 instance of #492's deferred question. -#: The COUNT is asserted beside the class because a structural -#: allowlist cannot notice a 115th member of it: 114 of 2016 pairs on -#: 2026-09-19, against 186 allowlisted and 984 failing before the -#: change. -_MAIDEN_AGREEMENT_EXCEPTIONS = 114 +#: The class is asserted beside the count because a structural +#: allowlist cannot notice a new member of it, and a COUNT cannot +#: notice a swap -- one pair leaving and another arriving keeps the +#: number. So the SET is pinned, by digest: sha256 over the sorted +#: "||" lines, printed by the assertion +#: when it fails, which is how a deliberate move is re-recorded. +#: Re-measured 2026-09-19 on the widened grid below: 1,026 of 18,144 +#: pairs, which is exactly 9x the 114 of 2,016 the single-marker, +#: single-policy grid held -- three markers x three policies, and the +#: class is indifferent to both, which is the finding. (Before #533 +#: the same grid had 186 allowlisted and 984 failing.) +_MAIDEN_AGREEMENT_EXCEPTIONS = 1026 +_MAIDEN_AGREEMENT_DIGEST = ( + "4b70727a2633fea1a9c219173d48b223cc0866a5d340b69a9eb4f14fc386ae6a") def _one_case(text: str) -> bool | None: @@ -295,9 +306,16 @@ def test_a_maiden_clause_does_not_change_how_a_trailing_word_reads( "Smith, Jane", "Jane Doe Jr.") bodies = ("Smith", "Yo-Yo", "van der Berg", "Jones Smith", "MA", "Ma") - parser = Parser() - - def side(text: str, word: str) -> str: + # three markers and the two 2.4 switches beside the default: the + # switches change WHICH tokens are in the class, and the marker + # spellings are what `own_words` stops at, so both are dimensions + # the allowlist's structural argument rests on. + markers = ("nee", "n\u00e9e", "geb.") + policies = (("default", Policy()), + ("caps", Policy(unlisted_caps_suffixes=True)), + ("nodot", Policy(unlisted_dotted_suffixes=False))) + + def side(parser: Parser, text: str, word: str) -> str: name = parser.parse(text) hits = [t for t in name.tokens if t.text == word] if not hits: @@ -305,31 +323,171 @@ def side(text: str, word: str) -> str: return ("credential" if hits[-1].role is Role.SUFFIX else "name") - pairs = allowed = 0 + pairs = 0 + allowed: list[str] = [] failures = [] - for head in heads: - for body in bodies: - for base in members: - for word in (base.lower(), base.title(), base.upper()): - clause = f"{head} nee {body} {word}" - plain = f"{head} {word}" - pairs += 1 - if side(clause, word) == side(plain, word): - continue - if _one_case(clause) and not _one_case(plain): - allowed += 1 - continue - failures.append( - f"{clause!r} reads {side(clause, word)} but " - f"{plain!r} reads {side(plain, word)}") + for label, policy in policies: + parser = Parser(policy=policy) + for head in heads: + for body in bodies: + for base in members: + for word in (base.lower(), base.title(), + base.upper()): + for marker in markers: + clause = f"{head} {marker} {body} {word}" + plain = f"{head} {word}" + pairs += 1 + if side(parser, clause, word) == side( + parser, plain, word): + continue + if _one_case(clause) and not _one_case(plain): + allowed.append( + f"{label}|{marker}|{clause}") + continue + failures.append( + f"[{label}] {clause!r} reads " + f"{side(parser, clause, word)} but " + f"{plain!r} reads " + f"{side(parser, plain, word)}") assert not failures, ( f"{len(failures)} of {pairs} pair(s) disagree outside the " f"one-case-head class:\n" + "\n".join(failures[:15])) - assert allowed == _MAIDEN_AGREEMENT_EXCEPTIONS, ( - f"the one-case-head class holds {allowed} of {pairs} pairs, " - f"recorded as {_MAIDEN_AGREEMENT_EXCEPTIONS} on 2026-09-19; a " - f"structural allowlist cannot notice its own growth, so this " - f"count is the control. Re-record it deliberately, saying why") + digest = hashlib.sha256( + "\n".join(sorted(allowed)).encode()).hexdigest() + assert (len(allowed), digest) == ( + _MAIDEN_AGREEMENT_EXCEPTIONS, _MAIDEN_AGREEMENT_DIGEST), ( + f"the one-case-head class holds {len(allowed)} of {pairs} " + f"pairs with digest {digest}, recorded as " + f"{_MAIDEN_AGREEMENT_EXCEPTIONS} / " + f"{_MAIDEN_AGREEMENT_DIGEST} on 2026-09-19. The SET is " + f"pinned, not only the size: a swap keeps the count. " + f"Re-record both deliberately, saying why. Members:\n" + + "\n".join(sorted(allowed)[:10])) + + +#: The tags that make a token a member of the ambiguous credential +#: class as the reader sees it -- the listed one and the by-shape one +#: a 2.4 switch writes. Spelled here rather than imported so the +#: property is stated in the terms rules.md#M2 states it in, and so a +#: rename in the pipeline cannot quietly narrow what this checks. +_CLASS_TAGS = frozenset({"vocab:suffix-ambiguous", "shape:acronym"}) + +#: Two-member trailing runs: the one shape that puts a maiden-clause +#: report and an assign-peel report on the same parse, each naming a +#: different token. Written out rather than generated, because what +#: makes them work is the CONTRAST between the two members' writing. +_TAILS = ("Ma JD", "MA JD", "MA Ma") + + +def _maiden_clause_grid() -> list[tuple[str, Parser, str]]: + """(name, parser, policy label) for the M2 release grid. + + Rich enough to hold every shape the #533 review found: title-led + and post-nominal-led comma heads, particle heads, a bound-given + head, by-shape and caps-on members, two adjacent particle members, + mixed/ALL-CAPS/lower writing, comma and no-comma, four markers, + and the default policy beside each 2.4 switch. + """ + heads = ("Jane Doe", "Doe, Jane", "Doe, Prof.", "Doe, Dr.", + "Jane Doe, Jr", "Jane Doe, PhD", "Doe, J.", "Doe, PhD", + "Berg, Jane van der", "Jane van der Berg", "Berg, abdul", + "abdul Berg", "J. Doe", "Doe", "Prof. Jane Doe", + "Doe, Jane van der", "Doe, Sir") + bodies = ("Smith", "Smith MA", "Smith Ma", "Smith ma", "Smith A.B.", + "Smith X.Y.Z.", "Smith XYZ", "Smith ba", "Smith DO", + "Smith Do", "Smith do", "Smith MA XYZ", "Smith DO DO", + "Smith Ma JD", "Smith MA JD", "Smith MA Ma", + "Smith V MA", "MA", "MA PhD", "MA ba", "Smith Jones MA", + "Smith PhD", "Smith Jr", "Jones Smith Ma", "Smith MA PhD") + policies = (("default", Policy()), + ("caps", Policy(unlisted_caps_suffixes=True)), + ("nodot", Policy(unlisted_dotted_suffixes=False))) + parsers = [(label, Parser(policy=p)) for label, p in policies] + texts: list[str] = [] + seen: set[str] = set() + for head in heads: + for body in bodies: + markers = ("nee", "née", "geb.", "z domu") if " " not in body \ + else ("nee",) + for marker in markers: + base = f"{head} {marker} {body}" + for text in (base, f"{base}, MD"): + for written in (text, text.upper(), text.lower()): + if written not in seen: + seen.add(written) + texts.append(written) + return [(t, parser, label) for t in texts for label, parser in parsers] + + +def _released_but_not_suffix(text: str, parser: Parser) -> list[str]: + """Words rules.md#M2 says must be SUFFIX-roled and are not. + + The clause's parent-style reach is everything from the marker to + the first CERTAIN suffix word -- suffix vocabulary that is not + initial-shaped, which is where the walk stopped before #533 and + still stops. Inside that reach a class member is either still in + the maiden name or was given up as a credential; any third answer + is a word the clause released into the current name, which is the + failure this property exists for. + """ + marker = _MARKER_RE.search(text) + if marker is None: + return [] + name = parser.parse(text) + if not name.maiden: + return [] + out = [] + for tok in name.tokens: + if tok.span is None or tok.span.start < marker.end(): + continue + if "vocab:suffix" in tok.tags and "initial" not in tok.tags: + break + if (not _CLASS_TAGS.isdisjoint(tok.tags) + and tok.role not in (Role.MAIDEN, Role.SUFFIX)): + out.append(f"{tok.text!r} -> {tok.role.value}") + return out + + +_MARKER_RE = re.compile( + r"(? None: + """rules.md#M2's invariant, over the whole release grid. + + The clause may hand a word to the trailing rule that reads it as + a credential, and it may keep the word. There is no third answer: + a released word that ends the parse in `given`, `middle` or + `family` has crossed from the BIRTH name into the current one, + silently, and that is the class of failure #424 named from the + other direction. + + The grid is the pin, and it is a grid rather than a name list so + that it could fail. At d97d3eb7 -- the commit this review round + started from -- it fails on 790 of its 8,466 parses, 290 distinct + names, covering every shape the review reported: 'Doe, Prof. nee + Smith A.B.' reading given 'A.B.' (32 parses, and 'X.Y.Z.' another + 32), 'DOE, PROF. NEE SMITH MA' reading given 'MA' (206, the + largest class, with 'ba' at 96), 'Berg, Jane van der nee Smith + DO' reading family 'van der DO Berg' (12) and 'Jane Doe nee Smith + DO DO' reading family 'DO DO' (108, two words apiece). At + 2f57ff21, the parent, it passes on all 8,466 -- the invariant is + what the conservative direction always held. + """ + failures = [] + grid = _maiden_clause_grid() + for text, parser, label in grid: + for bad in _released_but_not_suffix(text, parser): + failures.append(f"[{label}] {text!r}: {bad}") + assert not failures, ( + f"{len(failures)} of {len(grid)} parse(s) released a word the " + f"clause should have kept:\n" + "\n".join(failures[:20])) + # the grid has to be able to fail: every shape above must actually + # reach the walk, which it does only where a clause is taken + reached = sum(bool(p.parse(t).maiden) for t, p, _ in grid) + assert reached > len(grid) // 2, ( + f"only {reached} of {len(grid)} grid parses carry a maiden " + f"name; the rest cannot exercise M2 at all") def test_no_two_ambiguities_name_the_same_token_span() -> None: @@ -346,13 +504,25 @@ def test_no_two_ambiguities_name_the_same_token_span() -> None: spans the reports claim, so an OVERLAP fails it however the two spans differ in length. - Measured 2026-09-19: 0 over 20,412 parses, every particle shape + Measured 2026-09-19: 0 over 27,216 parses, every particle shape among them ('nee van der Berg Ma', 'nee de Ma', 'nee van Ma'). A COUNT of the parses that carry two reports rides along, because a comparison over one report is vacuous and nothing else would - say so: the grid as it stands has 2,160 of them, all from the two - particle heads, and a later edit that drops them fails here - rather than silently turning this into a test of nothing. + say so. + + The count is also what caught this test measuring the wrong + thing. Every one of the 2,160 multi-report parses the grid held + at the review paired the MAIDEN emitter with the particle-chain + emitter -- the pair this test is named for, maiden against + assign's peel, never occurred, because one trailing member is + either kept by the clause or peeled by assign and no grid row had + TWO. The `_TAILS` below are that shape: 'Ma JD' keeps 'Ma' and + peels 'JD', 'MA JD' releases 'MA' and peels 'JD' behind it, and + 'MA Ma' keeps the Title-case one with the caps one in front. The + count is re-pinned on the widened grid, and the pairing is + asserted directly beside it so a later edit that drops the tails + fails here rather than silently turning this back into a test of + the particle emitter. """ members = ("ba", "do", "ed", "jd", "ma", "x.y.z.", "r.a.i.") heads = ("Jane Doe", "Doe, Jane", "John", "J.", "Dr.", "Jane", @@ -390,14 +560,46 @@ def test_no_two_ambiguities_name_the_same_token_span() -> None: claimed |= span if overlap: failures.append(f"{text!r}: {spans}") + # the tails that put the maiden emitter and assign's peel on one + # parse -- the pair this test is named for, which the grid above + # cannot produce (see the docstring) + both = 0 + for head in heads: + for body in bodies: + for tail in _TAILS: + for marker in ("nee", "née", "geb."): + text = f"{head} {marker} {body} {tail}" + for parser in parsers: + claimed = set() + overlap = False + spans = [] + reports = parser.parse(text).ambiguities + multi += len(reports) > 1 + kinds = [a.kind for a in reports] + both += (kinds.count( + AmbiguityKind.SUFFIX_OR_NAME) > 1) + for a in reports: + span = {t.span if t.span is not None + else id(t) for t in a.tokens} + spans.append(sorted(map(str, span))) + overlap = overlap or bool(claimed & span) + claimed |= span + if overlap: + failures.append(f"{text!r}: {spans}") assert not failures, ( f"{len(failures)} parse(s) report one token twice:\n" + "\n".join(failures[:15])) - assert multi == 2160, ( + assert multi == 4320, ( f"{multi} of these parses carry more than one report, recorded " - f"as 2160 on 2026-09-19. A parse with one report cannot fail " + f"as 4320 on 2026-09-19. A parse with one report cannot fail " f"the check above, so this is what keeps the grid honest: move " f"the number deliberately, and never to 0") + assert both == 1944, ( + f"{both} parse(s) carry TWO suffix-or-name reports, recorded " + f"as 1944 on 2026-09-19. This is the pair the test is named " + f"for -- the maiden emitter against assign's trailing peel -- " + f"and it was 0 for the whole grid until the two-member tails " + f"were added. Never re-record it as 0") @pytest.mark.parametrize("text", _FORK_CORPUS) diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index 9bab66b5..35cb26e8 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -29,10 +29,12 @@ "Berg, Jan vd" "Berg, abd née Jones" "Berg, abdul V" +"Berg, abdul nee Jones MA" "Berg, abdul van" "Berg, abdul vd" "Davis Royce, Ed" "Del Toro" +"Doe, Dr. nee Smith MA" "Doe, John DO" "Doe, John DO Ed" "Doe, John Do" @@ -85,8 +87,12 @@ "Jack Wei Ma" "Jack X.Y.I." "Jane (née Jones) Smith" +"Jane Doe (nee Smith MA)" +"Jane Doe (nee Smith Ma)" +"Jane Doe (nee Smith) MA" "Jane Doe nee MA" "Jane Doe nee MA Smith" +"Jane Doe nee Smith DO DO" "Jane Doe nee Smith MA" "Jane Doe nee Smith MA Prof." "Jane Doe nee Smith Ma" diff --git a/tools/differential/corpus_shapes.jsonl b/tools/differential/corpus_shapes.jsonl index b161642e..0f8bd789 100644 --- a/tools/differential/corpus_shapes.jsonl +++ b/tools/differential/corpus_shapes.jsonl @@ -90,6 +90,7 @@ {"name": "محمد و علي", "shape": 1} {"name": "Beethoven, Ludwig van", "shape": 2} {"name": "Berg, Jan mc", "shape": 2} +{"name": "Berg, Jane van der nee Smith DO", "shape": 2} {"name": "Berg, abdul MA", "shape": 2} {"name": "Berg, abdul nee Jones MA", "shape": 2} {"name": "DOE, JOHN MA", "shape": 2} @@ -103,6 +104,7 @@ {"name": "Doe, J. nee MA ba", "shape": 2} {"name": "Doe, Jane Q. nee Smith MA", "shape": 2} {"name": "Doe, Jane nee Smith DO", "shape": 2} +{"name": "Doe, Jane nee Smith Do", "shape": 2} {"name": "Doe, Jane nee Smith MA", "shape": 2} {"name": "Doe, Jane nee Smith MA do", "shape": 2} {"name": "Doe, Jane nee Smith Ma", "shape": 2} @@ -134,7 +136,15 @@ {"name": "Doe, MA Smith", "shape": 2} {"name": "Doe, Mary Jo Ma", "shape": 2} {"name": "Doe, Mr. MA PhD", "shape": 2} +{"name": "Jane Doe (nee Smith MA)", "shape": 2} +{"name": "Jane Doe (nee Smith Ma)", "shape": 2} +{"name": "Jane Doe (nee Smith) MA", "shape": 2} +{"name": "Jane Doe nee MA PhD", "shape": 2} +{"name": "Jane Doe nee Smith DO DO", "shape": 2} +{"name": "Jane Doe nee Smith Ma JD", "shape": 2} +{"name": "Jane Doe nee Smith V MA", "shape": 2} {"name": "John Smith, 1.4", "shape": 2} +{"name": "John née Jones Smith Ma", "shape": 2} {"name": "Kennedy, John (Jack)", "shape": 2} {"name": "NASCIMENTO, EDSON ARANTES DO", "shape": 2} {"name": "Nascimento, Edson Arantes do", "shape": 2} diff --git a/tools/differential/expected_since_1.4.0.toml b/tools/differential/expected_since_1.4.0.toml index e2c3de99..12901d6a 100644 --- a/tools/differential/expected_since_1.4.0.toml +++ b/tools/differential/expected_since_1.4.0.toml @@ -364,6 +364,110 @@ explains, which is what keeps it live with only 'Doe, Jane nee Smith DO' actually landing here. """ +[[change]] +issue = "fix(#274/#533) the clause keeps the credential a join or a missing reader would have taken" +# Five names where the marker's clause takes words v1 read as a +# middle name and a suffix, and KEEPS the credential at the end of +# them. Against 1.4.0 the visible diff is #274's consumption -- v1 +# had no maiden reading at all, so `given`/`middle`/`family`/`suffix` +# empty as `maiden` fills -- and #533's half is that the last word +# goes INTO `maiden` rather than staying the suffix v1 read. +# +# Each is a route by which rules.md#M2's invariant declines the stop, +# and an earlier round of this branch took all five: +# 'Doe, Dr. nee Smith MA' -- the take would leave 'Dr. MA', a +# segment with no name word for a given part's trailing slot to +# be the end of, so nothing there reads the word as a credential +# 'Berg, abdul nee Jones MA' -- P5's lenient post-comma join would +# swallow the released word into the bound-given pair +# 'Berg, Jane van der nee Smith DO' -- the particle chain the +# marker just bounded would take it back into the family +# 'Jane Doe nee MA PhD' -- the FIRST-WORD FLOOR spares the member +# standing straight after the marker; only the unambiguous +# 'PhD' behind it leaves, which is why `suffix` narrows rather +# than empties here +# 'Doe, Jane nee Smith Do' -- #531's lean declines a Title-cased +# member after a comma +# The first three read as 2f57ff21 read them; the last two are +# #533's own clamp and lean. +# +# `_ambiguities` is NOT declared and cannot be at this baseline: the +# v1 surface has no such field, and the reports these names now +# carry are recorded in the 2.x ledgers where the comparison can see +# them. +name_regex = "^(?:Berg, Jane van der nee Smith DO|Berg, abdul nee Jones MA|Doe, Dr\\. nee Smith MA|Doe, Jane nee Smith Do|Jane Doe nee MA PhD)$" +fields = ["family", "given", "maiden", "middle", "suffix"] + +[[change.precedes_narrower]] +issue = "fix(comma-precomma-family) pre-comma run reads as family, not given" +why = """ +LIVE, and new with the #533 review's corpus rows. Four of the five +names here have a family comma, so the pre-comma rule's {given, +family} is a strict subset of these fields and nothing but file order +keeps them here. + +The discriminator is WHAT MOVED the given name. The precomma rule +describes 2.x splitting a pre-comma run that 1.4 read as one first +name; on these four the pre-comma run is a single word 1.4 and the +tree both read as the family, and what empties `given` is the MARKER +leaving with the word after it -- 'Doe, Dr. nee Smith MA' reads given +'nee' at 1.4.0 because v1 had no marker reading at all. Classified by +the precomma rule instead, each would be attributed to a comma +behavior none of them exercises. +""" + +[[change.precedes_narrower]] +issue = "fix(#411) the bound-given reserve stops counting words the maiden name takes" +why = """ +LIVE, and new with the #533 review's corpus rows. 'Berg, abdul nee +Jones MA' is reached by both and #411's fields are a strict subset. + +The discriminator is that #411's rule is about the RESERVE and this +name's diff is about the CREDENTIAL: the reserve half is real and is +why `given` reads 'abdul' rather than 'abdul nee', but the `suffix` +1.4.0 read is emptied by the clause KEEPING the 'MA', which #411 has +nothing to say about and whose field it does not declare. The review +round is what settled that reading -- an earlier round released the +word into P5's join and read given 'abdul MA' -- so a rule naming +#411 alone would attribute this change's decline to the reserve. +""" + +[[change.precedes_narrower]] +issue = "fix(#379) a tussenvoegsel after a family comma attaches to the family" +why = """ +LIVE, and new with the #533 review's corpus rows. 'Berg, Jane van der +nee Smith DO' and 'Doe, Jane nee Smith Do' each carry a maiden marker +AND a tussenvoegsel-shaped word after a family comma, and the +tussenvoegsel rule's fields are a strict subset of these. + +The discriminator is the one fix(#274)'s own block above this file +gives for the same pair: what 1.4.0 read as middle text plus a +trailing post-nominal, the tree reads as a maiden name, and the +marker leaving with its words is what every field of these diffs is +about. On 'Berg, Jane van der nee Smith DO' the tussenvoegsel is not +even the moved word -- 'van der' attaches to the family either way, +and the 'DO' the clause KEEPS is what empties `suffix`. +""" + +[[change]] +issue = "fix(#335/#533) a marker-led bracketed clause is the maiden name whatever its last word is" +# 'Jane Doe (nee Smith MA)', 'Jane Doe (nee Smith Ma)', 'Jane Doe +# (nee Smith) MA'. The role move is fix(#335)'s, above: 1.4 knew the +# parenthesis pair and had no maiden reading to give the clause, so +# it came back a nickname. +# +# Recorded as their own rule because what #533 does to them is +# NOTHING, and that is worth a row. The delimiters outrank every +# reading inside the span, so the whole span is the maiden name +# whatever its last word is and no fork is called at all +# (decisions.md#M2, rules.md#M2) -- 'Smith MA' and 'Smith Ma' read +# alike where the undelimited spellings do not. 'Jane Doe (nee +# Smith) MA' is the control: the member the writer left OUTSIDE the +# span is an ordinary trailing credential, which v1 also read as a +# suffix, so only the clause's role moves. +name_regex = "^(?:Jane Doe \\(nee Smith MA\\)|Jane Doe \\(nee Smith Ma\\)|Jane Doe \\(nee Smith\\) MA)$" +fields = ["maiden", "nickname"] + [[change]] issue = "fix(cjk-maiden-marker) maiden marker consumed, compounding with the CJK order flip" # Its own rule rather than a widening of fix(#274) above: that rule's @@ -4022,22 +4126,3 @@ issue = "fix(#434/#533) a marker PHRASE takes the maiden name, and its clause en name_regex = "^Maria Kowalska z domu Nowak MA$" fields = ["family", "maiden", "middle"] -[[change]] -issue = "fix(#533) accepted: the released credential lands where the segment it is released into puts it" -# Two names where the member leaves the clause and something other -# than the trailing peel catches it. Both are accepted and both are -# silent -- no report, because the rule that would raise one never -# reads the word. -# -# 'Doe, Dr. nee Smith MA': once 'Smith' leaves with the marker, -# segment 1 is 'Dr. MA' -- a no-name segment the credential-run gate -# reads whole (rules.md#C1) -- so the member is a suffix and the -# given part's own emitter never sees it. v1 read given 'nee', middle -# 'Smith'. 'Berg, abdul nee Jones MA': P5's lenient post-comma join -# takes the released member into the bound-given pair before assign -# can read it, exactly as it does in the clause-less 'Berg, abdul -# MA', so the clause form now AGREES with the bare form -- which is -# what makes this a consequence rather than a defect. v1 read given -# 'abdul nee', middle 'Jones', suffix 'MA'. -name_regex = "^(?:Berg, abdul nee Jones MA|Doe, Dr\\. nee Smith MA)$" -fields = ["given", "maiden", "middle", "suffix"] diff --git a/tools/differential/expected_since_2.0.0.toml b/tools/differential/expected_since_2.0.0.toml index 4ee895a2..768d1761 100644 --- a/tools/differential/expected_since_2.0.0.toml +++ b/tools/differential/expected_since_2.0.0.toml @@ -2735,6 +2735,11 @@ orders = ["DEFAULT"] [[change]] issue = "fix(#533) a credential ending a maiden clause reads as a credential" +# The PR review added one: 'Jane Doe nee Smith V MA', where a +# trailing roman numeral stands between the clause and the member. +# Both stops read the TRAILING word, so the numeral is not the word +# either fork asks about -- the acronym fork stops at 'MA' and the +# 'V' stays maiden text behind it. # The words a maiden marker takes now end where a trailing credential # begins, where the rule that reads the name left standing reads the # word as one -- both as the name is written and as the take would @@ -2769,11 +2774,27 @@ issue = "fix(#533) a credential ending a maiden clause reads as a credential" # Literal-anchored for the reason fix(#531)'s rules give: the subject # is a SLOT, so a regex for it would claim the names whose WRITING # declines the word as readily as the ones it takes. -name_regex = "^(?:Doe, Jane Q\\. nee Smith MA|Doe, Jane nee Smith DO|Doe, Jane nee Smith MA|Doe, Jane nee Smith ma|JANE DOE NEE SMITH MA|JANE DOE NEE YO-YO MA|Jane Doe geb\\. Smith MA|Jane Doe nee Smith MA|Jane Doe nee Smith MA JD|Jane Doe nee Smith MA PhD|Jane Doe nee Smith Ma JD|Jane Doe nee Smith Prof\\. MA|Jane Doe nee Smith do|jane doe nee smith ma)$" +name_regex = "^(?:Doe, Jane Q\\. nee Smith MA|Doe, Jane nee Smith DO|Doe, Jane nee Smith MA|Doe, Jane nee Smith ma|JANE DOE NEE SMITH MA|JANE DOE NEE YO-YO MA|Jane Doe geb\\. Smith MA|Jane Doe nee Smith MA|Jane Doe nee Smith MA JD|Jane Doe nee Smith MA PhD|Jane Doe nee Smith Ma JD|Jane Doe nee Smith Prof\\. MA|Jane Doe nee Smith V MA|Jane Doe nee Smith do|jane doe nee smith ma)$" fields = ["_ambiguities", "maiden", "suffix"] [[change]] issue = "fix(#533) the maiden clause reports the credential it keeps" +# The PR review added more, and they are the same declining half +# reached by four further routes -- each a shape where the clause +# CANNOT promise the word a credential reading, so rules.md#M2 keeps +# it. No name word is left AHEAD of the member for a given part's +# trailing slot to be the end of ('Doe, Dr. nee Smith MA'); a +# particle chain below the marker pass would absorb it into the +# family ('Berg, Jane van der nee Smith DO', and 'Jane Doe nee Smith +# DO DO' where the other released member is what would do the +# joining); P5's bound-given join would swallow it into the pair +# ('Berg, abdul nee Jones MA'); the FIRST-WORD FLOOR spares it and +# only an unambiguous credential behind it leaves ('Jane Doe nee MA +# PhD'); and the lean declines it after a comma ('Doe, Jane nee +# Smith Do'). Those first four READ as 2f57ff21 read them and gain +# only the report: an earlier round of this branch released all four +# and moved them in silence, which is what the review measured and +# reversed. # The other direction of the same fork, and the whole of what these # five names do at this baseline: no role moves and the decision # starts being REPORTED. A Title-cased member in a mixed-case name is @@ -2790,9 +2811,70 @@ issue = "fix(#533) the maiden clause reports the credential it keeps" # rule of its own rather than a widening of the one above: `fields` # is `_ambiguities` alone, so it cannot absorb a role diff on any of # the five. -name_regex = "^(?:Doe, J\\. nee MA ba|Doe, Jane nee Smith Ma|Jane Doe nee MA|Jane Doe nee Smith Ma|Jane Doe nee Yo-Yo Ma)$" +name_regex = "^(?:Doe, Dr\\. nee Smith MA|Doe, J\\. nee MA ba|Doe, Jane nee Smith Ma|Jane Doe nee MA|Jane Doe nee MA PhD|Jane Doe nee Smith DO DO|Jane Doe nee Smith Ma|Jane Doe nee Yo-Yo Ma)$" fields = ["_ambiguities"] +[[change]] +issue = "fix(#411/#533) the bound-given pair after a comma, and the clause it keeps its credential in" +# 'Berg, abdul nee Jones MA'. Two changes in one name and neither is +# enough on its own, which is why it is a rule rather than an +# alternative in either. The clause consumption (#274) and the +# reserve that stops counting the words it takes (#411) are what move +# `given`, `middle` and `maiden` against this baseline; the report is +# #533's. +# +# What #533 does here is NOT a role move, and the comment is the only +# place that can say so: an earlier round of this branch released the +# 'MA' from the clause and let P5's lenient post-comma join swallow +# it into the pair, reading given 'abdul MA' in silence. The review +# measured that as M2's invariant broken -- a word the clause gave up +# landing in a NAME part -- so the stop is declined, the clause keeps +# 'Jones MA', and the fork is reported instead. +name_regex = "^Berg, abdul nee Jones MA$" +fields = ["_ambiguities", "given", "maiden", "middle"] + +[[change]] +issue = "fix(#399/#533) a maiden marker bounds the particle chain, and the clause keeps the credential the chain would have taken" +# 'Berg, Jane van der nee Smith DO'. The marker bounds the particle +# join arriving from its left (#399, rules.md#M2), which is what +# moves `middle`, `family` and `maiden` against this baseline, and +# the chain still reports its own PARTICLE_OR_GIVEN. +# +# The SUFFIX_OR_NAME beside it is #533's, and it is a report without +# a role move on purpose. 'DO' is particle vocabulary, so releasing +# it from the clause would put it back in reach of the very chain the +# marker just bounded -- family 'van der DO Berg', a word crossing +# from the birth name into the current one, which an earlier round of +# this branch did in silence. rules.md#M2's invariant declines the +# stop, the clause keeps 'Smith DO', and the fork is reported. +name_regex = "^Berg, Jane van der nee Smith DO$" +fields = ["_ambiguities", "family", "maiden", "middle"] + +[[change]] +issue = "fix(#335/#533) a marker-led bracketed clause is the maiden name whatever its last word is" +# 'Jane Doe (nee Smith MA)', 'Jane Doe (nee Smith Ma)', 'Jane Doe +# (nee Smith) MA'. The role move is fix(#335)'s, one rule above: this +# baseline read the parenthesis pair as a nickname delimiter and +# looked no further. These three are separate because the FIRST TWO +# also lose a report -- the clause came back a nickname ending in a +# credential-class word, so the nickname reading raised its own +# SUFFIX_OR_NICKNAME, and consuming the clause as a maiden name +# retires it. +# +# They are here rather than as three more alternatives up there for +# the reason that rule's own comment gives about declaring a field a +# diff does not move: adding `_ambiguities` to it would let a report +# change hide on the five Latin names it already holds. +# +# What #533 does to them is NOTHING, and that is the point of +# recording them: the delimiters outrank every reading inside the +# span, so the whole span is the maiden name whatever its last word +# is and no fork is called (decisions.md#M2, rules.md#M2). 'Jane Doe +# (nee Smith) MA' is the control -- the member the writer left +# OUTSIDE the span is an ordinary trailing credential and reports. +name_regex = "^(?:Jane Doe \\(nee Smith MA\\)|Jane Doe \\(nee Smith Ma\\)|Jane Doe \\(nee Smith\\) MA)$" +fields = ["_ambiguities", "maiden", "nickname"] + [[change]] issue = "fix(#533) accepted: an unlisted dotted acronym ending a maiden clause is read by position" # 'Jane Doe nee Smith X.Y.Z.'. A by-shape member carries no writing @@ -2810,23 +2892,6 @@ issue = "fix(#533) accepted: an unlisted dotted acronym ending a maiden clause i name_regex = "^Jane Doe nee Smith X\\.Y\\.Z\\.$" fields = ["_ambiguities", "maiden", "suffix"] -[[change]] -issue = "fix(#533) accepted: the released credential lands where the segment it is released into puts it" -# 'Doe, Dr. nee Smith MA', and it is SILENT, which is why it is not -# with the movers above: a rule declaring `_ambiguities` over a name -# that gains no report is the over-declaration #452's check names. -# Once 'Smith' leaves with the marker, segment 1 is 'Dr. MA' -- a -# no-name segment the credential-run gate reads whole (rules.md#C1) -# -- so the member is a suffix and the given part's own emitter never -# sees it. -# -# 'Berg, abdul nee Jones MA' is the other name of this shape and is -# NOT here: from this baseline its diff is P5's, and fix(#411) above -# carries it. It joins this rule at 2.2.0 and 2.3.0, where the -# reserve's own change has already shipped. -name_regex = "^Doe, Dr\\. nee Smith MA$" -fields = ["maiden", "suffix"] - [[change]] issue = "fix(#533) restores the suffix reading a 2.4 retag had moved into the maiden name" # 'John Smith nee Jones R.A.I.', and it is a RESTORATION rather than diff --git a/tools/differential/expected_since_2.1.0.toml b/tools/differential/expected_since_2.1.0.toml index c76311e1..8370d597 100644 --- a/tools/differential/expected_since_2.1.0.toml +++ b/tools/differential/expected_since_2.1.0.toml @@ -2622,6 +2622,11 @@ orders = ["DEFAULT"] [[change]] issue = "fix(#533) a credential ending a maiden clause reads as a credential" +# The PR review added one: 'Jane Doe nee Smith V MA', where a +# trailing roman numeral stands between the clause and the member. +# Both stops read the TRAILING word, so the numeral is not the word +# either fork asks about -- the acronym fork stops at 'MA' and the +# 'V' stays maiden text behind it. # The words a maiden marker takes now end where a trailing credential # begins, where the rule that reads the name left standing reads the # word as one -- both as the name is written and as the take would @@ -2656,11 +2661,27 @@ issue = "fix(#533) a credential ending a maiden clause reads as a credential" # Literal-anchored for the reason fix(#531)'s rules give: the subject # is a SLOT, so a regex for it would claim the names whose WRITING # declines the word as readily as the ones it takes. -name_regex = "^(?:Doe, Jane Q\\. nee Smith MA|Doe, Jane nee Smith DO|Doe, Jane nee Smith MA|Doe, Jane nee Smith ma|JANE DOE NEE SMITH MA|JANE DOE NEE YO-YO MA|Jane Doe geb\\. Smith MA|Jane Doe nee Smith MA|Jane Doe nee Smith MA JD|Jane Doe nee Smith MA PhD|Jane Doe nee Smith Ma JD|Jane Doe nee Smith Prof\\. MA|Jane Doe nee Smith do|jane doe nee smith ma)$" +name_regex = "^(?:Doe, Jane Q\\. nee Smith MA|Doe, Jane nee Smith DO|Doe, Jane nee Smith MA|Doe, Jane nee Smith ma|JANE DOE NEE SMITH MA|JANE DOE NEE YO-YO MA|Jane Doe geb\\. Smith MA|Jane Doe nee Smith MA|Jane Doe nee Smith MA JD|Jane Doe nee Smith MA PhD|Jane Doe nee Smith Ma JD|Jane Doe nee Smith Prof\\. MA|Jane Doe nee Smith V MA|Jane Doe nee Smith do|jane doe nee smith ma)$" fields = ["_ambiguities", "maiden", "suffix"] [[change]] issue = "fix(#533) the maiden clause reports the credential it keeps" +# The PR review added more, and they are the same declining half +# reached by four further routes -- each a shape where the clause +# CANNOT promise the word a credential reading, so rules.md#M2 keeps +# it. No name word is left AHEAD of the member for a given part's +# trailing slot to be the end of ('Doe, Dr. nee Smith MA'); a +# particle chain below the marker pass would absorb it into the +# family ('Berg, Jane van der nee Smith DO', and 'Jane Doe nee Smith +# DO DO' where the other released member is what would do the +# joining); P5's bound-given join would swallow it into the pair +# ('Berg, abdul nee Jones MA'); the FIRST-WORD FLOOR spares it and +# only an unambiguous credential behind it leaves ('Jane Doe nee MA +# PhD'); and the lean declines it after a comma ('Doe, Jane nee +# Smith Do'). Those first four READ as 2f57ff21 read them and gain +# only the report: an earlier round of this branch released all four +# and moved them in silence, which is what the review measured and +# reversed. # The other direction of the same fork, and the whole of what these # five names do at this baseline: no role moves and the decision # starts being REPORTED. A Title-cased member in a mixed-case name is @@ -2677,9 +2698,70 @@ issue = "fix(#533) the maiden clause reports the credential it keeps" # rule of its own rather than a widening of the one above: `fields` # is `_ambiguities` alone, so it cannot absorb a role diff on any of # the five. -name_regex = "^(?:Doe, J\\. nee MA ba|Doe, Jane nee Smith Ma|Jane Doe nee MA|Jane Doe nee Smith Ma|Jane Doe nee Yo-Yo Ma)$" +name_regex = "^(?:Doe, Dr\\. nee Smith MA|Doe, J\\. nee MA ba|Doe, Jane nee Smith Ma|Jane Doe nee MA|Jane Doe nee MA PhD|Jane Doe nee Smith DO DO|Jane Doe nee Smith Ma|Jane Doe nee Yo-Yo Ma)$" fields = ["_ambiguities"] +[[change]] +issue = "fix(#411/#533) the bound-given pair after a comma, and the clause it keeps its credential in" +# 'Berg, abdul nee Jones MA'. Two changes in one name and neither is +# enough on its own, which is why it is a rule rather than an +# alternative in either. The clause consumption (#274) and the +# reserve that stops counting the words it takes (#411) are what move +# `given`, `middle` and `maiden` against this baseline; the report is +# #533's. +# +# What #533 does here is NOT a role move, and the comment is the only +# place that can say so: an earlier round of this branch released the +# 'MA' from the clause and let P5's lenient post-comma join swallow +# it into the pair, reading given 'abdul MA' in silence. The review +# measured that as M2's invariant broken -- a word the clause gave up +# landing in a NAME part -- so the stop is declined, the clause keeps +# 'Jones MA', and the fork is reported instead. +name_regex = "^Berg, abdul nee Jones MA$" +fields = ["_ambiguities", "given", "maiden", "middle"] + +[[change]] +issue = "fix(#399/#533) a maiden marker bounds the particle chain, and the clause keeps the credential the chain would have taken" +# 'Berg, Jane van der nee Smith DO'. The marker bounds the particle +# join arriving from its left (#399, rules.md#M2), which is what +# moves `middle`, `family` and `maiden` against this baseline, and +# the chain still reports its own PARTICLE_OR_GIVEN. +# +# The SUFFIX_OR_NAME beside it is #533's, and it is a report without +# a role move on purpose. 'DO' is particle vocabulary, so releasing +# it from the clause would put it back in reach of the very chain the +# marker just bounded -- family 'van der DO Berg', a word crossing +# from the birth name into the current one, which an earlier round of +# this branch did in silence. rules.md#M2's invariant declines the +# stop, the clause keeps 'Smith DO', and the fork is reported. +name_regex = "^Berg, Jane van der nee Smith DO$" +fields = ["_ambiguities", "family", "maiden", "middle"] + +[[change]] +issue = "fix(#335/#533) a marker-led bracketed clause is the maiden name whatever its last word is" +# 'Jane Doe (nee Smith MA)', 'Jane Doe (nee Smith Ma)', 'Jane Doe +# (nee Smith) MA'. The role move is fix(#335)'s, one rule above: this +# baseline read the parenthesis pair as a nickname delimiter and +# looked no further. These three are separate because the FIRST TWO +# also lose a report -- the clause came back a nickname ending in a +# credential-class word, so the nickname reading raised its own +# SUFFIX_OR_NICKNAME, and consuming the clause as a maiden name +# retires it. +# +# They are here rather than as three more alternatives up there for +# the reason that rule's own comment gives about declaring a field a +# diff does not move: adding `_ambiguities` to it would let a report +# change hide on the five Latin names it already holds. +# +# What #533 does to them is NOTHING, and that is the point of +# recording them: the delimiters outrank every reading inside the +# span, so the whole span is the maiden name whatever its last word +# is and no fork is called (decisions.md#M2, rules.md#M2). 'Jane Doe +# (nee Smith) MA' is the control -- the member the writer left +# OUTSIDE the span is an ordinary trailing credential and reports. +name_regex = "^(?:Jane Doe \\(nee Smith MA\\)|Jane Doe \\(nee Smith Ma\\)|Jane Doe \\(nee Smith\\) MA)$" +fields = ["_ambiguities", "maiden", "nickname"] + [[change]] issue = "fix(#533) accepted: an unlisted dotted acronym ending a maiden clause is read by position" # 'Jane Doe nee Smith X.Y.Z.'. A by-shape member carries no writing @@ -2697,23 +2779,6 @@ issue = "fix(#533) accepted: an unlisted dotted acronym ending a maiden clause i name_regex = "^Jane Doe nee Smith X\\.Y\\.Z\\.$" fields = ["_ambiguities", "maiden", "suffix"] -[[change]] -issue = "fix(#533) accepted: the released credential lands where the segment it is released into puts it" -# 'Doe, Dr. nee Smith MA', and it is SILENT, which is why it is not -# with the movers above: a rule declaring `_ambiguities` over a name -# that gains no report is the over-declaration #452's check names. -# Once 'Smith' leaves with the marker, segment 1 is 'Dr. MA' -- a -# no-name segment the credential-run gate reads whole (rules.md#C1) -# -- so the member is a suffix and the given part's own emitter never -# sees it. -# -# 'Berg, abdul nee Jones MA' is the other name of this shape and is -# NOT here: from this baseline its diff is P5's, and fix(#411) above -# carries it. It joins this rule at 2.2.0 and 2.3.0, where the -# reserve's own change has already shipped. -name_regex = "^Doe, Dr\\. nee Smith MA$" -fields = ["maiden", "suffix"] - [[change]] issue = "fix(#533) restores the suffix reading a 2.4 retag had moved into the maiden name" # 'John Smith nee Jones R.A.I.', and it is a RESTORATION rather than diff --git a/tools/differential/expected_since_2.2.0.toml b/tools/differential/expected_since_2.2.0.toml index 8b399531..d74d9755 100644 --- a/tools/differential/expected_since_2.2.0.toml +++ b/tools/differential/expected_since_2.2.0.toml @@ -1202,6 +1202,11 @@ orders = ["DEFAULT"] [[change]] issue = "fix(#533) a credential ending a maiden clause reads as a credential" +# The PR review added one: 'Jane Doe nee Smith V MA', where a +# trailing roman numeral stands between the clause and the member. +# Both stops read the TRAILING word, so the numeral is not the word +# either fork asks about -- the acronym fork stops at 'MA' and the +# 'V' stays maiden text behind it. # The words a maiden marker takes now end where a trailing credential # begins, where the rule that reads the name left standing reads the # word as one -- both as the name is written and as the take would @@ -1237,11 +1242,27 @@ issue = "fix(#533) a credential ending a maiden clause reads as a credential" # would claim the names whose WRITING declines the word -- which keep # their maiden reading and have the rule below -- as readily as the # ones it takes. _MUST_NOT_MATCH carries both directions. -name_regex = "^(?:Doe, J\\. nee MA ba|Doe, Jane Q\\. nee Smith MA|Doe, Jane nee Smith DO|Doe, Jane nee Smith MA|Doe, Jane nee Smith ma|JANE DOE NEE SMITH MA|JANE DOE NEE YO-YO MA|Jane Doe geb\\. Smith MA|Jane Doe nee Smith MA|Jane Doe nee Smith MA JD|Jane Doe nee Smith MA PhD|Jane Doe nee Smith Ma JD|Jane Doe nee Smith Prof\\. MA|Jane Doe nee Smith do|John née Jones Smith MA|Maria Kowalska z domu Nowak MA|jane doe nee smith ma)$" +name_regex = "^(?:Doe, J\\. nee MA ba|Doe, Jane Q\\. nee Smith MA|Doe, Jane nee Smith DO|Doe, Jane nee Smith MA|Doe, Jane nee Smith ma|JANE DOE NEE SMITH MA|JANE DOE NEE YO-YO MA|Jane Doe geb\\. Smith MA|Jane Doe nee Smith MA|Jane Doe nee Smith MA JD|Jane Doe nee Smith MA PhD|Jane Doe nee Smith Ma JD|Jane Doe nee Smith Prof\\. MA|Jane Doe nee Smith V MA|Jane Doe nee Smith do|John née Jones Smith MA|Maria Kowalska z domu Nowak MA|jane doe nee smith ma)$" fields = ["_ambiguities", "maiden", "suffix"] [[change]] issue = "fix(#533) the maiden clause reports the credential it keeps" +# The PR review added six more, and they are the same declining half +# reached by four further routes -- each one a shape where the clause +# CANNOT promise the word a credential reading, so rules.md#M2 keeps +# it. No name word is left AHEAD of the member for a given part's +# trailing slot to be the end of ('Doe, Dr. nee Smith MA'); a +# particle chain below the marker pass would absorb it into the +# family ('Berg, Jane van der nee Smith DO', and 'Jane Doe nee Smith +# DO DO' where the other released member is what would do the +# joining); P5's bound-given join would swallow it into the pair +# ('Berg, abdul nee Jones MA'); the FIRST-WORD FLOOR spares it and +# only an unambiguous credential behind it leaves ('Jane Doe nee MA +# PhD'); and the lean declines it after a comma ('Doe, Jane nee +# Smith Do'). The first four READ as 2f57ff21 read them and gain +# only the report: an earlier round of this branch released all four +# and moved them in silence, which is what the review measured and +# reversed. # The other direction of the same fork, and the whole of what these # eight names do at this baseline: no role moves and the decision # starts being REPORTED. A Title-cased member in a mixed-case name is @@ -1264,7 +1285,7 @@ issue = "fix(#533) the maiden clause reports the credential it keeps" # # Literal-anchored: the class is the slot's declining half, and a # regex for it would claim the seventeen movers above. -name_regex = "^(?:Doe, Jane nee Smith MA do|Doe, Jane nee Smith Ma|Doe, Jane nee Smith do|JOHN NEE JONES SMITH MA PHD|Jane Doe nee MA|Jane Doe nee Smith Ma|Jane Doe nee Yo-Yo Ma|John née Jones Smith Ma)$" +name_regex = "^(?:Berg, Jane van der nee Smith DO|Berg, abdul nee Jones MA|Doe, Dr\\. nee Smith MA|Doe, Jane nee Smith Do|Doe, Jane nee Smith MA do|Doe, Jane nee Smith Ma|Doe, Jane nee Smith do|JOHN NEE JONES SMITH MA PHD|Jane Doe nee MA|Jane Doe nee MA PhD|Jane Doe nee Smith DO DO|Jane Doe nee Smith Ma|Jane Doe nee Yo-Yo Ma|John née Jones Smith Ma)$" fields = ["_ambiguities"] [[change]] @@ -1284,26 +1305,6 @@ issue = "fix(#533) accepted: an unlisted dotted acronym ending a maiden clause i name_regex = "^Jane Doe nee Smith X\\.Y\\.Z\\.$" fields = ["_ambiguities", "maiden", "suffix"] -[[change]] -issue = "fix(#533) accepted: the released credential lands where the segment it is released into puts it" -# Two names where the member leaves the clause and something other -# than the trailing peel catches it. Both are SILENT, which is why -# they are not with the movers above: a rule declaring `_ambiguities` -# over a name that gains no report is the over-declaration #452's -# check names. -# -# 'Doe, Dr. nee Smith MA': once 'Smith' leaves with the marker, -# segment 1 is 'Dr. MA' -- a no-name segment the credential-run gate -# reads whole (rules.md#C1) -- so the member is a suffix and the -# given part's own emitter never sees it. 'Berg, abdul nee Jones MA': -# P5's lenient post-comma join takes the released member into the -# bound-given pair before assign can read it, exactly as it does in -# the clause-less 'Berg, abdul MA', so the clause form now AGREES -# with the bare form. That agreement is what makes each a consequence -# of the change rather than a defect in it. -name_regex = "^(?:Berg, abdul nee Jones MA|Doe, Dr\\. nee Smith MA)$" -fields = ["given", "maiden", "suffix"] - [[change]] issue = "fix(#533) restores the suffix reading a 2.4 retag had moved into the maiden name" # 'John Smith nee Jones R.A.I.', and it is a RESTORATION rather than diff --git a/tools/differential/expected_since_2.3.0.toml b/tools/differential/expected_since_2.3.0.toml index a937d30e..4b3368b1 100644 --- a/tools/differential/expected_since_2.3.0.toml +++ b/tools/differential/expected_since_2.3.0.toml @@ -526,6 +526,11 @@ orders = ["DEFAULT"] [[change]] issue = "fix(#533) a credential ending a maiden clause reads as a credential" +# The PR review added one: 'Jane Doe nee Smith V MA', where a +# trailing roman numeral stands between the clause and the member. +# Both stops read the TRAILING word, so the numeral is not the word +# either fork asks about -- the acronym fork stops at 'MA' and the +# 'V' stays maiden text behind it. # The words a maiden marker takes now end where a trailing credential # begins, where the rule that reads the name left standing reads the # word as one -- both as the name is written and as the take would @@ -561,11 +566,27 @@ issue = "fix(#533) a credential ending a maiden clause reads as a credential" # would claim the names whose WRITING declines the word -- which keep # their maiden reading and have the rule below -- as readily as the # ones it takes. _MUST_NOT_MATCH carries both directions. -name_regex = "^(?:Doe, J\\. nee MA ba|Doe, Jane Q\\. nee Smith MA|Doe, Jane nee Smith DO|Doe, Jane nee Smith MA|Doe, Jane nee Smith ma|JANE DOE NEE SMITH MA|JANE DOE NEE YO-YO MA|Jane Doe geb\\. Smith MA|Jane Doe nee Smith MA|Jane Doe nee Smith MA JD|Jane Doe nee Smith MA PhD|Jane Doe nee Smith Ma JD|Jane Doe nee Smith Prof\\. MA|Jane Doe nee Smith do|John née Jones Smith MA|Maria Kowalska z domu Nowak MA|jane doe nee smith ma)$" +name_regex = "^(?:Doe, J\\. nee MA ba|Doe, Jane Q\\. nee Smith MA|Doe, Jane nee Smith DO|Doe, Jane nee Smith MA|Doe, Jane nee Smith ma|JANE DOE NEE SMITH MA|JANE DOE NEE YO-YO MA|Jane Doe geb\\. Smith MA|Jane Doe nee Smith MA|Jane Doe nee Smith MA JD|Jane Doe nee Smith MA PhD|Jane Doe nee Smith Ma JD|Jane Doe nee Smith Prof\\. MA|Jane Doe nee Smith V MA|Jane Doe nee Smith do|John née Jones Smith MA|Maria Kowalska z domu Nowak MA|jane doe nee smith ma)$" fields = ["_ambiguities", "maiden", "suffix"] [[change]] issue = "fix(#533) the maiden clause reports the credential it keeps" +# The PR review added six more, and they are the same declining half +# reached by four further routes -- each one a shape where the clause +# CANNOT promise the word a credential reading, so rules.md#M2 keeps +# it. No name word is left AHEAD of the member for a given part's +# trailing slot to be the end of ('Doe, Dr. nee Smith MA'); a +# particle chain below the marker pass would absorb it into the +# family ('Berg, Jane van der nee Smith DO', and 'Jane Doe nee Smith +# DO DO' where the other released member is what would do the +# joining); P5's bound-given join would swallow it into the pair +# ('Berg, abdul nee Jones MA'); the FIRST-WORD FLOOR spares it and +# only an unambiguous credential behind it leaves ('Jane Doe nee MA +# PhD'); and the lean declines it after a comma ('Doe, Jane nee +# Smith Do'). The first four READ as 2f57ff21 read them and gain +# only the report: an earlier round of this branch released all four +# and moved them in silence, which is what the review measured and +# reversed. # The other direction of the same fork, and the whole of what these # eight names do at this baseline: no role moves and the decision # starts being REPORTED. A Title-cased member in a mixed-case name is @@ -588,7 +609,7 @@ issue = "fix(#533) the maiden clause reports the credential it keeps" # # Literal-anchored: the class is the slot's declining half, and a # regex for it would claim the seventeen movers above. -name_regex = "^(?:Doe, Jane nee Smith MA do|Doe, Jane nee Smith Ma|Doe, Jane nee Smith do|JOHN NEE JONES SMITH MA PHD|Jane Doe nee MA|Jane Doe nee Smith Ma|Jane Doe nee Yo-Yo Ma|John née Jones Smith Ma)$" +name_regex = "^(?:Berg, Jane van der nee Smith DO|Berg, abdul nee Jones MA|Doe, Dr\\. nee Smith MA|Doe, Jane nee Smith Do|Doe, Jane nee Smith MA do|Doe, Jane nee Smith Ma|Doe, Jane nee Smith do|JOHN NEE JONES SMITH MA PHD|Jane Doe nee MA|Jane Doe nee MA PhD|Jane Doe nee Smith DO DO|Jane Doe nee Smith Ma|Jane Doe nee Yo-Yo Ma|John née Jones Smith Ma)$" fields = ["_ambiguities"] [[change]] @@ -608,26 +629,6 @@ issue = "fix(#533) accepted: an unlisted dotted acronym ending a maiden clause i name_regex = "^Jane Doe nee Smith X\\.Y\\.Z\\.$" fields = ["_ambiguities", "maiden", "suffix"] -[[change]] -issue = "fix(#533) accepted: the released credential lands where the segment it is released into puts it" -# Two names where the member leaves the clause and something other -# than the trailing peel catches it. Both are SILENT, which is why -# they are not with the movers above: a rule declaring `_ambiguities` -# over a name that gains no report is the over-declaration #452's -# check names. -# -# 'Doe, Dr. nee Smith MA': once 'Smith' leaves with the marker, -# segment 1 is 'Dr. MA' -- a no-name segment the credential-run gate -# reads whole (rules.md#C1) -- so the member is a suffix and the -# given part's own emitter never sees it. 'Berg, abdul nee Jones MA': -# P5's lenient post-comma join takes the released member into the -# bound-given pair before assign can read it, exactly as it does in -# the clause-less 'Berg, abdul MA', so the clause form now AGREES -# with the bare form. That agreement is what makes each a consequence -# of the change rather than a defect in it. -name_regex = "^(?:Berg, abdul nee Jones MA|Doe, Dr\\. nee Smith MA)$" -fields = ["given", "maiden", "suffix"] - [[change]] issue = "fix(#533) restores the suffix reading a 2.4 retag had moved into the maiden name" # 'John Smith nee Jones R.A.I.', and it is a RESTORATION rather than From d5310b33be631d73a051f3acc53c7149d6a8a8e4 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Sat, 19 Sep 2026 21:58:35 -0700 Subject: [PATCH 5/5] refactor(#533): simplify pass over the maiden walk's decision code No behavior change: every reading over 222,208 names (the corpora, the case rows and 220,686 generated maiden-clause names) under six policies -- 1,333,248 rows, plus the facade -- is byte-identical to b45707af, and the reference frame counts hold at 412 / 449. - _pieces: `trailing_start` loses its `numeral_only` flag. It had one caller, the maiden walk's numeral re-ask, which now asks the `peel_walk` + `peel_trailing` pair and reads `.numeral`, as the walk's first numeral read already did. One frame cheaper on 'John nee Jones Smith V' (371 -> 370); no probe rose. - _group: the maiden emitter binds its token once. - Comments in _group, _pieces and _assign compressed to what the code does, the measurement and the reason; every measurement, population and price kept, the narration of earlier drafts removed. Rejected by measurement: a closure for the left / view / view_tags triple costs a frame on 5 of 12 probes, and the two views are never the same list (they differed 30 of 30 where both run), so there is no "compute once". mechanisms.md#ONE-PREDICATE-PER-QUESTION said `peel_trailing` has no caller outside `_pieces.py`; #533 gave it one, the maiden walk, and the entry now says why that is a reader of the peel rather than a second spelling of it. Co-Authored-By: Claude Fable 5.1 --- docs/design/mechanisms.md | 2 +- nameparser/_pipeline/_assign.py | 14 +-- nameparser/_pipeline/_group.py | 190 ++++++++++++++------------------ nameparser/_pipeline/_pieces.py | 36 +++--- 4 files changed, 106 insertions(+), 136 deletions(-) diff --git a/docs/design/mechanisms.md b/docs/design/mechanisms.md index fafad208..9fa6e560 100644 --- a/docs/design/mechanisms.md +++ b/docs/design/mechanisms.md @@ -55,7 +55,7 @@ Problem shape. "Which stage does X?" — asked before attributing behavior in pr ## ONE-PREDICATE-PER-QUESTION — one predicate answers it, and every other site calls that -Problem shape. Two stages need the same answer about the same input, and the one that does not own the decision is about to test for it. Contract statement. Where two sites ask the same question, exactly one predicate answers it and every other site calls that one — never a condition written to match it. The predicate belongs to the QUESTION, not to whichever stage decides: it may sit in a leaf both stages import, and for the leading-title test it must, since the deciding stage is assign and group cannot import assign. How it works. A hand-written mirror agrees with its original only until one of them moves, and the drift is invisible in both directions: each site keeps passing its own tests while they disagree about an input neither covers. Five instances, every one found as a defect before it was found as a pattern — #319 lifted the wholly-suffix predicate into the vocabulary layer "so the comma decision and the honorific peel's segment test cannot drift apart"; #401/#421 lifted the trailing-numeral fork out of assign so the bound-given reserve stopped carrying a copy, its hand-written mirror having been falsified in review more than once — the lesson recorded there being that what must be mirrored is assign's WALK, not merely its condition; #425 replaced that reserve's hand re-derivation of the trailing peel with one function over the view the join would leave; #424 moved assign's leading-title test down because group's own `title()` does not see H2's unlisted abbreviations, so `Xyz. van Johnson` chained where `Dr. van Johnson` did not; #429 moved the no-name-segment test down because group asked by segment INDEX where assign asks by CONTENT. The destination follows the LAYER, not the topic: a predicate over token text goes to `_vocab`, one over pieces and tags to `_pieces`. Both are leaves the stages sit on. The piece layer got its own module only in #439 — until then those predicates collected in `_group`, not because grouping owned them but because `_assign` imports `_group` and cannot be imported back, so group was the one place both stages could reach; five had accumulated across four PRs before the module existed. Stage order is this mechanism's limit, and it forecloses the alternative: where the reader comes AFTER the decider, record the answer on the state instead — `ParseState.order` is that shape, "Recorded rather than recomputed downstream, because the two can differ" — which is unavailable whenever the EARLIER stage is the one asking. (The concrete assign→group import that forced the `_group` collection is gone since #439; what remains is the ordering it was a symptom of, and tests/v2/test_layering.py is where the leaf's contract is now written down.) The cost is a second evaluation of the same predicate, measured for #429 at 1.2–2.2% of a family-comma parse and 0% of every other; recording that number was the right answer there over plumbing a state field the two sites would not otherwise share. Lives in. nameparser/_pipeline/_vocab.py over text (is_wholly_suffix; is_trailing_numeral_suffix — the #401/#421 instance, whose only caller since #439 is the shared peel rather than a stage; and maiden_marker_run, the #434 instance and the clearest two-stage case, called by classify over token texts and by extract over a clause's whitespace words, with group reading the tags classify recorded because it runs later; and delimiter_cores, the #436/#437 instance, read by group where a tail segment DROPS a configured delimiter core and by post_rules where the suffix view's entry boundary asks whether a dropped token was one, with a third reader inside this same module, is_wholly_suffix, where a configured core counts as suffix-shaped; and in_initialless_script, the #322/#323 instance and the only one here that is a REPERTOIRE test rather than a vocabulary one — the script half of the #320 initial veto, read by is_initial one function away and by _pieces.is_leading_title, so "a script with no initials has no period abbreviations either" is one predicate over _policy._NO_INITIALS rather than a second reading of that table; it lost its leading underscore when the second caller arrived; and caps_shape_candidate, the #516 instance and the newest, called from three sites that each needed the identical question answered — classify's own tag emission, this module's ambiguous_class_candidate, and _segment.py's multi-token run test — where the usual reason for keeping such copies apart (a shared call costing every default-policy parse a frame it cannot use) does not hold, because every one of the three callers' own FIRST conjunct is the caller-configured switch itself, `Policy.unlisted_caps_suffixes`, False by default, so the shared call is never reached at the default regardless of how many callers share it (decisions.md#S2)) and nameparser/_pipeline/_pieces.py over pieces: is_suffix_piece, leading_titles and peel_walk are called by both stages, while is_leading_title, is_title_piece and trailing_start are called by group alone (measured 2026-09-06 by call site: `is_leading_title` has no caller in `_assign.py`, which reads `leading_titles` instead — a first draft of this clause listed it among the shared ones) — `trailing_start` being the one to know, since it answers where the trailing run begins and is what P2's chain and M2's walk stop at — and segment_suffix_reading by assign alone since #436/#437, that last one being #430's instance, where THREE readers shared one answer until the render join, group's third, was replaced by a rule over the commas the writer typed (decisions.md#C1, 2026-09-06); it stays where it is, one call site being no reason to move a predicate that two sites will contest again. `trailing_titles` was that last shape for one day (2026-09-08, the #316/#489 bundle, rules.md#H5), and since the /simplify round of 2026-09-09 the SHARED predicate is `tail_reading` instead — the peel-and-chain fixed point that answers where the name pieces end (decisions.md#H5). Assign calls it at its main walk and group's bound-given reserve calls it twice, once per view the join compares, because that reserve reads the name words assign will leave and this walk is half of what leaves them (rules.md#P5; counting a trailing title word among them joined 'Prof. abdul rahman Prof.' where 'Prof. abdul rahman' does not). `peel_trailing` and `trailing_titles` are what that fixed point is BUILT from, and neither is a two-stage question any longer: `peel_trailing` has no caller outside `_pieces.py` at all, `trailing_start` and `tail_reading` being the only two and both in the leaf, and `trailing_titles` has exactly one, assign's family-comma segment-1 walk, which reads the chain without the re-peel; `_group.py` imports neither. The tail reading is in the leaf rather than inline because each assign site had been given a cheap frame-free gate written to match the walk's own first condition, which is a second implementation of the question and was removed in review; what the leaf costs is one frame per entry point, measured, and the walk's own first test is a compiled regex rather than a call, so an ordinary name pays a match and stops. The reserve's two calls cost the reference name nothing — it never enters that branch, having no bound given word — and the parse and facade frame counts did not move (measured 2026-09-09). Re-measured 2026-09-09 by an AST call-site census over `_pipeline/*.py` — every call node whose callee is one of these names, keyed by module and enclosing function, which is what caught the census claiming a share for `peel_trailing` that the round had just taken away — the rest of it holds unchanged: is_suffix_piece, leading_titles, peel_walk and now tail_reading shared, is_leading_title, is_title_piece and trailing_start group-only — assign still reads `leading_titles` and never `is_leading_title`, which is what keeps H2's shape inference out of the trailing slot. And nameparser/_pipeline/_post_rules.py over a state: suffix_entries, the #511 instance, the R1 entry pass as a function, the one instance living in a stage rather than in a leaf — it is a pass over a whole ParseState and no leaf takes one, and AGENTS.md names it as the exception — run by post_rules last in the stage (through its in-place worker) and by Parser.revise over a sub-parse whose roles it has forced, so a suffix value handed to revise() derives its entries by the rule a whole name uses rather than by a second reading of the value's commas (decisions.md#C1, 2026-09-06 #511). tests/v2/test_layering.py holds each module's contract, and a piece predicate growing a dependency on a STAGE shows up there as a widened entry. Reach for it when. You are about to write a condition that mirrors, matches or "does what X does" — or you find a comment saying one does. Grep for the other site's predicate and call it instead. +Problem shape. Two stages need the same answer about the same input, and the one that does not own the decision is about to test for it. Contract statement. Where two sites ask the same question, exactly one predicate answers it and every other site calls that one — never a condition written to match it. The predicate belongs to the QUESTION, not to whichever stage decides: it may sit in a leaf both stages import, and for the leading-title test it must, since the deciding stage is assign and group cannot import assign. How it works. A hand-written mirror agrees with its original only until one of them moves, and the drift is invisible in both directions: each site keeps passing its own tests while they disagree about an input neither covers. Five instances, every one found as a defect before it was found as a pattern — #319 lifted the wholly-suffix predicate into the vocabulary layer "so the comma decision and the honorific peel's segment test cannot drift apart"; #401/#421 lifted the trailing-numeral fork out of assign so the bound-given reserve stopped carrying a copy, its hand-written mirror having been falsified in review more than once — the lesson recorded there being that what must be mirrored is assign's WALK, not merely its condition; #425 replaced that reserve's hand re-derivation of the trailing peel with one function over the view the join would leave; #424 moved assign's leading-title test down because group's own `title()` does not see H2's unlisted abbreviations, so `Xyz. van Johnson` chained where `Dr. van Johnson` did not; #429 moved the no-name-segment test down because group asked by segment INDEX where assign asks by CONTENT. The destination follows the LAYER, not the topic: a predicate over token text goes to `_vocab`, one over pieces and tags to `_pieces`. Both are leaves the stages sit on. The piece layer got its own module only in #439 — until then those predicates collected in `_group`, not because grouping owned them but because `_assign` imports `_group` and cannot be imported back, so group was the one place both stages could reach; five had accumulated across four PRs before the module existed. Stage order is this mechanism's limit, and it forecloses the alternative: where the reader comes AFTER the decider, record the answer on the state instead — `ParseState.order` is that shape, "Recorded rather than recomputed downstream, because the two can differ" — which is unavailable whenever the EARLIER stage is the one asking. (The concrete assign→group import that forced the `_group` collection is gone since #439; what remains is the ordering it was a symptom of, and tests/v2/test_layering.py is where the leaf's contract is now written down.) The cost is a second evaluation of the same predicate, measured for #429 at 1.2–2.2% of a family-comma parse and 0% of every other; recording that number was the right answer there over plumbing a state field the two sites would not otherwise share. Lives in. nameparser/_pipeline/_vocab.py over text (is_wholly_suffix; is_trailing_numeral_suffix — the #401/#421 instance, whose only caller since #439 is the shared peel rather than a stage; and maiden_marker_run, the #434 instance and the clearest two-stage case, called by classify over token texts and by extract over a clause's whitespace words, with group reading the tags classify recorded because it runs later; and delimiter_cores, the #436/#437 instance, read by group where a tail segment DROPS a configured delimiter core and by post_rules where the suffix view's entry boundary asks whether a dropped token was one, with a third reader inside this same module, is_wholly_suffix, where a configured core counts as suffix-shaped; and in_initialless_script, the #322/#323 instance and the only one here that is a REPERTOIRE test rather than a vocabulary one — the script half of the #320 initial veto, read by is_initial one function away and by _pieces.is_leading_title, so "a script with no initials has no period abbreviations either" is one predicate over _policy._NO_INITIALS rather than a second reading of that table; it lost its leading underscore when the second caller arrived; and caps_shape_candidate, the #516 instance and the newest, called from three sites that each needed the identical question answered — classify's own tag emission, this module's ambiguous_class_candidate, and _segment.py's multi-token run test — where the usual reason for keeping such copies apart (a shared call costing every default-policy parse a frame it cannot use) does not hold, because every one of the three callers' own FIRST conjunct is the caller-configured switch itself, `Policy.unlisted_caps_suffixes`, False by default, so the shared call is never reached at the default regardless of how many callers share it (decisions.md#S2)) and nameparser/_pipeline/_pieces.py over pieces: is_suffix_piece, leading_titles and peel_walk are called by both stages, while is_leading_title, is_title_piece and trailing_start are called by group alone (measured 2026-09-06 by call site: `is_leading_title` has no caller in `_assign.py`, which reads `leading_titles` instead — a first draft of this clause listed it among the shared ones) — `trailing_start` being the one to know, since it answers where the trailing run begins and is what P2's chain and M2's walk stop at — and segment_suffix_reading by assign alone since #436/#437, that last one being #430's instance, where THREE readers shared one answer until the render join, group's third, was replaced by a rule over the commas the writer typed (decisions.md#C1, 2026-09-06); it stays where it is, one call site being no reason to move a predicate that two sites will contest again. `trailing_titles` was that last shape for one day (2026-09-08, the #316/#489 bundle, rules.md#H5), and since the /simplify round of 2026-09-09 the SHARED predicate is `tail_reading` instead — the peel-and-chain fixed point that answers where the name pieces end (decisions.md#H5). Assign calls it at its main walk and group's bound-given reserve calls it twice, once per view the join compares, because that reserve reads the name words assign will leave and this walk is half of what leaves them (rules.md#P5; counting a trailing title word among them joined 'Prof. abdul rahman Prof.' where 'Prof. abdul rahman' does not). `peel_trailing` and `trailing_titles` are what that fixed point is BUILT from, and neither is a two-stage question any longer: `peel_trailing` has one caller outside `_pieces.py`, the maiden walk in `_group.py`, which asks the peel itself because it needs ONE half of the answer at a time -- the numeral's over the pieces as written and again over the view its take would leave (#424), the acronym's beside it (#533) -- where `trailing_start` and `tail_reading`, the two callers in the leaf, fold both halves into one index; that walk is a reader of the peel and not a second spelling of it, the question being asked of a different name each time. `trailing_titles` has exactly one caller, assign's family-comma segment-1 walk, which reads the chain without the re-peel, and `_group.py` does not import it. The tail reading is in the leaf rather than inline because each assign site had been given a cheap frame-free gate written to match the walk's own first condition, which is a second implementation of the question and was removed in review; what the leaf costs is one frame per entry point, measured, and the walk's own first test is a compiled regex rather than a call, so an ordinary name pays a match and stops. The reserve's two calls cost the reference name nothing — it never enters that branch, having no bound given word — and the parse and facade frame counts did not move (measured 2026-09-09). Re-measured 2026-09-09 by an AST call-site census over `_pipeline/*.py` — every call node whose callee is one of these names, keyed by module and enclosing function, which is what caught the census claiming a share for `peel_trailing` that the round had just taken away — the rest of it holds unchanged: is_suffix_piece, leading_titles, peel_walk and now tail_reading shared, is_leading_title, is_title_piece and trailing_start group-only — assign still reads `leading_titles` and never `is_leading_title`, which is what keeps H2's shape inference out of the trailing slot. And nameparser/_pipeline/_post_rules.py over a state: suffix_entries, the #511 instance, the R1 entry pass as a function, the one instance living in a stage rather than in a leaf — it is a pass over a whole ParseState and no leaf takes one, and AGENTS.md names it as the exception — run by post_rules last in the stage (through its in-place worker) and by Parser.revise over a sub-parse whose roles it has forced, so a suffix value handed to revise() derives its entries by the rule a whole name uses rather than by a second reading of the value's commas (decisions.md#C1, 2026-09-06 #511). tests/v2/test_layering.py holds each module's contract, and a piece predicate growing a dependency on a STAGE shows up there as a widened entry. Reach for it when. You are about to write a condition that mirrors, matches or "does what X does" — or you find a comment saying one does. Grep for the other site's predicate and call it instead. ## RENDER-HONORS-THE-PARSE — the parse decides it, the views honor it diff --git a/nameparser/_pipeline/_assign.py b/nameparser/_pipeline/_assign.py index f77d13a6..90f2e596 100644 --- a/nameparser/_pipeline/_assign.py +++ b/nameparser/_pipeline/_assign.py @@ -823,15 +823,15 @@ def reads_as_a_suffix(m: int, titled: tuple[int, ...]) -> bool: # drift would have been silent -- each # site's own tests would have gone on # passing (mechanisms.md - # #ONE-PREDICATE-PER-QUESTION). It costs - # one frame PER MEMBER asked at this slot, - # not one frame per name -- against + # #ONE-PREDICATE-PER-QUESTION). The call + # costs one frame PER MEMBER asked at this + # slot, not one per name: against # 2f57ff21, 'Doe, John MA' is 310 -> 311 # and 'Doe, John MA Ma MA', which asks - # four times, is 439 -> 443. A name with - # no member here never reaches it and pays - # nothing ('Smith, John' 206, 'MA JD' 185, - # both unchanged). Measured 2026-09-19 per + # four times, 439 -> 443, while a name + # with no member here never reaches it + # ('Smith, John' 206, 'MA JD' 185, both + # unchanged). Measured 2026-09-19 per # `Parser.parse`; Derek took that trade # deliberately. if credential_at_the_given_slot( diff --git a/nameparser/_pipeline/_group.py b/nameparser/_pipeline/_group.py index 70e09988..1798a4f4 100644 --- a/nameparser/_pipeline/_group.py +++ b/nameparser/_pipeline/_group.py @@ -357,44 +357,29 @@ def _maiden_take(pieces: Sequence[Sequence[int]], # née PhD' -- nothing after the marker but a suffix, so the marker # stays a word -- as 1.4.0 read it. # - # `one_case` is LIVE at these sites since #533, and it was not - # before: `numeral_only` answered off `peeled.numeral`, and the - # numeral fork is decided before the peel ever reads a lean, so - # the fact reached only the bare-acronym fork -- which that - # reading discarded. The acronym fork is asked now, so the writing - # decides here as it decides at the trailing slot of a name. - # - # Measured 2026-09-19 with a runtime wrapper that forces this - # function's `one_case` argument to None, over the population - # decisions.md#S2's 2026-09-18 recipe names -- the distinct union - # of every `tools/differential/corpus*.jsonl` entry, every - # `tests/v2/cases.py` text, and `tests/test_variations.TEST_NAMES` - # with the three comma permutations that entry names (no-comma, - # family-comma, and suffix-comma where the name has a suffix, - # built off the PARSE as `test_variations_of_TEST_NAMES` builds - # them -- not off a word split, which is what a first draft of - # this comment counted and why it read 2,429 names), empty - # strings dropped, under six policies (the default, both - # family-first orders, strict commas, and each 2.4 switch - # flipped). THE PAIR IS THE FINDING: over the corpus as it stood - # the day before this change it moved 0 parses, and over the - # corpus WITH this change's own rows it moves 36 of 10,752 (1,792 - # names), on 6 distinct names ('Doe, Jane nee Smith DO', 'Doe, - # Jane nee Smith Ma', 'Jane Doe nee Smith Ma', 'Jane Doe nee Smith - # Ma JD', 'Jane Doe nee Yo-Yo Ma', 'John née Jones Smith MA'). The - # plumbing was live either way; the corpus simply held no name - # that could show it, which is the blindness mechanisms.md's - # corpus field note asks to be measured before any "N names move" - # is written down. The 2026-09-18 record of 0 of 9,852 under the - # numeral-only reading stands as what was true then and is - # superseded here. + # `one_case` is LIVE at these sites since #533 and was not before: + # the numeral fork is decided before the peel ever reads a lean, + # so the fact reached only the bare-acronym fork, which the + # numeral-only reading discarded. The acronym fork is asked now, + # so the writing decides here as it decides at the trailing slot + # of a name. Measured 2026-09-19 with a runtime wrapper forcing + # this function's `one_case` argument to None, over the population + # and the six policies decisions.md#S2's 2026-09-18 recipe names + # -- 36 of 10,752 parses move (1,792 names), on 6 distinct names + # ('Doe, Jane nee Smith DO', 'Doe, Jane nee Smith Ma', 'Jane Doe + # nee Smith Ma', 'Jane Doe nee Smith Ma JD', 'Jane Doe nee Yo-Yo + # Ma', 'John née Jones Smith MA'). THE PAIR IS THE FINDING: over + # the same corpus WITHOUT this change's own rows it moves 0, so + # the plumbing was live either way and the corpus simply held no + # name that could show it -- the blindness mechanisms.md's corpus + # field note asks to be measured before any "N names move" is + # written down. # # The chain-tail measure below (`tail`, and the re-peel after the - # chain) is the opposite, and the 2026-09-18 sweep says so: - # dropping it moves 18 of that 9,852, on 'John van der Berg Ma', - # 'John de Ma' and 'Freiherr von Berg MA' under every one of the - # six. A review round called all three sites inert together; two - # are. + # chain) is the opposite, and the 2026-09-18 sweep over the + # pre-change 9,852 says so: dropping it moves 18 of them, on + # 'John van der Berg Ma', 'John de Ma' and 'Freiherr von Berg MA' + # under every one of the six policies. skip = frozenset(range(len(pieces))) - frozenset(seen) rest = peel_walk(seen[m], ptags, skip) peeled = peel_trailing(rest, pieces, ptags, tokens, one_case) @@ -406,18 +391,25 @@ def _maiden_take(pieces: Sequence[Sequence[int]], # ('J. née Jones Smith V'). So the numeral must read as the suffix # as the take would leave the name too, and the question is asked # the way P5's reserve asks it (#425): the peel is run over the - # VIEW the take would leave, not one condition of it -- the first - # re-ask checked the preceding piece alone, and a title before the - # marker ('Dr. née Jones Smith V') leaves the numeral as assign's - # whole rest, where no fork fires at all (the code review). + # VIEW the take would leave, not one condition of it. Checking the + # preceding piece alone misses a title before the marker, where + # 'Dr. née Jones Smith V' leaves the numeral as assign's whole + # rest and no fork fires at all. if trailing < len(pieces): left = [i for i in seen if i < seen[m] or i >= trailing] view = [pieces[i] for i in left] view_tags = [ptags[i] for i in left] - if trailing_start(leading_titles(view, view_tags, tokens), - view, view_tags, tokens, - numeral_only=True, - one_case=one_case) == len(view): + # The same peel pair as above, read over the view -- and only + # `Peel.numeral` off it, because the bare-acronym fork COUNTS + # pieces and this view no longer holds the pieces it counted + # ('John née Jones Smith Ma' peeled over the pieces as written + # reads the acronym as a credential with words to spare, and + # once 'Jones Smith' has left it is the family of what + # remains). The acronym fork builds a view of its own below. + view_rest = peel_walk(leading_titles(view, view_tags, tokens), + view_tags) + if peel_trailing(view_rest, view, view_tags, tokens, + one_case).numeral is None: trailing = len(pieces) # #533: the ACRONYM fork, asked the way the numeral is -- the peel # over the pieces as they stand, then again over the name the take @@ -441,72 +433,59 @@ def _maiden_take(pieces: Sequence[Sequence[int]], # at that suffix word of its own accord. stop = max(rest[peeled.names], seen[m + run + 1]) head = pieces[stop] - # `len(head) == 1` is DEFENSIVE, and measured inert on - # 2026-09-19 -- NOT unreachable, which is what an earlier - # round of this comment claimed on a population that could not - # contain the shape. It asks a LONE piece's question, and the - # answer below reads `head[0]` as if the piece were the word. - # - # The multi-token piece it keeps out is the Ph. D. merge - # above. That piece carries the `suffix` ptag, so `peel_walk` - # never returns it and the peel half of `stop` cannot be it -- - # but the FIRST-WORD FLOOR is the other half, and the floor is - # an index rather than a walk, so it reaches the merged piece - # whenever that piece is the second word after the marker. - # 'BERG, ABDUL Z DOMU MA PH. D.' is that shape. - # - # THE NEGATIVE CONTROL, re-measured 2026-09-19 over a - # population built to HOLD the shape -- 4,224 names (twelve - # heads x four markers x 23 bodies, each also with a ', MD' - # tail and in upper and lower case) under six policies and two - # lexicons, the default and one listing `ph` ambiguous, 50,688 - # parses. A probe that fires wherever the tag test admits a - # head this length test then DECLINES -- the only sites where - # dropping it could matter -- fires 1,440 times, against - # 25,920 reaches of this site and 19,296 tag admissions. The - # earlier "0 over 21,504" was the population, not the branch; - # mechanisms.md's corpus field note is about exactly this. + # `len(head) == 1` is DEFENSIVE and measured inert + # (2026-09-19) rather than unreachable: it asks a LONE piece's + # question, and the answer below reads `head[0]` as if the + # piece were the word. The multi-token piece it keeps out is + # the Ph. D. merge above, whose `suffix` ptag stops `peel_walk` + # ever returning it -- so the peel half of `stop` cannot be it, + # but the FIRST-WORD FLOOR is an index rather than a walk and + # does reach the merged piece whenever that piece is the second + # word after the marker ('BERG, ABDUL Z DOMU MA PH. D.'). # - # Inert it still is, and now that is a measurement rather than - # a structure: dropping it is byte-identical -- fields, - # ambiguities and every token's role and tags -- over 905,796 - # parses (that population plus the review's 142,518-name - # corpus under the six policies). What it buys is the price, - # and the price is real where the count is not: with `ph` - # listed, dropping it takes 'BERG, ABDUL Z DOMU MA PH. D.' - # from 438 frames to 453. Kept for the reason - # `_assign.previous_kept` is: an inert branch is cheaper than - # a question asked of the wrong shape, and the three sibling - # sites (`_pieces.segment_suffix_reading` -- the `_pieces.py` - # block that pairs the same two conditions -- the GIVEN_SLOT - # branch below, and the emitter at the end of this function) - # each pair a length test with a tag test the same way. + # THE NEGATIVE CONTROL, measured 2026-09-19 over a population + # built to HOLD that shape -- 4,224 names (twelve heads x four + # markers x 23 bodies, each also with a ', MD' tail and in + # upper and lower case) under six policies and two lexicons, + # the default and one listing `ph` ambiguous, 50,688 parses. A + # probe that fires wherever the tag test admits a head this + # length test then DECLINES -- the only sites where dropping it + # could matter -- fires 1,440 times, against 25,920 reaches of + # this site and 19,296 tag admissions. Dropping it is + # byte-identical all the same -- fields, ambiguities and every + # token's role and tags -- over 905,796 parses (that population + # plus the review's 142,518-name corpus under the six + # policies). What it buys is the price, which is real where the + # count is not: with `ph` listed, dropping it takes 'BERG, + # ABDUL Z DOMU MA PH. D.' from 438 frames to 453. Kept for the + # reason `_assign.previous_kept` is: an inert branch is cheaper + # than a question asked of the wrong shape, and the three + # sibling sites (`_pieces.segment_suffix_reading`, the + # GIVEN_SLOT branch below, and the emitter at the end of this + # function) each pair a length test with a tag test the same + # way. # # The tag is the CLASS the rule is stated in terms of, and it # is not redundant with the walk the way the length test is: # 'J. née Jones Smith V' reaches here on a piece # `is_suffix_piece` REFUSES for being initial-shaped, and what - # declines it is the view check rather than the walk. What the - # tag buys is the cost, and that control is a price rather - # than a count: dropping it runs the view machinery over every - # ordinary credential the peel took, measured on a scratch - # copy of the package with the test deleted -- 'Jane Doe nee - # Smith PhD' 341 -> 353 frames and 'Doe, Jane nee Smith PhD' - # 367 -> 372, counted per `Parser.parse` the way + # declines it is the view check rather than the walk. Its own + # control is a price too, not a count: dropping it runs the + # view machinery over every ordinary credential the peel took + # -- 'Jane Doe nee Smith PhD' 341 -> 353 frames and 'Doe, Jane + # nee Smith PhD' 367 -> 372, counted per `Parser.parse` the way # tests/v2/test_benchmark counts them. No test pins those # numbers: `_CALL_BASELINE` is per-interpreter and per entry # point, and a row for one name would have to be guessed for # the four interpreters only CI runs. # - # A THIRD condition stood here and is gone: `stop < trailing`. - # It guarded nothing, structurally -- `stop` is the larger of - # a walked piece and the floor, and both are bounded by - # `trailing`, so at worst `stop == trailing` and the - # assignment below would set `trailing` to what it already is. - # Measured over the same 1,760,904 parses: `stop > trailing` - # never once, `stop == trailing` 15,696 times, and the tag - # test declined every one of those, so removing it moved no - # parse and no frame. + # A THIRD condition stood here and is gone: `stop < trailing` + # guarded nothing, `stop` being the larger of a walked piece + # and the floor and both bounded by `trailing`, so at worst + # `stop == trailing` and the assignment below sets `trailing` + # to what it already is. Measured over 1,760,904 parses: + # `stop > trailing` never once, `stop == trailing` 15,696 + # times, and the tag test declined every one of those. if (len(head) == 1 and AMBIGUOUS_ACRONYM_TAG in tokens[head[0]].tags): left = [i for i in seen if i < seen[m] or i >= stop] @@ -603,12 +582,12 @@ def _maiden_take(pieces: Sequence[Sequence[int]], # byte-identical (fields, ambiguities, token roles and tags) over # the 905,796-parse oracle at the same frame counts. last = pieces[seen[j - 1]] + word = tokens[last[0]] if (reader is not TailReader.NONE - and not tokens[last[0]].tags.isdisjoint( - _AMBIGUOUS_CREDENTIAL_TAGS)): + and not word.tags.isdisjoint(_AMBIGUOUS_CREDENTIAL_TAGS)): ambiguities.append(PendingAmbiguity( AmbiguityKind.SUFFIX_OR_NAME, - f"{tokens[last[0]].text!r} ending the maiden name is also " + f"{word.text!r} ending the maiden name is also " f"a post-nominal; the maiden marker's clause keeps it " f"rather than reading it as one", tuple(last))) @@ -666,10 +645,9 @@ def _group_segment(seg: tuple[int, ...], additional: int, # two parameters. They are given the SAME list wherever nothing is # suppressed, which is every segment that is NOT after a family # comma; what the split buys is the other case, where `None` on - # the first must not reach the second. An earlier spelling - # defaulted the maiden channel to whatever the first was, so a - # caller passing `ambiguities=None` silenced both -- which the - # required argument now makes unsayable (the review's finding). + # the first must not reach the second -- a maiden channel + # defaulting to whatever the first was would let a caller passing + # `ambiguities=None` silence both (the review's finding). def title(k: int) -> bool: return is_title_piece(pieces[k], ptags[k], tokens) diff --git a/nameparser/_pipeline/_pieces.py b/nameparser/_pipeline/_pieces.py index d4240e3b..2a648317 100644 --- a/nameparser/_pipeline/_pieces.py +++ b/nameparser/_pipeline/_pieces.py @@ -406,7 +406,6 @@ def peel_walk(start: int, ptags: Sequence[Set[str]], def trailing_start(start: int, pieces: Sequence[Sequence[int]], ptags: Sequence[Set[str]], tokens: Sequence[WorkToken], skip: Set[int] = frozenset(), - numeral_only: bool = False, *, one_case: bool | None) -> int: """Where assign's trailing suffix run begins, read over the pieces as they stand from `start`: the index of the first piece the S2 @@ -417,19 +416,14 @@ def trailing_start(start: int, pieces: Sequence[Sequence[int]], or a bare acronym with words to spare, into the family or the maiden name. - `numeral_only` answers off `Peel.numeral` alone: the bare-acronym - fork counts pieces, and a caller that REMOVES the pieces it - counted cannot use that count as it stands ('John née Jones Smith - Ma' peeled over the pieces as written reads the acronym as a - credential with words to spare, and once 'Jones Smith' has left - it is the family of what remains). Its one caller is the maiden - walk's numeral re-ask, over the view the take would leave; the - walk asks the acronym fork itself, over a view of its own - (#533).""" + Both forks, always. A caller that needs one of them alone -- the + maiden walk re-asking the numeral over the view its take would + leave, where the acronym fork's piece COUNT no longer describes + the name -- calls the `peel_walk` + `peel_trailing` pair this + wraps and reads the half it wants (#533). A `numeral_only` flag + lived here for that one caller and cost it a frame.""" rest = peel_walk(start, ptags, skip) peeled = peel_trailing(rest, pieces, ptags, tokens, one_case) - if numeral_only: - return rest[-1] if peeled.numeral is not None else len(pieces) return rest[peeled.names] if peeled.names < len(rest) else len(pieces) @@ -447,16 +441,14 @@ def trailing_start(start: int, pieces: Sequence[Sequence[int]], # the call at all. # # A THIRD caller since #533 -- credential_at_the_given_slot just -# below -- deliberately does NOT pre-check, because it is the one -# predicate that owns #531's reading and membership is its caller's -# to decide (its own docstring says so). It pushes the test out to -# ITS callers rather than asking it twice, and the frame argument -# holds transitively because both of them do ask it inline: assign's -# walkable pass tests `AMBIGUOUS_ACRONYM_TAG in tok.tags` after a -# `len(piece) == 1` before the call (_assign.py, the given part's -# trailing slot), and the maiden walk's view check spells the same -# pair inside its `all(...)` (_group.py, `_maiden_take`). So no -# non-member piece reaches this function down that route either. +# below -- deliberately does NOT pre-check: it owns #531's reading +# and leaves membership to its own callers (its docstring says so), +# and the frame argument holds transitively because both of them ask +# inline -- `AMBIGUOUS_ACRONYM_TAG in tok.tags` after a +# `len(piece) == 1` at assign's given-part trailing slot, and the +# same pair inside the `all(...)` of `_group.py`'s `_maiden_take` +# view check. So no non-member piece reaches this function down that +# route either. def listed_lean(token: WorkToken, one_case: bool | None) -> Lean | None: """`ambiguous_lean` for a LISTED bare-ambiguous token, or None if the token is not tagged a listed member, is admitted by SHAPE