diff --git a/docs/developer/api.md b/docs/developer/api.md index 51c9fb5..49c272a 100644 --- a/docs/developer/api.md +++ b/docs/developer/api.md @@ -51,7 +51,8 @@ generic parse-length failure. ### Boundaries -- Only parsing and profile-specific factories construct artifacts. +- Only parsing, `complete_checksum()`, and profile-specific factories construct + artifacts. - `bech32.py` keeps the recognizable BIP173 reference names and accumulator structure. Encoding takes an explicit checksum specification. Decoding without one returns the HRP and complete data part after checking only the @@ -99,7 +100,7 @@ generic parse-length failure. hidden state. Private Python names are convention rather than access control. The supported -surface is the 22-name package `__all__`; direct use of private helpers is +surface is the 23-name package `__all__`; direct use of private helpers is unsupported but remains in the review scope. ### Size budget @@ -118,6 +119,7 @@ base artifact types rather than falling back to a registered application. |---|---:|---:|---:|---:| | parse S/share | yes | yes | unshared S only | yes | | semantic S bytes | no | 16, 20, 24, 28, 32, or 64 | exactly 32 | no | +| checksum completion API | yes | yes | unshared S only | yes | | recovery and API share derivation | yes | yes | no | yes | | `codex32` recovery/share/correction | yes | yes | correction only | yes | | unshared generation / shared ceremony API | no | six supported sizes | no | no | diff --git a/docs/security/model.md b/docs/security/model.md index 6eabc34..00821dc 100644 --- a/docs/security/model.md +++ b/docs/security/model.md @@ -115,6 +115,28 @@ complete explicit `ms1` string. ## Creation, sharing, and recovery controls +`ms32 checksum` generates a checksum only for new Codex32 Book worksheet +data. It is separate from `check` and `correct` and refuses input of any complete +`ms` string length, so a damaged complete string cannot be mistaken for a longer +unchecksummed payload. The non-pink worksheet data is entered twice, with the +terminal cleared before the second entry, and the entries must match before +anything is computed or displayed. Each entry must be ASCII and entirely +uppercase or lowercase. The full completed string is then displayed so the +operator can copy its checksum into the pink squares. After the operator presses Enter, +the terminal and scrollback are cleared where supported and only the pink +checksum squares must be re-entered, without editable prefill. These entries +catch typing errors; they cannot detect an error already written on the +worksheet. A mismatch, EOF, or interruption releases no completed string to +stdout, although the preview has already been displayed on stderr. Format errors +give no detail. `--plain` preserves the first entry's case; default terminal +output uses the normal card formatting. Plain or redirected output does not skip +checksum confirmation; noninteractive input is rejected before it is read. +Agreement cannot establish how the worksheet data was generated or detect an +error already present in its non-pink data. Terminal clearing remains best effort. +The public `complete_checksum()` API applies the same parsing and profile +payload rules for any HRP but has none of these interactive safeguards; +applications that expose it must provide their own. + Fresh shared creation generates *k* random initial shares. Each uses a separate full-payload OS-CSPRNG request. The current share string must be re-entered exactly, ignoring case and whitespace, before the next request. Confirmation @@ -179,8 +201,8 @@ reallocate the ledger's mass to additional hypotheses. `codex32` checks and corrects unshared CL secrets, and checks, corrects, recovers, and derives shares for compatible BIP39 and opaque-HRP sets. `ms32` -applies an `ms` input filter and alone owns Bitcoin creation, xprv, and wallet -commands. Both façades offer worksheet-residue correction; only `ms32 correct` +applies an `ms` input filter and alone owns Bitcoin creation, checksum +generation, xprv, and wallet commands. Both façades offer worksheet-residue correction; only `ms32 correct` offers `--bytes`. ### Low-discrimination correction disclosure diff --git a/docs/user/guide.md b/docs/user/guide.md index b875ab0..1734ddc 100644 --- a/docs/user/guide.md +++ b/docs/user/guide.md @@ -279,14 +279,30 @@ follow Bitcoin Core v32's maintained confirmed.” `--plain` skips card confirmation. Redirected damaged input can still require an interactive correction confirmation. +### Advanced: Codex32 Book checksum worksheet + +`ms32 checksum` computes the checksum for new data from the +[Codex32 Book](https://secretcodex32.com) checksum worksheet, instead of +computing it by hand. To verify a hand-computed checksum, enter the complete +string in `ms32 check`. `ms32 checksum` is not a validation or repair tool: it refuses +complete strings, and you must never use it to replace the checksum of an +existing backup. A checksum cannot make weak input secure, so use only data +from the Book's dice procedure. The command needs an interactive terminal, +asks for the non-pink worksheet data twice, clearing the screen between +entries, and then shows the full completed string. Copy the checksum into the +pink squares and press Enter; the screen clears, then you type back only the +pink checksum squares. The repeated entries catch typing errors, not errors +already on the worksheet, so check the worksheet carefully first. Each entry must be entirely uppercase or lowercase. `--plain` preserves the first +entry's case; the default terminal output uses the usual uppercase card layout. +The completed string reaches stdout only after the checksum is confirmed. + ### Advanced: completing missing trailing characters The correction command can fill trailing erasures when the final characters are actually unknown. For a normal codex32 string, replace the final 13 missing characters with `?`; a long-checksum string uses 15. Thirteen genuinely unreadable characters at the end of an existing backup are therefore a valid -recovery case. The same mechanism can complete newly generated worksheet data -whose final 13 squares have not yet been filled. +recovery case. To checksum new Codex32 Book data instead, use `ms32 checksum`. Never intentionally delete or replace the existing final characters from a backup that merely fails validation. Doing so can hide a transcription or diff --git a/src/codex32/__init__.py b/src/codex32/__init__.py index 56ceb77..9e94d51 100644 --- a/src/codex32/__init__.py +++ b/src/codex32/__init__.py @@ -4,6 +4,7 @@ Header, Secret, Share, + complete_checksum, derive_share, parse_codex32, recover_secret, @@ -44,6 +45,7 @@ "Secret", "Share", "WorksheetCorrection", + "complete_checksum", "correct", "correct_worksheet_residue", "derive_share", diff --git a/src/codex32/_cli_parser.py b/src/codex32/_cli_parser.py index c247210..16d3d7a 100644 --- a/src/codex32/_cli_parser.py +++ b/src/codex32/_cli_parser.py @@ -183,6 +183,14 @@ def parser(prog: str = "codex32", *, master_seed: bool = False) -> argparse.Argu help="use an existing Bitcoin codex32 secret or hexadecimal seed", ) + checksum = _command(commands, "checksum", "generate the checksum for a new Codex32 Book worksheet") + checksum.description = ( + "Generate the checksum for new data on a Codex32 Book checksum worksheet. " + "Never use it on an existing backup: use 'ms32 check' or 'ms32 correct'. " + "Requires an interactive terminal and re-entry of the pink checksum squares." + ) + _terminal_output(checksum) + wallet = _command(commands, "wallet", "restore a Bitcoin Core wallet") _wallet_options(wallet) diff --git a/src/codex32/bip93.py b/src/codex32/bip93.py index 238f7dd..1668264 100644 --- a/src/codex32/bip93.py +++ b/src/codex32/bip93.py @@ -217,6 +217,17 @@ def _from_parts( return parse_codex32(text.upper() if uppercase else text) +def complete_checksum(unchecksummed_text: str) -> Share | Secret: + """Add the checksum to new worksheet data, such as a Codex32 Book checksum worksheet. + + Never use this on an existing backup: it turns transcription errors into a + valid string. A valid checksum does not establish safe entropy. + """ + hrp, body = bech32_decode(unchecksummed_text) + header = Header._from_symbols(tuple(body[:6])) + return _from_parts(hrp, header, tuple(body[6:]), uppercase=unchecksummed_text.isupper()) + + def _lagrange_weights(points: tuple[int, ...], target: int) -> tuple[int, ...]: weights: list[int] = [] for position, point in enumerate(points): diff --git a/src/codex32/cli.py b/src/codex32/cli.py index 1ffd1f1..8902262 100644 --- a/src/codex32/cli.py +++ b/src/codex32/cli.py @@ -34,12 +34,14 @@ from codex32._cli_input import read_artifacts as _artifacts from codex32._cli_input import read_text as _text from codex32._cli_parser import parser as _parser +from codex32.bech32 import _validate_single_case_ascii from codex32.bip93 import ( IDX_SORT, Header, Secret, Share, _normalize_target, + complete_checksum, derive_share, parse_codex32, recover_secret, @@ -57,6 +59,7 @@ generate_master_seed, ) from codex32.profiles import Profile, _profile_rules +from codex32.profiles.ms32 import TEXT_LENGTHS as _MS_TEXT_LENGTHS from codex32.profiles.ms32 import MasterSeed from codex32.profiles.ms32 import ( _text_length as _ms_text_length, @@ -553,6 +556,61 @@ def _create( return 0 +def _worksheet_text(prompt: str) -> str: + value = _text(prompt, prompt_end=":\n\n> MS1") + return value if "1" in value else ("MS1" if value.isupper() else "ms1") + value + + +def _same_entry(entered: str, expected: str) -> bool: + try: + _validate_single_case_ascii(entered) + except CodexError: + return False + return entered.lower() == expected.lower() + + +def _checksum(plain: bool) -> int: + # Generation only: complete strings belong to check and correct (codex32 discussion #78). + if not sys.stdin.isatty(): + raise _UsageError("Checksum generation requires an interactive terminal.") + warning = ( + "DANGER: Incorrect input can cause permanent loss of funds.\n" + "Use this only for new data from the Codex32 Book and its Dice De-biasing Worksheet.\n" + "Never use it to replace the checksum of an existing backup. Use 'ms32 check' or 'ms32 correct'.\n" + "Check every non-pink square on the worksheet; errors written there cannot be detected here.\n" + "This command cannot verify how your data was generated.\n" + "Do not enter raw dice rolls, seed words, hexadecimal seeds, or passwords.\n" + ) + _print(warning, err=True, danger=True) + text = _worksheet_text("Checksum worksheet non-pink bold squares") + if len(text) in _MS_TEXT_LENGTHS: + raise _UsageError("This command does not accept complete strings. Use 'ms32 check' to check one.") + try: + # The Book's checksum worksheet has 32 non-pink squares after the ms prefix. + if len(text) != 35 or text[:3].lower() != "ms1": + raise ValueError + artifact = complete_checksum(text) + except (CodexError, ValueError) as error: + raise _UsageError( + "The worksheet input is not in the expected format.\n" + "Consult the Codex32 Book and check the non-pink bold squares." + ) from error + clear = "\x1b[3J\x1b[2J\x1b[H" if sys.stderr.isatty() else "" + if not _same_entry(_worksheet_text(clear + "Re-enter the worksheet non-pink bold squares"), text): + raise _CommandError("Entries do not match. Restart and re-enter the worksheet.") + _emit(artifact, plain, err=True) + _text( + "Copy the checksum into the worksheet's pink squares, then press Enter", + optional=True, + prompt_end=". ", + ) + entered = _text(clear + "Re-enter only the pink checksum squares", prompt_end=":\n> ") + if not _same_entry(entered, artifact.text[-13:]): + raise _CommandError("Checksum not confirmed. Check the pink squares before using this backup.") + _emit(artifact, plain) + return 0 + + def _correct( residue: bool, erasures: tuple[int, ...], @@ -715,6 +773,8 @@ def _dispatch(arguments: argparse.Namespace, context: _CliContext) -> int: cast(str | None, arguments.indices), bool(arguments.existing), ) + if command == "checksum": + return _checksum(plain) if command == "correct": return _correct( bool(arguments.residue), diff --git a/tests/test_cli.py b/tests/test_cli.py index 07124c1..c770c17 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -147,7 +147,9 @@ def _invoke(args: list[str], *lines: str) -> _Result: contextlib.redirect_stdout(stdout), contextlib.redirect_stderr(stderr), ): - entrypoint = ms_main if args[0] in {"create", "xprv", "wallet"} or "--bytes" in args else main + entrypoint = ( + ms_main if args[0] in {"create", "checksum", "xprv", "wallet"} or "--bytes" in args else main + ) status = entrypoint(args) return _Result(status, stdout.getvalue(), stderr.getvalue()) @@ -3044,3 +3046,176 @@ def emit(_artifact, _plain, **kwargs): assert emitted_fingerprints and all(fingerprint is None for fingerprint in emitted_fingerprints) assert checked == [secret.seed_bytes] assert core.expected == core.fingerprint(secret) + + +def _invoke_checksum(*entries: str, args: tuple[str, ...] = ()) -> tuple[_Result, list[str]]: + prompts: list[str] = [] + answers = iter(entries) + + def answer(prompt: str) -> str: + prompts.append(prompt) + return next(answers) + + stdout, stderr = io.StringIO(), io.StringIO() + with ( + patch.object(sys, "stdin", _TTYInput()), + patch("builtins.input", answer), + patch("codex32._cli_input._line_editor", None), + contextlib.redirect_stdout(stdout), + contextlib.redirect_stderr(stderr), + ): + status = ms_main(["checksum", *args]) + return _Result(status, stdout.getvalue(), stderr.getvalue()), prompts + + +@pytest.mark.parametrize("plain", [False, True]) +def test_checksum_rejects_noninteractive_input_before_reading(plain: bool) -> None: + with patch("codex32.cli._text") as read: + result = _invoke(["checksum", *(["--plain"] if plain else [])], VECTOR_1["secret_s"][:-13]) + assert result.exit_code == 2 + assert result.stdout == "" + assert result.stderr == "ms32 checksum: Checksum generation requires an interactive terminal.\n" + read.assert_not_called() + + +@pytest.mark.parametrize("uppercase", [False, True]) +@pytest.mark.parametrize("include_prefix", [False, True]) +def test_checksum_completes_the_book_worksheet_after_checksum_confirmation( + uppercase: bool, include_prefix: bool +) -> None: + expected = VECTOR_1["secret_s"].upper() if uppercase else VECTOR_1["secret_s"] + entered = expected[:-13] if include_prefix else expected[3:-13] + result, prompts = _invoke_checksum(entered, entered, "", expected[-13:], args=("--plain",)) + assert result.exit_code == 0 + assert result.stdout == expected + "\n" + assert result.stderr.startswith("DANGER: Incorrect input can cause permanent loss of funds.") + assert "Never use it to replace the checksum of an existing backup." in result.stderr + assert prompts == [ + "Checksum worksheet non-pink bold squares:\n\n> MS1", + "Re-enter the worksheet non-pink bold squares:\n\n> MS1", + "Copy the checksum into the worksheet's pink squares, then press Enter. ", + "Re-enter only the pink checksum squares:\n> ", + ] + + +@pytest.mark.parametrize( + "entered", + [ + VECTOR_1["secret_s"][3:-14], # one square short + VECTOR_1["secret_s"][3:-13] + "x", # one square extra + "0tests" + "x" * 32, # a 20-byte secret does not fit the Book worksheet + "Ms10tests" + "x" * 26, # mixed case + "ms10testsxxxxbxxxxxxxxxxxxxxxxx", # b is not a bech32 character + "cl10tests" + "x" * 26, # only ms + "xtests" + "x" * 26, # invalid threshold + ], +) +def test_checksum_rejects_other_input_without_hints(entered: str) -> None: + result, prompts = _invoke_checksum(entered) + assert result.exit_code == 2 + assert result.stdout == "" + assert result.stderr.endswith( + "ms32 checksum: The worksheet input is not in the expected format.\n" + "Consult the Codex32 Book and check the non-pink bold squares.\n" + ) + assert len(prompts) == 1 + + +@pytest.mark.parametrize( + "entered", + [ + VECTOR_1["secret_s"], + VECTOR_1["secret_s"][3:], + # A damaged 16-byte string is as long as an unchecksummed 24-byte one. + VECTOR_1["secret_s"][:-1] + "q", + VECTOR_2["share_A"], + ], +) +def test_checksum_refuses_complete_strings(entered: str) -> None: + result, prompts = _invoke_checksum(entered) + assert result.exit_code == 2 + assert result.stdout == "" + assert result.stderr.endswith( + "ms32 checksum: This command does not accept complete strings. Use 'ms32 check' to check one.\n" + ) + assert len(prompts) == 1 + + +@pytest.mark.parametrize("ending", ["match", "retyped", "mismatch", "eof", "interrupt"]) +def test_checksum_confirms_both_entries_before_stdout(monkeypatch, ending): + cli = importlib.import_module("codex32.cli") + original = VECTOR_1["secret_s"] + calls = [] + stdout, stderr = io.StringIO(), _TTYOutput() + + def read(prompt, *, prompt_end, optional=False): + assert stdout.getvalue() == "" + calls.append((prompt, prompt_end)) + if len(calls) == 1: + return original[3:-13] + if len(calls) == 2: + assert prompt == "\x1b[3J\x1b[2J\x1b[HRe-enter the worksheet non-pink bold squares" + assert original not in stderr.getvalue() + return original[3:-14] + ("q" if ending == "retyped" else original[-14]) + if len(calls) == 3: + assert optional + assert original in stderr.getvalue() + return "" + assert prompt == "\x1b[3J\x1b[2J\x1b[HRe-enter only the pink checksum squares" + if ending == "eof": + raise EOFError + if ending == "interrupt": + raise KeyboardInterrupt + return "q" * 13 if ending == "mismatch" else original[-13:].upper() + + monkeypatch.setattr(cli.sys, "stdin", _TTYInput()) + monkeypatch.setattr(cli, "_text", read) + with contextlib.redirect_stdout(stdout), contextlib.redirect_stderr(stderr): + status = ms_main(["checksum", "--plain"]) + assert len(calls) == (2 if ending == "retyped" else 4) + assert status == {"match": 0, "retyped": 1, "mismatch": 1, "eof": 2, "interrupt": 130}[ending] + assert stdout.getvalue() == (original + "\n" if ending == "match" else "") + + +def test_checksum_rejects_positional_input_before_reading() -> None: + with patch("codex32.cli._text") as read: + result = _invoke(["checksum", "2testa"]) + assert result.exit_code == 2 + assert "Remove or correct these arguments: 2testa" in result.stderr + read.assert_not_called() + + +@pytest.mark.parametrize("entered", ["xrpp870hKkqrm", "xrpp870hKkqrm", "XRPP870HKKQRM", "xrpp870hkkqrmx", ""]) +def test_checksum_confirmation_rejects_invalid_pink_squares(entered: str) -> None: + original = VECTOR_2["share_A"] + result, prompts = _invoke_checksum(original[:-13], original[:-13], "", entered, args=("--plain",)) + assert result.exit_code != 0 + assert result.stdout == "" + assert len(prompts) == 4 + + +@pytest.mark.parametrize("uppercase", [False, True]) +@pytest.mark.parametrize("plain", [False, True]) +def test_checksum_terminal_output_preserves_plain_case_and_pretty_default( + uppercase: bool, plain: bool +) -> None: + original = VECTOR_2["share_A"] if uppercase else VECTOR_2["share_A"].lower() + stdout, stderr = _TTYOutput(), io.StringIO() + with ( + patch.object(sys, "stdin", _TTYInput()), + patch( + "builtins.input", + side_effect=[original[:-13], original[:-13].swapcase(), "", original[-13:].swapcase()], + ), + patch("codex32._cli_input._line_editor", None), + contextlib.redirect_stdout(stdout), + contextlib.redirect_stderr(stderr), + ): + status = ms_main(["checksum", *(["--plain"] if plain else [])]) + assert status == 0 + if plain: + assert stdout.getvalue() == original + "\n" + else: + assert " ".join(original.upper()[i : i + 4] for i in range(0, len(original), 4)) in " ".join( + re.sub(r"\x1b\[[0-9;]*m", "", stdout.getvalue()).split() + ) diff --git a/tests/test_generic_hrp.py b/tests/test_generic_hrp.py index 9111a7d..172136f 100644 --- a/tests/test_generic_hrp.py +++ b/tests/test_generic_hrp.py @@ -122,6 +122,7 @@ def test_cli_split_and_unknown_neutral_summary() -> None: " share derive a share from codex32 strings\n" " correct suggest repairs for a damaged codex32 string\n" " create create or confirm a backup, or split an existing secret\n" + " checksum generate the checksum for a new Codex32 Book worksheet\n" " wallet restore a Bitcoin Core wallet\n" " xprv export the root extended private key\n\n" "Never include a secret or share in command arguments.\n" @@ -129,9 +130,8 @@ def test_cli_split_and_unknown_neutral_summary() -> None: ) for command in ("check", "correct", "secret", "share"): assert command in generic_help and command in ms_help - for command in ("create", "xprv", "wallet"): + for command in ("create", "checksum", "xprv", "wallet"): assert command not in generic_help and command in ms_help - assert "checksum" not in generic_help and "checksum" not in ms_help correct_help = _invoke(main, ["correct", "--help"])[1] assert ( "Suggest repairs for wrong, unreadable, missing, extra, or swapped characters\n" diff --git a/tests/test_public_api.py b/tests/test_public_api.py index 70106c5..4900452 100644 --- a/tests/test_public_api.py +++ b/tests/test_public_api.py @@ -8,9 +8,8 @@ from pathlib import Path import pytest -from data.bip93_vectors import VECTOR_2 +from data.bip93_vectors import VECTOR_1, VECTOR_2, VECTOR_3, VECTOR_4, VECTOR_5, VECTOR_6 -import codex32 from codex32 import ( CorrectionCandidate, CorrectionContext, @@ -18,6 +17,7 @@ MasterSeed, Profile, Share, + complete_checksum, correct, correct_worksheet_residue, parse_codex32, @@ -64,9 +64,19 @@ def test_python_bip32_boundary_stops_at_the_root() -> None: assert imports == {"hashlib", "hmac"} -def test_checksum_completion_is_not_public_api() -> None: - assert "complete_checksum" not in codex32.__all__ - assert not hasattr(codex32, "complete_checksum") +@pytest.mark.parametrize( + ("text", "checksum_length"), + [ + (VECTOR_1["secret_s"], 13), + (VECTOR_2["share_A"], 13), + (VECTOR_3["share_a"], 13), + (VECTOR_4["secret_s"], 13), + (VECTOR_5["secret_s"], 15), + (VECTOR_6["codex32_luea"], 13), + ], +) +def test_complete_checksum_restores_vector_checksums(text: str, checksum_length: int) -> None: + assert complete_checksum(text[:-checksum_length]).text == text def test_share_has_symbols_but_no_byte_or_padding_api() -> None: