Skip to content

sf push/sync write the web's default format (cipher_version 3, row-bound, PQ); sync --watch refreshes its session - #10

Merged
chogarcia merged 1 commit into
mainfrom
fix/p2-v1-format
Oct 1, 2026
Merged

chogarcia merged 1 commit into
mainfrom
fix/p2-v1-format

Conversation

@chogarcia

Copy link
Copy Markdown
Member

Problem

From the 2026-09-30 audit (crypto finding 9, plus the mcp-cli-desktop "Claims" and sf sync --watch findings):

  • sf push / sf sync wrote the deprecated v0 format (cipher_version 1: AES-GCM chunks under a random 4-byte nonce prefix, no header), which crypto/spec/format-v0.md forbids for new writes. It has no truncation detection: a server can drop trailing chunks and the file still decrypts. It has no row binding, so a server can move a file's bytes onto another row. It has no post-quantum layer, while the package was described as a "post-quantum vault".
  • sf sync --watch signed in once. After the access token expired (about 1 h), every pass failed every changed file while the process kept looping and never exited non-zero.
  • @shieldfive/crypto floated on ^1.0.0-beta.1.

Fix

Format. Uploads are now exactly what the web writes by default (encryptModal.tsx, v1-pq-hybrid): cipher_version 3, suite 0x03, v1 wire format, produced by @shieldfive/crypto's encryptStreamPqHybridV1.

  • The header file_id is the row id that create-upload-session returned (uuidToBytes(session.fileId)), which binds the object to its row. The MAC'd header carries the plaintext size and chunk count, and the chunk AAD binds index, total and is-final.
  • The recipient is the owner's ML-KEM-1024 identity, derived locally from the root key with createIdentity({ masterSecret: rootKey }), exactly as web utils/pqIdentity.ts does. It is never fetched, so the server cannot substitute it. csk_wrapped holds the classical envelope key under the root key, as on the web. pqk_fk_wrapped is left for the owner's backfill, also as on the web.
  • Storage parts: part 1 = header || frame_0, part i = frame_i. The split is computed from the byte format (header length, then each frame's length prefix), not from how the encoder enqueues. Chunk size is the server's v1 chunk (5 MiB, session.chunkSize).
  • The upload proof uses buildUploadProofV3 from the library, the same function the web and MCP use.
  • Request body: cipherVersion: 3, and no cipher_nonce_prefix or chunkSize (the server rejects both for v1).
  • Name row binding. At finalize the CLI sends nameEncryptedV6 (encryptNameV6, AAD = row id), the same best-effort upgrade the web sends, so the v4 name written at create is replaced by a row-bound one.
  • The v0 writer (encryptFileChunk, encryptChunkWithDigest, the v0 proof, the nonce-prefix generator) is removed from src/. Reading v0: the CLI has no download or decrypt path, so there is nothing to keep. Files uploaded by 0.3.0 and earlier still open in the web and mobile apps, which read v0.

sf sync --watch. authAndUnlock now also returns the refresh token, the expiry and a refreshSession (the same refreshAccessToken exchange the agent uses). runSync asks for a token before every upload. It refreshes once the token is within 120 s of expiry. The refresh is single-flight and the rotated refresh token replaces the old one, because Supabase revokes a session whose refresh token is reused. If the refresh fails, sync stops with an error (non-zero exit) instead of failing every file forever. A single sf sync pass with failed files now exits non-zero.

Re-uploads as new root-level copies: not fixed, documented. Replacing or trashing the previous fileId means moving it into the Bin. That needs the Bin's folder key to re-wrap csk_wrapped and the name, and this client holds only the root key, so it is not cheap. The sync.mjs header now states the behaviour, and it is listed as a follow-up below.

Also: @shieldfive/crypto is pinned to ~1.0.1. The description no longer says "post-quantum vault"; it names the hybrid scheme. README "How the upload works" is rewritten for the new format, and CHANGELOG is updated.

Tests (behavioural)

  • test/upload.flow.test.mjs (rewritten assertions): every byte the client would send is captured and the stored object is opened with @shieldfive/crypto's reference reader (autoDecryptBlob), using only the session fields and the root key (test/v1Reference.mjs, which imports none of the CLI's crypto). The tests assert:
    • plaintext round-trip, suite 3, header file_id == row id, cipherVersion: 3 and no v0 fields in the request;
    • the proof equals an independent HMAC built from the server verifier's description;
    • the finalize nameEncryptedV6 opens for this row and not for another one;
    • multipart, out-of-order completion, retry and direct paths.
  • test/v1Format.test.mjs with shared vectors test/vectors/v3-upload.json (regenerate with node scripts/generate-vectors.mjs; decrypt vectors, since encryption is randomised). The tests check:
    • the vector opens through the blob reader and through the web's streaming reader decryptStreamPqHybridV1;
    • the ML-KEM public key fingerprint equals the web derivation;
    • v4 and v6 names open through the library and v6 is row-bound;
    • dropping the last chunk or swapping two chunks is rejected;
    • one part per frame for odd-sized input, and a file that changes size mid-read is refused.
  • test/sync.test.mjs: three new tests: --watch refreshes once (with the held refresh token) and later uploads use the new token; a refresh failure ends sync with an error before any upload is attempted with the expiring token; a single pass with a failed file rejects.

Fails on origin/main, passes here. The new test files were copied onto an origin/main checkout:

  • upload.flow: 3 of 4 fail (v0 output, no v3 header, file_id is not the row);
  • v1Format: fails (no v3 writer);
  • sync: the refresh test fails, and the refresh-failure test never terminates on main (the watch loop runs forever), which is the bug.

On this branch, full npm test (macOS, Node 20): 133 tests, 133 pass.

Rollout

Client-only. The server already accepts cipher_version 3 with folderId: null from the web. Merge, then the founder releases. No tag pushed, nothing published.

Not verified against the live server. All upload tests use mocks written from the server source (create-upload-session, complete-upload/finalize.ts, uploadProofClient.ts). Before release, run one real sf push of a small file and a file over 5 MiB on a test account and open both on web and Android.

Follow-ups

  • Changed files still become a new root-level copy on every change (see above). Trashing the previous version needs a client-side Bin folder-key unwrap (or a server move-to-bin that keeps the wraps).
  • sf sync still uploads into the vault root only; there is no folder target.
  • Uploads are left without pqk_fk_wrapped, as web uploads are. Folder shares and agent grants can read them only after the owner's backfill.

…ow-bound); refresh the session under sync --watch
@chogarcia
chogarcia merged commit b87e05e into main Oct 1, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant