diff --git a/CHANGELOG.md b/CHANGELOG.md index aaab6e1..815762e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Security +- **Symlinks are not followed.** `push`, `sync` and the agent list files with + `lstat` and skip symlinks, and files are opened with `O_NOFOLLOW`, so a link + inside the folder can no longer upload a file from outside it. +- **`push` skips dotfiles by default** (`.env`, `.git`, ...), as `sync` already + did. `sf push --include-dotfiles` restores the old behaviour. +- **Sync manifest written 0600.** `.shieldfive-sync.json` lists file names and + vault ids; it is no longer world-readable. +- **README and npm description** now state the scheme precisely: contents are + AES-256-GCM under a post-quantum hybrid key, names are AES-256-GCM under the + vault key, and size, part hashes and timestamps are visible to the server. + +### Fixed + +- A failed upload cancels its pending session (`POST /api/files/delete`, as the + web app does) instead of leaving a row holding quota until the reaper runs. + Not done when only the finalize answer was lost, since the file may be in. +- 0-byte files: uploaded as a header-only cipher-v3 object with the server's + empty-file proof when `/api/files/upload-capabilities` says `emptyCipherV3`; + otherwise skipped with a clear message (exit code 2) before any session. + - **Downgrade pin for sign-in.** After a sign-in with a login secret (auth version 2), the CLI records the account (a SHA-256 of the email) in `~/.shieldfive/auth-pins.json` (mode 0600). A later server answer of diff --git a/README.md b/README.md index 3587444..5ddef6c 100644 --- a/README.md +++ b/README.md @@ -1,24 +1,35 @@ # @shieldfive/cli > Command-line client for [ShieldFive](https://shieldfive.com). Encrypt a folder -> on your machine, then push or continuously sync it to your EU-hosted, -> post-quantum vault. Everything is encrypted locally before it leaves your -> machine. Read the code and verify that for yourself. +> on your machine, then push or continuously sync it to your EU-hosted vault. +> File contents and names are encrypted locally before they leave your machine +> (contents with a post-quantum hybrid key). Read the code and verify that for +> yourself. [![CI](https://github.com/shieldfive/cli/actions/workflows/ci.yml/badge.svg)](https://github.com/shieldfive/cli/actions/workflows/ci.yml) [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE) [![Status](https://img.shields.io/badge/status-early-yellow.svg)](#honest-scope) The point of this tool being open source is that you do not have to trust a -marketing claim. The plaintext of your files is never sent to a ShieldFive -server. The only bytes that leave your machine are ciphertext (post-quantum -hybrid: ML-KEM-1024 + X25519 key encapsulation, AES-256-GCM chunks) and a short -cryptographic proof. Two files carry the whole story: +marketing claim. The plaintext of your files and their names is never sent to a +ShieldFive server: + +- **File contents** — AES-256-GCM chunks under a per-file key that combines a + random envelope key (wrapped by your vault root key) with an ML-KEM-1024 + + X25519 hybrid encapsulation to your vault identity. +- **File names** — AES-256-GCM under a key derived from your vault root key + (not the post-quantum layer), plus a keyed HMAC of the lowercased name so the + server can spot duplicates without reading it. + +What the server does see, by design: the plaintext size of each file, part +hashes (SHA-1 of ciphertext), the wrapped keys, an upload proof, timestamps and +your account. Sign-in sends a secret derived from your password, never the +password itself. Two files carry the whole story: - [`src/uploadCrypto.mjs`](src/uploadCrypto.mjs) — the encryption applied to every chunk and to the filename. -- [`src/upload.mjs`](src/upload.mjs) — the only place bytes are sent over the - network. Every request body is ciphertext. +- [`src/upload.mjs`](src/upload.mjs) — the only place file bytes are sent over + the network. Every file body it sends is ciphertext. If you want to see encryption happen without an account at all, run `sf encrypt ` — it encrypts each file locally and prints the ciphertext @@ -81,7 +92,7 @@ npm test # 107 tests sf login [--idle=] sign in once; an agent holds the session in memory (locks after 8 h idle) sf status is an agent running, and for which account sf logout [--everywhere] revoke the session and stop the agent; --everywhere signs out every device -sf push encrypt and upload every file once +sf push [--include-dotfiles] encrypt and upload every file once (skips dotfiles unless asked; never follows symlinks) sf sync [--watch] [--interval=N] upload new/changed files; --watch keeps a poll loop (N seconds, default 5) sf verify ... report whether each file is safely stored in your vault sf encrypt encrypt each file locally, upload nothing (no account needed) @@ -89,7 +100,14 @@ sf inspect read-only recursive migration inven ``` `sf push` and `sf sync` pick the direct or multipart upload path automatically -from the file size. +from the file size. Both read only regular files directly inside the folder: +symlinks are skipped (and files are opened with `O_NOFOLLOW`), and names +starting with `.` are skipped unless you pass `sf push --include-dotfiles`. +An empty (0-byte) file is uploaded only when the server says it accepts empty +files; otherwise it is reported as skipped and the exit code is 2. If an upload +fails after its session was created, the pending session is cancelled so it +does not hold quota. `sf sync` writes its manifest (`.shieldfive-sync.json`) +with mode 0600. `sf inspect` reports JSON counts of files, directories, bytes and skipped entries. It includes nested/hidden files and empty folders, reads metadata only, and diff --git a/package.json b/package.json index 86e13fb..608e37a 100644 --- a/package.json +++ b/package.json @@ -2,7 +2,7 @@ "name": "@shieldfive/cli", "version": "0.3.1", "type": "module", - "description": "Open-source command-line client for ShieldFive: encrypt a folder on your machine and sync it to your EU-hosted vault with post-quantum hybrid encryption (ML-KEM-1024 + X25519). Read the code; it encrypts before upload.", + "description": "Open-source command-line client for ShieldFive: encrypts files on your machine before upload (contents: AES-256-GCM under an ML-KEM-1024 + X25519 hybrid key; names: AES-256-GCM under your vault key) and pushes or syncs them to your vault. Read the code.", "license": "Apache-2.0", "author": "Cho Garcia ", "homepage": "https://github.com/shieldfive/cli", diff --git a/src/agent/server.mjs b/src/agent/server.mjs index e2f5a3c..2110440 100644 --- a/src/agent/server.mjs +++ b/src/agent/server.mjs @@ -5,7 +5,7 @@ // See docs/agent-design.md for why it exists and what it will never do. import { createHmac, randomUUID } from 'node:crypto' -import { chmod, open, readFile, rm, stat } from 'node:fs/promises' +import { chmod, lstat, open, readFile, rm, stat } from 'node:fs/promises' import { connect, createServer } from 'node:net' import { basename, dirname, join } from 'node:path' @@ -189,7 +189,9 @@ export function createAgent({ } async function upload(path) { - const before = await stat(path) + // lstat: a symlink is refused, not followed (and readFileChunks opens with + // O_NOFOLLOW, so a swap after this check fails too). + const before = await lstat(path) if (!before.isFile()) throw new AgentError('not_a_file', `${path} is not a regular file.`) const token = await accessToken() diff --git a/src/cli.mjs b/src/cli.mjs index 973365f..d392290 100644 --- a/src/cli.mjs +++ b/src/cli.mjs @@ -15,7 +15,7 @@ // signed in as another. See docs/agent-design.md. import { webcrypto } from 'node:crypto' -import { readFile, readdir, stat } from 'node:fs/promises' +import { lstat, readFile, readdir } from 'node:fs/promises' import { join, resolve } from 'node:path' import { promptHidden, promptLine } from './prompt.mjs' @@ -31,6 +31,8 @@ function usage() { ' logout [--everywhere] revoke the session and stop the agent', ' --everywhere also signs out your browser and phone', ' push encrypt and upload every file in ', + ' [--include-dotfiles] also upload names starting with "."', + ' (symlinks are never followed)', ' sync [--watch] upload new and changed files', ' [--interval=] poll interval for --watch (default 5)', ' verify ... report whether each file is safely in your vault', @@ -52,10 +54,15 @@ function usage() { ) } -async function* filesIn(folder) { +// Regular files directly in `folder`. lstat, not stat: a symlink is skipped +// rather than followed, so `sf push ~/project` cannot upload ~/.ssh/id_ed25519 +// through a link that points out of the folder. Dotfiles (.env, .git, .DS_Store, +// sync's own manifest) are skipped unless includeDotfiles is set, matching sync. +async function* filesIn(folder, { includeDotfiles = false } = {}) { for (const name of await readdir(folder)) { + if (!includeDotfiles && name.startsWith('.')) continue const path = join(folder, name) - const info = await stat(path) + const info = await lstat(path) if (info.isFile()) yield { name, path, size: info.size } } } @@ -397,8 +404,16 @@ async function cmdPush(folder, flags) { const { request } = await import('./agent/client.mjs') process.stderr.write(`pushing to account ${agent.account} through the agent\n`) let differs = 0 - for await (const { name, path } of filesIn(resolve(folder))) { - const result = await request('upload', { path }, { timeoutMs: 0 }) + for await (const { name, path } of filesIn(resolve(folder), pushListOptions(flags))) { + let result + try { + result = await request('upload', { path }, { timeoutMs: 0 }) + } catch (err) { + if (err.code !== 'empty_file_unsupported') throw err + process.stdout.write(`skipped ${name}: ${err.message}\n`) + differs++ + continue + } if (result.matchesFileNow) { process.stdout.write(`pushed ${name}\n`) } else { @@ -419,17 +434,30 @@ async function cmdPush(folder, flags) { const { uploadFile } = await import('./upload.mjs') const { accessToken, rootKey } = await authAndUnlock(cfg) - for await (const { name, path, size } of filesIn(folder)) { - await uploadFile({ - apiBaseUrl: cfg.apiBaseUrl, - accessToken, - rootKey, - name, - path, - size, - }) + let skipped = 0 + for await (const { name, path, size } of filesIn(folder, pushListOptions(flags))) { + try { + await uploadFile({ + apiBaseUrl: cfg.apiBaseUrl, + accessToken, + rootKey, + name, + path, + size, + }) + } catch (err) { + if (err.code !== 'empty_file_unsupported') throw err + process.stdout.write(`skipped ${name}: ${err.message}\n`) + skipped++ + continue + } process.stdout.write(`pushed ${name}\n`) } + if (skipped) process.exitCode = 2 +} + +function pushListOptions(flags) { + return { includeDotfiles: flags['include-dotfiles'] === true } } async function syncViaAgent(folder, { watch, intervalMs }, agent) { diff --git a/src/sync.mjs b/src/sync.mjs index b12a7ba..02b091f 100644 --- a/src/sync.mjs +++ b/src/sync.mjs @@ -17,7 +17,7 @@ // app. Replacing the previous fileId needs the Bin's folder key to re-wrap it, // which this client does not hold yet. -import { open, readFile, readdir, rename, stat } from 'node:fs/promises' +import { lstat, open, readFile, readdir, rename } from 'node:fs/promises' import { join } from 'node:path' export const MANIFEST_NAME = '.shieldfive-sync.json' @@ -28,7 +28,8 @@ export function signatureOf({ size, mtimeMs }) { return `${size}:${mtimeMs}` } -// List regular files to consider for sync: skips directories, dotfiles (which +// List regular files to consider for sync: skips directories, symlinks (lstat, +// so a link never pulls in a file from outside the folder), dotfiles (which // includes the manifest itself), and any explicitly excluded names. export async function scanFiles(folder, { exclude = [] } = {}) { const skip = new Set(exclude) @@ -36,7 +37,7 @@ export async function scanFiles(folder, { exclude = [] } = {}) { for (const name of await readdir(folder)) { if (name.startsWith('.') || skip.has(name)) continue const path = join(folder, name) - const info = await stat(path) + const info = await lstat(path) if (info.isFile()) { entries.push({ name, path, size: info.size, mtimeMs: info.mtimeMs }) } @@ -60,7 +61,10 @@ export async function loadManifest(manifestPath) { } // Atomic-ish write: write a temp sibling then rename over the target, so a crash -// during the write can't leave a half-written manifest. +// during the write can't leave a half-written manifest. Mode 0600: the manifest +// lists every synced file name with its vault file id, which is nobody else's +// business on a shared machine. chmod as well as the open mode, because a temp +// file left by a crashed run keeps whatever mode it had. export async function saveManifest(manifestPath, manifest) { const body = JSON.stringify( { version: MANIFEST_VERSION, files: manifest.files }, @@ -68,8 +72,9 @@ export async function saveManifest(manifestPath, manifest) { 2, ) const tmp = `${manifestPath}.tmp` - const handle = await open(tmp, 'w') + const handle = await open(tmp, 'w', 0o600) try { + await handle.chmod(0o600) await handle.writeFile(body) await handle.sync() } finally { diff --git a/src/upload.mjs b/src/upload.mjs index faaad93..16141e9 100644 --- a/src/upload.mjs +++ b/src/upload.mjs @@ -17,6 +17,7 @@ // also carries the header. The header's file_id is the row id the server // assigned at create-upload-session, so the stored object is bound to its row. +import { constants as fsConstants } from 'node:fs' import { open } from 'node:fs/promises' import { encryptNameV6 } from '@shieldfive/crypto/vault' @@ -25,6 +26,7 @@ import { base64ToBytes, bytesToBase64, CIPHER_VERSION_PQ, + computeV3EmptyUploadProof, computeV3UploadProof, encryptMetadataV4, encryptPartsV3, @@ -41,6 +43,8 @@ import { const CREATE_SESSION_PATH = '/api/files/create-upload-session' const COMPLETE_PATH = '/api/files/complete-upload' const PART_URL_PATH = '/api/files/upload-part-url' +const DELETE_PATH = '/api/files/delete' +const CAPABILITIES_PATH = '/api/files/upload-capabilities' // Backblaze documents retrying b2_upload_part on 5xx / network failures, and a // part upload is idempotent by (part number, SHA-1) — retrying overwrites the @@ -56,8 +60,16 @@ const partRetryDelayMs = (attempt) => // Read a file in fixed-size chunks without holding it all in memory. Each yield // is an independent copy (the read buffer is reused), so the caller may retain // or transform a chunk across the next read safely. +// +// O_NOFOLLOW: callers list files with lstat and skip symlinks, but a regular +// file could be swapped for a symlink between that check and this open. With +// O_NOFOLLOW the open fails (ELOOP) instead of uploading whatever the link +// points at, possibly outside the folder being pushed. (Undefined on Windows, +// where it is simply not applied.) +const OPEN_READ_NOFOLLOW = fsConstants.O_RDONLY | (fsConstants.O_NOFOLLOW ?? 0) + export async function* readFileChunks(path, chunkSize) { - const handle = await open(path, 'r') + const handle = await open(path, OPEN_READ_NOFOLLOW) try { const buffer = Buffer.allocUnsafe(chunkSize) for (;;) { @@ -274,14 +286,22 @@ async function finalizeUpload({ ciphertextHash, nameEncryptedV6, }) { - const res = await fetch(new URL(COMPLETE_PATH, apiBaseUrl), { - method: 'POST', - headers: { - Authorization: `Bearer ${accessToken}`, - 'Content-Type': 'application/json', - }, - body: JSON.stringify({ fileId, b2FileId, partSha1Array, proof, ciphertextHash, nameEncryptedV6 }), - }) + let res + try { + res = await fetch(new URL(COMPLETE_PATH, apiBaseUrl), { + method: 'POST', + headers: { + Authorization: `Bearer ${accessToken}`, + 'Content-Type': 'application/json', + }, + body: JSON.stringify({ fileId, b2FileId, partSha1Array, proof, ciphertextHash, nameEncryptedV6 }), + }) + } catch (err) { + // The request may have reached the server and finalized the file; only the + // answer was lost. Cancelling now could bin a file that is in the vault. + err.finalizeOutcomeUnknown = true + throw err + } const body = await res.json().catch(() => ({})) if (!res.ok) { throw new Error( @@ -294,7 +314,7 @@ async function finalizeUpload({ // Single-chunk file: one part (header || frame_0), SHA-1 + proof, PUT it at // the presigned URL, finalize. No b2FileId is sent -- the server HEADs the // stored object and derives the storage id itself. -async function uploadDirect({ apiBaseUrl, accessToken, session, parts, nameEncryptedV6 }) { +async function uploadDirect({ apiBaseUrl, accessToken, session, parts, nameEncryptedV6, empty = false }) { let ciphertext = null for await (const part of parts) { if (ciphertext !== null) { @@ -303,8 +323,12 @@ async function uploadDirect({ apiBaseUrl, accessToken, session, parts, nameEncry ciphertext = part } if (ciphertext === null) throw new Error('Direct upload produced no ciphertext.') + // For a 0-byte file part 1 is the header alone, and the proof covers it: + // the server's empty-file contract is HMAC(proofKey, [3][3] || header). const sha1 = await sha1Hex(ciphertext) - const proof = await computeV3UploadProof({ proofKeyHex: session.proofKey, part0: ciphertext }) + const proof = empty + ? computeV3EmptyUploadProof({ proofKeyHex: session.proofKey, header: ciphertext }) + : await computeV3UploadProof({ proofKeyHex: session.proofKey, part0: ciphertext }) await putDirectPresigned({ session, ciphertext }) @@ -444,6 +468,55 @@ async function uploadMultipart({ apiBaseUrl, accessToken, session, parts, nameEn }) } +// Best-effort cancel of a session that will never finalize. The pending row +// otherwise holds quota (and, for multipart, an unfinished Backblaze large +// file) until the server's stale-upload reaper gets to it. POST +// /api/files/delete with the row id is what the web app's cancel path uses: +// for a non-ready row it cancels the large file and drops the metadata. Never +// throws; a failure here must not hide the error that caused it. +export async function cancelUploadSession({ apiBaseUrl, accessToken, fileId }) { + try { + const res = await fetch(new URL(DELETE_PATH, apiBaseUrl), { + method: 'POST', + headers: { + Authorization: `Bearer ${accessToken}`, + 'Content-Type': 'application/json', + }, + body: JSON.stringify({ ids: [fileId] }), + }) + return res.ok + } catch { + return false + } +} + +export class EmptyFileUnsupportedError extends Error { + constructor(name) { + super( + `"${name}" is empty (0 bytes) and this ShieldFive server does not accept ` + + 'empty files yet. Nothing was uploaded for it.', + ) + this.name = 'EmptyFileUnsupportedError' + this.code = 'empty_file_unsupported' + } +} + +// Zero-byte cipher-v3 uploads are behind a server capability (web +// docs/desktop-empty-file-contract.md). Ask before creating a session, so an +// older or flag-off server gets no pending row at all. +async function serverAcceptsEmptyFiles({ apiBaseUrl, accessToken }) { + try { + const res = await fetch(new URL(CAPABILITIES_PATH, apiBaseUrl), { + headers: { Authorization: `Bearer ${accessToken}` }, + }) + if (!res.ok) return false + const body = await res.json().catch(() => ({})) + return body?.emptyCipherV3 === true + } catch { + return false + } +} + // `onPlaintextChunk`, when given, is called with every plaintext chunk in file // order, from the same read that feeds encryption. The agent hashes those // chunks to record what was actually uploaded; hashing the file separately, @@ -452,6 +525,9 @@ export async function uploadFile({ apiBaseUrl, accessToken, rootKey, name, path, // Derived before anything is sent, so a vault whose identity cannot be // derived fails before a session (and a pending row) exists. const recipientPublicKey = await vaultMlKemPublicKey(rootKey) + if (size === 0 && !(await serverAcceptsEmptyFiles({ apiBaseUrl, accessToken }))) { + throw new EmptyFileUnsupportedError(name) + } const { session, csk } = await createUploadSession({ apiBaseUrl, accessToken, @@ -459,6 +535,24 @@ export async function uploadFile({ apiBaseUrl, accessToken, rootKey, name, path, name, size, }) + // From here on a pending row exists. Any failure before finalize answers + // cancels it (see cancelUploadSession). + const envelopeKey = base64ToBytes(csk) + try { + await sendUpload({ apiBaseUrl, accessToken, name, path, size, onPlaintextChunk, session, envelopeKey, rootKey, recipientPublicKey }) + } catch (err) { + if (!err?.finalizeOutcomeUnknown && session?.fileId) { + await cancelUploadSession({ apiBaseUrl, accessToken, fileId: session.fileId }) + } + throw err + } finally { + envelopeKey.fill(0) + } + + return { fileId: session.fileId } +} + +async function sendUpload({ apiBaseUrl, accessToken, name, path, size, onPlaintextChunk, session, envelopeKey, rootKey, recipientPublicKey }) { if (!session.proofKey) throw new Error('Upload session is missing its proof key.') // The server's v1 chunk size (5 MiB): each encrypted chunk is one storage part. @@ -469,7 +563,6 @@ export async function uploadFile({ apiBaseUrl, accessToken, rootKey, name, path, yield chunk } } - const envelopeKey = base64ToBytes(csk) const parts = encryptPartsV3({ plaintextChunks: plaintextChunks(), plaintextSize: size, @@ -484,28 +577,22 @@ export async function uploadFile({ apiBaseUrl, accessToken, rootKey, name, path, await encryptNameV6({ name, folderKey: rootKey, rowId: session.fileId }), ) - try { - if (session.uploadKind === 'direct') { - // No authToken and no storagePath check: since web b52b4c8 (PR #726) the - // direct path is a presigned S3 PUT, so the signature inside uploadUrl is - // the only credential and it already binds the object key. Requiring the - // token the server stopped issuing is what made every <= 5 MiB upload fail - // from 2026-09-03. The 'large' branch below still needs both. - if (!session.uploadUrl) { - throw new Error('Direct upload session is missing an upload URL.') - } - await uploadDirect({ apiBaseUrl, accessToken, session, parts, nameEncryptedV6 }) - } else if (session.uploadKind === 'large') { - if (!session.uploadUrl || !session.authToken) { - throw new Error('Multipart upload session is missing credentials.') - } - await uploadMultipart({ apiBaseUrl, accessToken, session, parts, nameEncryptedV6 }) - } else { - throw new Error(`Unknown uploadKind "${session.uploadKind}" for "${name}".`) + if (session.uploadKind === 'direct') { + // No authToken and no storagePath check: since web b52b4c8 (PR #726) the + // direct path is a presigned S3 PUT, so the signature inside uploadUrl is + // the only credential and it already binds the object key. Requiring the + // token the server stopped issuing is what made every <= 5 MiB upload fail + // from 2026-09-03. The 'large' branch below still needs both. + if (!session.uploadUrl) { + throw new Error('Direct upload session is missing an upload URL.') } - } finally { - envelopeKey.fill(0) + await uploadDirect({ apiBaseUrl, accessToken, session, parts, nameEncryptedV6, empty: size === 0 }) + } else if (session.uploadKind === 'large') { + if (!session.uploadUrl || !session.authToken) { + throw new Error('Multipart upload session is missing credentials.') + } + await uploadMultipart({ apiBaseUrl, accessToken, session, parts, nameEncryptedV6 }) + } else { + throw new Error(`Unknown uploadKind "${session.uploadKind}" for "${name}".`) } - - return { fileId: session.fileId } } diff --git a/src/uploadCrypto.mjs b/src/uploadCrypto.mjs index 435dba5..ec1103f 100644 --- a/src/uploadCrypto.mjs +++ b/src/uploadCrypto.mjs @@ -4,7 +4,7 @@ // default format, written by @shieldfive/crypto (see the end of this file). Same @noble versions as the apps // (@noble/ciphers@2, @noble/hashes@2) so the bytes match. -import { webcrypto } from 'node:crypto' +import { createHmac, webcrypto } from 'node:crypto' import { gcm } from '@noble/ciphers/aes.js' import { argon2id } from '@noble/hashes/argon2.js' @@ -282,7 +282,13 @@ export async function* encryptPartsV3({ } const header = take(headerLength) const first = await readFrame() - if (!first) throw new Error('Encrypted stream has no chunks.') + if (!first) { + // A 0-byte file is a header with no chunk frames (total_chunks 0); it is + // uploaded as a single header-only part. Anything else is a bug. + if (plaintextSize !== 0) throw new Error('Encrypted stream has no chunks.') + yield header + return + } const part0 = new Uint8Array(header.length + first.length) part0.set(header, 0) part0.set(first, header.length) @@ -302,3 +308,21 @@ export async function* encryptPartsV3({ export function computeV3UploadProof({ proofKeyHex, part0 }) { return buildUploadProofV3({ proofKeyHex, ciphertext: part0 }) } + +/** + * Upload proof for a 0-byte file, whose only part is the header. The library's + * buildUploadProofV3 requires a first chunk frame, which an empty file does not + * have; the server's empty-file contract (web docs/desktop-empty-file-contract.md, + * emptyFileUploadProof) is base64([3][3] || HMAC-SHA256(proofKey, [3][3] || header)). + */ +export function computeV3EmptyUploadProof({ proofKeyHex, header }) { + if (typeof proofKeyHex !== 'string' || !/^[0-9a-f]{64}$/i.test(proofKeyHex)) { + throw new Error('Upload session proof key is not 32 bytes of hex.') + } + const prefix = Buffer.from([3, 3]) + const mac = createHmac('sha256', Buffer.from(proofKeyHex, 'hex')) + .update(prefix) + .update(header) + .digest() + return Buffer.concat([prefix, mac]).toString('base64') +} diff --git a/test/upload.hardening.test.mjs b/test/upload.hardening.test.mjs new file mode 100644 index 0000000..de6abac --- /dev/null +++ b/test/upload.hardening.test.mjs @@ -0,0 +1,196 @@ +// Batch I hardening: cancel a pending session on failure, 0-byte files, no +// symlink following, manifest mode 0600, dotfiles skipped by push. + +import assert from 'node:assert/strict' +import { execFile } from 'node:child_process' +import { createHmac, webcrypto } from 'node:crypto' +import { mkdtemp, rm, stat, symlink, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { test } from 'node:test' +import { fileURLToPath } from 'node:url' +import { promisify } from 'node:util' + +import { parseHeader } from '@shieldfive/crypto/format' + +import { readFileChunks, uploadFile } from '../src/upload.mjs' +import { saveManifest, scanFiles } from '../src/sync.mjs' + +const API = 'https://api.test' +const PROOF_KEY = 'cd'.repeat(32) +const FILE_ID = '5a1c2a9e-7b4d-4c1e-9a2f-0d6e8b5c4a72' + +function jsonResponse(status, body) { + return { ok: status >= 200 && status < 300, status, json: async () => body, text: async () => JSON.stringify(body) } +} + +async function tempDir(fn) { + const dir = await mkdtemp(join(tmpdir(), 'sf-cli-hard-')) + try { + return await fn(dir) + } finally { + await rm(dir, { recursive: true, force: true }) + } +} + +async function withFetch(handler, fn) { + const original = globalThis.fetch + globalThis.fetch = async (input, init = {}) => handler(typeof input === 'string' ? input : input.toString(), init) + try { + return await fn() + } finally { + globalThis.fetch = original + } +} + +const rootKey = () => webcrypto.getRandomValues(new Uint8Array(32)) + +test('a failed upload cancels its pending session', async () => { + const deletes = [] + await tempDir(async (dir) => { + const path = join(dir, 'a.bin') + await writeFile(path, 'hello') + await withFetch( + (url, init) => { + if (url === `${API}/api/files/create-upload-session`) { + return jsonResponse(200, { uploadKind: 'direct', fileId: FILE_ID, uploadUrl: 'https://s3.test/put', proofKey: PROOF_KEY }) + } + if (url === 'https://s3.test/put') return jsonResponse(500, { error: 'boom' }) + if (url === `${API}/api/files/delete`) { + deletes.push({ body: JSON.parse(init.body), auth: init.headers.Authorization }) + return jsonResponse(200, {}) + } + throw new Error(`unexpected fetch: ${url}`) + }, + () => + assert.rejects( + uploadFile({ apiBaseUrl: API, accessToken: 'tok', rootKey: rootKey(), name: 'a.bin', path, size: 5 }), + /Direct upload failed \(HTTP 500\)/, + ), + ) + }) + assert.deepEqual(deletes, [{ body: { ids: [FILE_ID] }, auth: 'Bearer tok' }]) +}) + +test('a lost finalize answer does NOT cancel (the file may already be in the vault)', async () => { + let deleted = false + await tempDir(async (dir) => { + const path = join(dir, 'a.bin') + await writeFile(path, 'hello') + await withFetch( + (url) => { + if (url === `${API}/api/files/create-upload-session`) { + return jsonResponse(200, { uploadKind: 'direct', fileId: FILE_ID, uploadUrl: 'https://s3.test/put', proofKey: PROOF_KEY }) + } + if (url === 'https://s3.test/put') return jsonResponse(200, {}) + if (url === `${API}/api/files/complete-upload`) throw new TypeError('fetch failed') + if (url === `${API}/api/files/delete`) { + deleted = true + return jsonResponse(200, {}) + } + throw new Error(`unexpected fetch: ${url}`) + }, + () => assert.rejects(uploadFile({ apiBaseUrl: API, accessToken: 'tok', rootKey: rootKey(), name: 'a.bin', path, size: 5 })), + ) + }) + assert.equal(deleted, false) +}) + +test('0-byte file: refused before any session when the server lacks the capability', async () => { + const calls = [] + await tempDir(async (dir) => { + const path = join(dir, 'empty') + await writeFile(path, '') + await withFetch( + (url) => { + calls.push(url) + if (url === `${API}/api/files/upload-capabilities`) return jsonResponse(200, { version: 1, emptyCipherV3: false }) + throw new Error(`unexpected fetch: ${url}`) + }, + () => + assert.rejects( + uploadFile({ apiBaseUrl: API, accessToken: 'tok', rootKey: rootKey(), name: 'empty', path, size: 0 }), + (err) => err.code === 'empty_file_unsupported', + ), + ) + }) + assert.deepEqual(calls, [`${API}/api/files/upload-capabilities`]) +}) + +test('0-byte file: header-only part and the server empty-file proof', async () => { + let put = null + let finalize = null + let createBody = null + await tempDir(async (dir) => { + const path = join(dir, 'empty') + await writeFile(path, '') + await withFetch( + (url, init) => { + if (url === `${API}/api/files/upload-capabilities`) return jsonResponse(200, { version: 1, emptyCipherV3: true }) + if (url === `${API}/api/files/create-upload-session`) { + createBody = JSON.parse(init.body) + return jsonResponse(200, { uploadKind: 'direct', fileId: FILE_ID, uploadUrl: 'https://s3.test/put', proofKey: PROOF_KEY }) + } + if (url === 'https://s3.test/put') { + put = new Uint8Array(init.body) + return jsonResponse(200, {}) + } + if (url === `${API}/api/files/complete-upload`) { + finalize = JSON.parse(init.body) + return jsonResponse(200, { ok: true }) + } + throw new Error(`unexpected fetch: ${url}`) + }, + () => uploadFile({ apiBaseUrl: API, accessToken: 'tok', rootKey: rootKey(), name: 'empty', path, size: 0 }), + ) + }) + assert.equal(createBody.sizeBytes, 0) + const header = parseHeader(put) + assert.equal(header.headerLength, put.length, 'part is the header alone') + assert.equal(header.totalChunks, 0) + assert.equal(header.plaintextSize, 0) + const prefix = Buffer.from([3, 3]) + const mac = createHmac('sha256', Buffer.from(PROOF_KEY, 'hex')).update(prefix).update(put).digest() + assert.equal(finalize.proof, Buffer.concat([prefix, mac]).toString('base64')) + assert.equal(finalize.partSha1Array.length, 1) +}) + +test('sync scan skips symlinks and dotfiles; reading through a symlink fails', async () => { + await tempDir(async (dir) => { + await writeFile(join(dir, 'real.txt'), 'x') + await writeFile(join(dir, '.env'), 'SECRET=1') + const outside = await mkdtemp(join(tmpdir(), 'sf-cli-outside-')) + try { + await writeFile(join(outside, 'id_ed25519'), 'private') + await symlink(join(outside, 'id_ed25519'), join(dir, 'link')) + const names = (await scanFiles(dir)).map((e) => e.name) + assert.deepEqual(names, ['real.txt']) + await assert.rejects(async () => { + for await (const _ of readFileChunks(join(dir, 'link'), 8)) void _ + }, (err) => err.code === 'ELOOP' || err.code === 'EMLINK') + } finally { + await rm(outside, { recursive: true, force: true }) + } + }) +}) + +test('sync manifest is written with mode 0600', { skip: process.platform === 'win32' }, async () => { + await tempDir(async (dir) => { + const p = join(dir, '.shieldfive-sync.json') + await writeFile(`${p}.tmp`, 'stale', { mode: 0o644 }) + await saveManifest(p, { files: {} }) + assert.equal((await stat(p)).mode & 0o777, 0o600) + }) +}) + +test('push/encrypt listing skips dotfiles and symlinks', async () => { + const cli = fileURLToPath(new URL('../src/cli.mjs', import.meta.url)) + await tempDir(async (dir) => { + await writeFile(join(dir, 'doc.txt'), 'hello') + await writeFile(join(dir, '.env'), 'SECRET=1') + await symlink(join(dir, 'doc.txt'), join(dir, 'alias.txt')) + const { stdout } = await promisify(execFile)(process.execPath, [cli, 'encrypt', dir]) + assert.match(stdout, /encrypted doc\.txt/) + assert.doesNotMatch(stdout, /\.env|alias\.txt/) + }) +})