fix: cancel failed sessions, 0-byte files, no symlinks, 0600 manifest, push skips dotfiles

CHANGELOG.md

@@ -9,6 +9,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Security
 
+- **Symlinks are not followed.** `push`, `sync` and the agent list files with
+  `lstat` and skip symlinks, and files are opened with `O_NOFOLLOW`, so a link
+  inside the folder can no longer upload a file from outside it.
+- **`push` skips dotfiles by default** (`.env`, `.git`, ...), as `sync` already
+  did. `sf push --include-dotfiles` restores the old behaviour.
+- **Sync manifest written 0600.** `.shieldfive-sync.json` lists file names and
+  vault ids; it is no longer world-readable.
+- **README and npm description** now state the scheme precisely: contents are
+  AES-256-GCM under a post-quantum hybrid key, names are AES-256-GCM under the
+  vault key, and size, part hashes and timestamps are visible to the server.
+
+### Fixed
+
+- A failed upload cancels its pending session (`POST /api/files/delete`, as the
+  web app does) instead of leaving a row holding quota until the reaper runs.
+  Not done when only the finalize answer was lost, since the file may be in.
+- 0-byte files: uploaded as a header-only cipher-v3 object with the server's
+  empty-file proof when `/api/files/upload-capabilities` says `emptyCipherV3`;
+  otherwise skipped with a clear message (exit code 2) before any session.
+
 - **Downgrade pin for sign-in.** After a sign-in with a login secret (auth
   version 2), the CLI records the account (a SHA-256 of the email) in
   `~/.shieldfive/auth-pins.json` (mode 0600). A later server answer of

README.md

@@ -1,24 +1,35 @@
 # @shieldfive/cli
 
 > Command-line client for [ShieldFive](https://shieldfive.com). Encrypt a folder
-> on your machine, then push or continuously sync it to your EU-hosted,
-> post-quantum vault. Everything is encrypted locally before it leaves your
-> machine. Read the code and verify that for yourself.
+> on your machine, then push or continuously sync it to your EU-hosted vault.
+> File contents and names are encrypted locally before they leave your machine
+> (contents with a post-quantum hybrid key). Read the code and verify that for
+> yourself.
 
 [![CI](https://github.com/shieldfive/cli/actions/workflows/ci.yml/badge.svg)](https://github.com/shieldfive/cli/actions/workflows/ci.yml)
 [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)
 [![Status](https://img.shields.io/badge/status-early-yellow.svg)](#honest-scope)
 
 The point of this tool being open source is that you do not have to trust a
-marketing claim. The plaintext of your files is never sent to a ShieldFive
-server. The only bytes that leave your machine are ciphertext (post-quantum
-hybrid: ML-KEM-1024 + X25519 key encapsulation, AES-256-GCM chunks) and a short
-cryptographic proof. Two files carry the whole story:
+marketing claim. The plaintext of your files and their names is never sent to a
+ShieldFive server:
+
+- **File contents** — AES-256-GCM chunks under a per-file key that combines a
+  random envelope key (wrapped by your vault root key) with an ML-KEM-1024 +
+  X25519 hybrid encapsulation to your vault identity.
+- **File names** — AES-256-GCM under a key derived from your vault root key
+  (not the post-quantum layer), plus a keyed HMAC of the lowercased name so the
+  server can spot duplicates without reading it.
+
+What the server does see, by design: the plaintext size of each file, part
+hashes (SHA-1 of ciphertext), the wrapped keys, an upload proof, timestamps and
+your account. Sign-in sends a secret derived from your password, never the
+password itself. Two files carry the whole story:
 
 - [`src/uploadCrypto.mjs`](src/uploadCrypto.mjs) — the encryption applied to
   every chunk and to the filename.
-- [`src/upload.mjs`](src/upload.mjs) — the only place bytes are sent over the
-  network. Every request body is ciphertext.
+- [`src/upload.mjs`](src/upload.mjs) — the only place file bytes are sent over
+  the network. Every file body it sends is ciphertext.
 
 If you want to see encryption happen without an account at all, run
 `sf encrypt <folder>` — it encrypts each file locally and prints the ciphertext
@@ -81,15 +92,22 @@ npm test        # 107 tests
 sf login [--idle=<hours>]                    sign in once; an agent holds the session in memory (locks after 8 h idle)
 sf status                                    is an agent running, and for which account
 sf logout [--everywhere]                     revoke the session and stop the agent; --everywhere signs out every device
-sf push <folder>                             encrypt and upload every file once
+sf push <folder> [--include-dotfiles]        encrypt and upload every file once (skips dotfiles unless asked; never follows symlinks)
 sf sync <folder> [--watch] [--interval=N]    upload new/changed files; --watch keeps a poll loop (N seconds, default 5)
 sf verify <file>...                          report whether each file is safely stored in your vault
 sf encrypt <folder>                          encrypt each file locally, upload nothing (no account needed)
 sf inspect <folder>                          read-only recursive migration inventory; upload nothing
 ```
 
 `sf push` and `sf sync` pick the direct or multipart upload path automatically
-from the file size.
+from the file size. Both read only regular files directly inside the folder:
+symlinks are skipped (and files are opened with `O_NOFOLLOW`), and names
+starting with `.` are skipped unless you pass `sf push --include-dotfiles`.
+An empty (0-byte) file is uploaded only when the server says it accepts empty
+files; otherwise it is reported as skipped and the exit code is 2. If an upload
+fails after its session was created, the pending session is cancelled so it
+does not hold quota. `sf sync` writes its manifest (`.shieldfive-sync.json`)
+with mode 0600.
 
 `sf inspect` reports JSON counts of files, directories, bytes and skipped entries.
 It includes nested/hidden files and empty folders, reads metadata only, and

package.json

@@ -2,7 +2,7 @@
   "name": "@shieldfive/cli",
   "version": "0.3.1",
   "type": "module",
-  "description": "Open-source command-line client for ShieldFive: encrypt a folder on your machine and sync it to your EU-hosted vault with post-quantum hybrid encryption (ML-KEM-1024 + X25519). Read the code; it encrypts before upload.",
+  "description": "Open-source command-line client for ShieldFive: encrypts files on your machine before upload (contents: AES-256-GCM under an ML-KEM-1024 + X25519 hybrid key; names: AES-256-GCM under your vault key) and pushes or syncs them to your vault. Read the code.",
   "license": "Apache-2.0",
   "author": "Cho Garcia <security@shieldfive.com>",
   "homepage": "https://github.com/shieldfive/cli",

src/agent/server.mjs

@@ -5,7 +5,7 @@
 // See docs/agent-design.md for why it exists and what it will never do.
 
 import { createHmac, randomUUID } from 'node:crypto'
-import { chmod, open, readFile, rm, stat } from 'node:fs/promises'
+import { chmod, lstat, open, readFile, rm, stat } from 'node:fs/promises'
 import { connect, createServer } from 'node:net'
 import { basename, dirname, join } from 'node:path'
 
@@ -189,7 +189,9 @@ export function createAgent({
   }
 
   async function upload(path) {
-    const before = await stat(path)
+    // lstat: a symlink is refused, not followed (and readFileChunks opens with
+    // O_NOFOLLOW, so a swap after this check fails too).
+    const before = await lstat(path)
     if (!before.isFile()) throw new AgentError('not_a_file', `${path} is not a regular file.`)
     const token = await accessToken()
 

src/cli.mjs

@@ -15,7 +15,7 @@
 // signed in as another. See docs/agent-design.md.
 
 import { webcrypto } from 'node:crypto'
-import { readFile, readdir, stat } from 'node:fs/promises'
+import { lstat, readFile, readdir } from 'node:fs/promises'
 import { join, resolve } from 'node:path'
 
 import { promptHidden, promptLine } from './prompt.mjs'
@@ -31,6 +31,8 @@ function usage() {
       '  logout [--everywhere]       revoke the session and stop the agent',
       '                              --everywhere also signs out your browser and phone',
       '  push <folder>               encrypt and upload every file in <folder>',
+      '                              [--include-dotfiles]  also upload names starting with "."',
+      '                              (symlinks are never followed)',
       '  sync <folder> [--watch]     upload new and changed files',
       '                              [--interval=<seconds>]  poll interval for --watch (default 5)',
       '  verify <file>...            report whether each file is safely in your vault',
@@ -52,10 +54,15 @@ function usage() {
   )
 }
 
-async function* filesIn(folder) {
+// Regular files directly in `folder`. lstat, not stat: a symlink is skipped
+// rather than followed, so `sf push ~/project` cannot upload ~/.ssh/id_ed25519
+// through a link that points out of the folder. Dotfiles (.env, .git, .DS_Store,
+// sync's own manifest) are skipped unless includeDotfiles is set, matching sync.
+async function* filesIn(folder, { includeDotfiles = false } = {}) {
   for (const name of await readdir(folder)) {
+    if (!includeDotfiles && name.startsWith('.')) continue
     const path = join(folder, name)
-    const info = await stat(path)
+    const info = await lstat(path)
     if (info.isFile()) yield { name, path, size: info.size }
   }
 }
@@ -397,8 +404,16 @@ async function cmdPush(folder, flags) {
     const { request } = await import('./agent/client.mjs')
     process.stderr.write(`pushing to account ${agent.account} through the agent\n`)
     let differs = 0
-    for await (const { name, path } of filesIn(resolve(folder))) {
-      const result = await request('upload', { path }, { timeoutMs: 0 })
+    for await (const { name, path } of filesIn(resolve(folder), pushListOptions(flags))) {
+      let result
+      try {
+        result = await request('upload', { path }, { timeoutMs: 0 })
+      } catch (err) {
+        if (err.code !== 'empty_file_unsupported') throw err
+        process.stdout.write(`skipped ${name}: ${err.message}\n`)
+        differs++
+        continue
+      }
       if (result.matchesFileNow) {
         process.stdout.write(`pushed ${name}\n`)
       } else {
@@ -419,17 +434,30 @@ async function cmdPush(folder, flags) {
   const { uploadFile } = await import('./upload.mjs')
   const { accessToken, rootKey } = await authAndUnlock(cfg)
 
-  for await (const { name, path, size } of filesIn(folder)) {
-    await uploadFile({
-      apiBaseUrl: cfg.apiBaseUrl,
-      accessToken,
-      rootKey,
-      name,
-      path,
-      size,
-    })
+  let skipped = 0
+  for await (const { name, path, size } of filesIn(folder, pushListOptions(flags))) {
+    try {
+      await uploadFile({
+        apiBaseUrl: cfg.apiBaseUrl,
+        accessToken,
+        rootKey,
+        name,
+        path,
+        size,
+      })
+    } catch (err) {
+      if (err.code !== 'empty_file_unsupported') throw err
+      process.stdout.write(`skipped ${name}: ${err.message}\n`)
+      skipped++
+      continue
+    }
     process.stdout.write(`pushed ${name}\n`)
   }
+  if (skipped) process.exitCode = 2
+}
+
+function pushListOptions(flags) {
+  return { includeDotfiles: flags['include-dotfiles'] === true }
 }
 
 async function syncViaAgent(folder, { watch, intervalMs }, agent) {

src/sync.mjs

@@ -17,7 +17,7 @@
 // app. Replacing the previous fileId needs the Bin's folder key to re-wrap it,
 // which this client does not hold yet.
 
-import { open, readFile, readdir, rename, stat } from 'node:fs/promises'
+import { lstat, open, readFile, readdir, rename } from 'node:fs/promises'
 import { join } from 'node:path'
 
 export const MANIFEST_NAME = '.shieldfive-sync.json'
@@ -28,15 +28,16 @@ export function signatureOf({ size, mtimeMs }) {
   return `${size}:${mtimeMs}`
 }
 
-// List regular files to consider for sync: skips directories, dotfiles (which
+// List regular files to consider for sync: skips directories, symlinks (lstat,
+// so a link never pulls in a file from outside the folder), dotfiles (which
 // includes the manifest itself), and any explicitly excluded names.
 export async function scanFiles(folder, { exclude = [] } = {}) {
   const skip = new Set(exclude)
   const entries = []
   for (const name of await readdir(folder)) {
     if (name.startsWith('.') || skip.has(name)) continue
     const path = join(folder, name)
-    const info = await stat(path)
+    const info = await lstat(path)
     if (info.isFile()) {
       entries.push({ name, path, size: info.size, mtimeMs: info.mtimeMs })
     }
@@ -60,16 +61,20 @@ export async function loadManifest(manifestPath) {
 }
 
 // Atomic-ish write: write a temp sibling then rename over the target, so a crash
-// during the write can't leave a half-written manifest.
+// during the write can't leave a half-written manifest. Mode 0600: the manifest
+// lists every synced file name with its vault file id, which is nobody else's
+// business on a shared machine. chmod as well as the open mode, because a temp
+// file left by a crashed run keeps whatever mode it had.
 export async function saveManifest(manifestPath, manifest) {
   const body = JSON.stringify(
     { version: MANIFEST_VERSION, files: manifest.files },
     null,
     2,
   )
   const tmp = `${manifestPath}.tmp`
-  const handle = await open(tmp, 'w')
+  const handle = await open(tmp, 'w', 0o600)
   try {
+    await handle.chmod(0o600)
     await handle.writeFile(body)
     await handle.sync()
   } finally {