diff --git a/README.md b/README.md index 8ea1614..bc59628 100644 --- a/README.md +++ b/README.md @@ -95,7 +95,7 @@ remaining root-relative labels and metadata privately before sharing. See [the report guide](docs/SCANNER_REPORTS.md) for status meanings, privacy, CSP, output errors and residual filesystem/interrupted-write limits. Terminal/JSON output -and detection stay unchanged. No history, comparison, monitoring or repair is added. +and detection stay unchanged. HTML alone does not enable history; no comparison, monitoring or repair is added. ## Reproduce a synthetic finding @@ -253,3 +253,22 @@ The available Scanner remains an offline, read-only experimental beta. It does n continuously monitor services, generate repairs, execute scanned code or open pull requests. Its supported patterns, limited catalog, uncertainty and security boundaries above remain unchanged. + +## Local history — unpublished Phase B candidate + +This development branch adds explicit local history for review; Phase B is not yet +merged or advertised as publicly available. Plain scans and HTML-only scans still +create no history. Initialize a private directory, then supply its project UUID: + +```sh +node dist/index.js history init --directory ./my-history --json +node dist/index.js scan ./my-project --history-dir ./my-history --project-id PROJECT_UUID +node dist/index.js history list --directory ./my-history --html ./new-history-list.html +node dist/index.js history show SCAN_UUID --directory ./my-history --html ./new-historical-report.html +node dist/index.js history delete SCAN_UUID --directory ./my-history +``` + +Deletion/pruning preview by default and require `--apply` to unlink records. No +automatic history, browser opening, rescanning, deletion, telemetry or Cloud sync. +[The history guide](docs/SCANNER_HISTORY.md) documents storage, bounds, strict parsing, +privacy, original evidence, age/keep-last selection and filesystem/interruption risks. diff --git a/docs/SCANNER_HISTORY.md b/docs/SCANNER_HISTORY.md new file mode 100644 index 0000000..8b15452 --- /dev/null +++ b/docs/SCANNER_HISTORY.md @@ -0,0 +1,149 @@ +# Local scan history — unpublished Phase B candidate + +This local development branch implements opt-in history for founder review. Phase B +is not advertised as merged or publicly available. Use Node 24 and the reviewed build. +No account, database, new dependency, telemetry or network connection is required. + +## Explicit directory and project identity + +History is disabled by default. There is no implicit home-directory location, +`--save-history` shortcut, persistent enablement flag, scheduler or Cloud sync. +Initialize a dedicated private directory (its parent must already exist): + +```sh +node dist/index.js history init --directory ./my-history --json +``` + +The returned project UUID is an explicit grouping chosen by you, not proof of Git +identity or repository authenticity. One project belongs to each directory; use a +different initialized directory for another project. No target path or Git remote +is registered. Newly created directories use mode 0700, files mode 0600. Existing +directories must be empty, private, owned by your account and non-symlinked; Scanner +does not silently chmod existing locations. Use a non-synced private location. + +Saving requires both flags on each manually initiated scan: + +```sh +node dist/index.js scan ./my-project --history-dir ./my-history --project-id PROJECT_UUID +node dist/index.js scan ./my-project --history-dir ./my-history --project-id PROJECT_UUID --json --html ./new-live-report.html +``` + +No scanned source is executed or modified. The explicitly selected history location +may be inside a target, but a separate location is recommended. Init is separate; +scans never automatically initialize a directory. Existing terminal/JSON stdout is +unchanged. An explicit save prints `History saved: SCAN_UUID` to stderr; this receipt +is not an error or a compatibility verdict. Plain scans and HTML-only scans create +no history and retain existing stderr behavior. + +## What is stored + +An immutable `project.json` marker contains schema 1 and the random project UUID. +Each `.json` has independent saved-envelope schema 1: scan/project UUID, +local save timestamp, the privacy-filtered Phase A view with original scan timestamps, +Scanner version/build identity (or explicit unknown), catalog revision/fingerprint, +coverage/unknowns and structured integration/finding identities. Change-record and +client references are opaque SHA-256 hashes; references may change when source +positions/aliases change. No comparison is implemented or fix inferred. + +There is no raw source, absolute target path, raw argv, environment value, client +variable name or credential field in generated records. Only the existing privacy +projection is retained; source/configuration values withheld by Phase A stay withheld. +Root-relative filenames and permitted version metadata can still be sensitive; +private inspection is required. Imported records are untrusted and strictly parsed; +schema validation is not a signature or evidence that Scanner genuinely produced a +file. Do not edit records or rely on them as current compatibility evidence. + +## List and reopen without rescanning + +```sh +node dist/index.js history list --directory ./my-history --json +node dist/index.js history list --directory ./my-history --html ./new-history-list.html +node dist/index.js history show SCAN_UUID --directory ./my-history --json +node dist/index.js history show SCAN_UUID --directory ./my-history --html ./new-historical-report.html +``` + +Listing shows scan dates, providers, attention/unknown counts, recorded completeness +and readability. JSON listing contains summaries; show JSON contains the full saved +privacy-filtered envelope. Missing/corrupt/unsafe records fail clearly; listing +reports corrupt record UUIDs and counts of unknown entries without exposing their +filenames or contents. Unknown/corrupt files remain untouched. + +The static listing HTML is generated only to an explicit new destination. There +is no persistent index or automatically followed file link. Its reopen command is +a redacted equivalent; supply your own history location and a new HTML destination. +Reopened reports state **historical snapshot, not current compatibility**, preserve +original timestamps/catalog identity and do not require the original project or +current catalog. No automatic browser launch occurs. Open explicitly with Chrome +File → Open File. All HTML uses escaping, restrictive hashed-CSS CSP and no scripts, +external assets, telemetry or uploads. Manually following official documentation +links can leave a report, as in Phase A. + +## Delete and prune: preview first + +```sh +node dist/index.js history delete SCAN_UUID --directory ./my-history --json +node dist/index.js history delete SCAN_UUID --directory ./my-history --apply --json +node dist/index.js history prune --directory ./my-history --project-id PROJECT_UUID --keep-last 20 --older-than-days 90 --json +node dist/index.js history prune --directory ./my-history --project-id PROJECT_UUID --keep-last 20 --older-than-days 90 --apply --json +``` + +Without `--apply`, selected IDs are previewed and no records deleted. At least one +retention criterion is required; integers are 0–1,000,000. With both criteria, only +records older than the age threshold **and** outside the most recent N valid records +are selected. A single supplied criterion applies alone. Equal save timestamps use +UUID sorting; timestamps are local wall-clock values, not attested chronology. +Future timestamps fail pruning closed. Policies are not persisted or scheduled. + +Apply revalidates current records, directory/file identity and content. Preview is +not a reservation across invocations. Only validated UUID files for the selected +project can be unlinked; arbitrary paths, wildcards, recursive deletion, symlinked/ +hardlinked records and unknown/corrupt entries are never deletion targets. Output +reports selected, actually deleted and failed IDs; partial failures return 1. Local +unlink is not secure erasure or deletion from backups. The project marker is retained. + +## Bounds, coordination and failure limits + +Read limits: 32 MiB per record, 256 MiB total file sizes per directory listing/save, +10,000 directory entries, nesting 64 and 100,000 structured nodes. Strings are bounded +to 10,000 characters and arrays to 10,000 entries. Rendering/output remains capped at +32 MiB. There is no automatic eviction or paid scan quota. Ordinary scanning remains +available without history when a history budget is reached. Large directories may +require targeted deletion of a known valid UUID rather than a full listing. + +Safe reads verify owner/permissions, regular files, single links, ancestor/path/opened +descriptor identity, size and timestamps before/after bounded UTF-8 reads. Strict +validation rejects duplicate/escaped-equivalent JSON keys, unsafe prototype keys, +unknown fields/schemas, malformed UUIDs/dates/statuses/URLs and inconsistent counts. +Corrupt or partial JSON is not promoted into findings. No target code or JSON is +executed; parsing is bounded but not a hard process CPU/heap sandbox. + +Mutations use a fail-fast private `.history-lock` directory to coordinate cooperating +writers and enforce storage bounds. There are no retries or automatic stale-lock +deletions. If an interrupted process leaves a lock, stop and privately confirm no +writer is active and inspect the directory before manually removing that exact empty +lock directory. Do not automate recovery or delete arbitrary files. Listings report +busy/changed directories rather than claiming a consistent snapshot during writes. + +Writes are exclusive, fsynced and checked; no overwrite is possible through the CLI. +An interrupted write can leave an incomplete UUID record, which later reads reject +and pruning leaves alone. No automatic cleanup is attempted. Init may leave a newly +created directory or marker if interrupted. A requested change may already have +occurred when later cleanup fails: inspect outcomes and the private directory before +retrying. Deletion receipts preserve actual successful IDs if a later unlink fails. + +As with scanning/Phase A, portable Node path checks cannot eliminate hostile +concurrent ancestor replacement or guarantee atomic pathname deletion. Use stable, +private, owned directories, not attacker-controlled concurrent trees. Detection of +observed changes fails closed, but this is not an OS filesystem sandbox. Encryption +or secure erasure is not supplied; OS/disk protections and independent backups matter. + +Combined HTML/history output is not an atomic multi-file transaction. All content is +prepared before writes; HTML is written first. If history subsequently fails, exit 1 +states that HTML was saved and the record may be incomplete; no success JSON is emitted. +No earlier valid output is deleted to compensate. A full or invalid history directory +never causes silent deletion, guessing or a compatibility pass. + +Local history is a free manual evidence feature. It adds no continuous monitoring, +refreshed provider intelligence, advanced impact analysis, repair, customer-patch +verification, GitHub repair PR, hosted/team sync, account or billing capability. +Cloud remains planned and unavailable. Phase C comparison requires separate work. diff --git a/docs/SCANNER_REPORTS.md b/docs/SCANNER_REPORTS.md index f2f12fd..0b4ccd1 100644 --- a/docs/SCANNER_REPORTS.md +++ b/docs/SCANNER_REPORTS.md @@ -72,8 +72,9 @@ Normal terminal/JSON output retains its existing sensitive-data caveats. All displayed text is escaped. CSS is embedded with a SHA-256 CSP; scripts, images, fonts, forms and automatic network requests are prohibited. Source locations are inert text. Validated official documentation links leave the local report only if -you follow them manually. No telemetry, history, comparison, upload or browser -storage is added. +you follow them manually. HTML alone enables no history. No telemetry, comparison, upload or browser +storage is added. Explicit opt-in history on this local candidate branch is described +in [the history guide](SCANNER_HISTORY.md). Existing destinations, including symlinks/hardlinks/directories, are refused. Files are created exclusively with mode 0600, descriptor/ancestor identity checks and diff --git a/scripts/create-pilot-package.mjs b/scripts/create-pilot-package.mjs index 20f8bf7..8d9a500 100644 --- a/scripts/create-pilot-package.mjs +++ b/scripts/create-pilot-package.mjs @@ -28,7 +28,7 @@ const expected = [ 'dist/index.js', 'licenses/semver.LICENSE', 'licenses/typescript.LICENSE.txt', 'licenses/typescript.ThirdPartyNoticeText.txt', 'package.json', ] ; -if(publicCandidate)expected.push('SECURITY.md','CONTRIBUTING.md','FEEDBACK.md','PUBLIC_PROVENANCE.md','docs/SCANNER_REPORTS.md'); +if(publicCandidate)expected.push('SECURITY.md','CONTRIBUTING.md','FEEDBACK.md','PUBLIC_PROVENANCE.md','docs/SCANNER_REPORTS.md','docs/SCANNER_HISTORY.md'); expected.sort(); try { await mkdir(resolve(staging, 'licenses')); @@ -68,6 +68,7 @@ try { if(publicCandidate) { await mkdir(resolve(staging, 'docs')); await copyFile(publicTemplates === '.' ? 'docs/SCANNER_REPORTS.md' : resolve(publicTemplates, 'SCANNER_REPORTS.md'), resolve(staging, 'docs/SCANNER_REPORTS.md')); + await copyFile(publicTemplates === '.' ? 'docs/SCANNER_HISTORY.md' : resolve(publicTemplates, 'SCANNER_HISTORY.md'), resolve(staging, 'docs/SCANNER_HISTORY.md')); } const notices = ['Bundled runtime dependencies (not registry dependencies):']; for (const [name, files] of [['typescript', ['LICENSE.txt', 'ThirdPartyNoticeText.txt']], ['semver', ['LICENSE']]]) { diff --git a/scripts/verify-pilot-package.mjs b/scripts/verify-pilot-package.mjs index fff1881..aa7fb98 100644 --- a/scripts/verify-pilot-package.mjs +++ b/scripts/verify-pilot-package.mjs @@ -23,7 +23,7 @@ try { // Empty prefix and empty npm cache: no project dependencies or registry fallback. execFileSync('npm', ['install', '--offline', '--ignore-scripts', '--no-audit', '--no-fund', '--package-lock=false', '--no-save', '--prefix', prefix, archive, '--cache', resolve(work, 'empty-npm-cache')], { cwd: work, encoding: 'utf8' }); const expected = ['LICENSE', 'README.md', 'THIRD_PARTY_NOTICES.txt', 'dist/data/catalog.v1.json', 'dist/index.js', 'licenses/semver.LICENSE', 'licenses/typescript.LICENSE.txt', 'licenses/typescript.ThirdPartyNoticeText.txt', 'package.json']; - if(publicCandidate)expected.push('SECURITY.md','CONTRIBUTING.md','FEEDBACK.md','PUBLIC_PROVENANCE.md','docs/SCANNER_REPORTS.md'); + if(publicCandidate)expected.push('SECURITY.md','CONTRIBUTING.md','FEEDBACK.md','PUBLIC_PROVENANCE.md','docs/SCANNER_REPORTS.md','docs/SCANNER_HISTORY.md'); expected.sort(); const packed = execFileSync('tar', ['-tzf', archive], { encoding: 'utf8' }).trim().split('\n').map((file) => file.replace(/^package\//, '')).sort(); assert.deepEqual(packed, expected); @@ -88,6 +88,21 @@ registerHooks({ resolve(specifier, context, nextResolve) { assert.equal(noOverwrite.status, 1); assert.equal(noOverwrite.stdout, ''); assert.equal(await readFile(htmlPath, 'utf8'), html); + const history = resolve(work, 'synthetic history'); + const project = JSON.parse(run(['history', 'init', '--directory', history, '--json'])); + assert.equal(run(['scan', fixture, '--json', '--history-dir', history, '--project-id', project.projectId]), json); + assert.equal(run(['scan', fixture, '--json', '--history-dir', history, '--project-id', project.projectId]), json); + const saved = JSON.parse(run(['history', 'list', '--directory', history, '--json', '--html', resolve(work, 'history-list.html')])); + assert.equal(saved.records.length, 2); + const first = saved.records[0].scanId; + const reopened = JSON.parse(run(['history', 'show', first, '--directory', history, '--json', '--html', resolve(work, 'historical.html')])); + assert.ok(!JSON.stringify(reopened).includes(fixture)); + assert.ok(!JSON.stringify(reopened).includes('SYNTHETIC_PRIVATE_SENTINEL')); + assert.match(await readFile(resolve(work, 'historical.html'), 'utf8'), /Historical snapshot/); + assert.equal(JSON.parse(run(['history', 'delete', first, '--directory', history, '--json'])).deleted.length, 0); + assert.equal(JSON.parse(run(['history', 'delete', first, '--directory', history, '--apply', '--json'])).deleted[0], first); + assert.equal(JSON.parse(run(['history', 'list', '--directory', history, '--json'])).records.length, 1); + assert.equal(JSON.parse(run(['history', 'prune', '--directory', history, '--project-id', project.projectId, '--keep-last', '0', '--json'])).deleted.length, 0); assert.equal(await readFile(resolve(fixture, 'index.ts'), 'utf8'), source); assert.equal(await readFile(resolve(fixture, 'package.json'), 'utf8'), manifestText); const changes = JSON.parse(run(['changes', '--coverage', '--json'])); @@ -95,6 +110,6 @@ registerHooks({ resolve(specifier, context, nextResolve) { assert.equal(changes.catalogCoverage.liveMonitoring, false); const invalid = spawnSync(process.execPath, [...args, cli, 'scan', '../outside', '--json'], settings); assert.equal(invalid.status, 1); - console.log(JSON.stringify({ archive, sha256: digest, installation: 'passed: empty prefix/cache, offline, scripts disabled', runtime: 'self-contained; parent dependencies blocked', network: 'common Node transports denied; help/terminal/JSON/HTML/catalog passed', html: 'privacy/CSP/JSON compatibility/no-overwrite passed', executable: 'passed', targetUnchanged: true, ignoredEnv: true, traversalRejected: true, files: expected, node: process.version }, null, 2)); + console.log(JSON.stringify({ archive, sha256: digest, installation: 'passed: empty prefix/cache, offline, scripts disabled', runtime: 'self-contained; parent dependencies blocked', network: 'common Node transports denied; help/terminal/JSON/HTML/catalog/history passed', html: 'privacy/CSP/JSON compatibility/no-overwrite passed', history: 'init/two saves/list/reopen/delete preview/apply/prune preview passed', executable: 'passed', targetUnchanged: true, ignoredEnv: true, traversalRejected: true, files: expected, node: process.version }, null, 2)); } finally { await rm(work, { recursive: true, force: true }); } assert.equal(root, process.cwd()); diff --git a/src/cli/history.ts b/src/cli/history.ts new file mode 100644 index 0000000..46f722d --- /dev/null +++ b/src/cli/history.ts @@ -0,0 +1,64 @@ +import type { CliOutput } from './run.ts'; +import { LocalScanHistoryRepository } from '../history/repository.ts'; +import { formatHistoryTerminal, formatHistoryHtml, formatSavedTerminal, historySummary } from '../reports/history.ts'; +import { formatHtml } from '../reports/html.ts'; +import { writeNewArtifact } from '../artifacts/filesystem.ts'; +import { uuid } from '../history/validation.ts'; + +export const historyHelp = `PatchWRK local scan history (offline, opt-in) + +history init --directory DIR [--json] +scan TARGET --history-dir DIR --project-id UUID [--json] [--html NEW.html] +history list --directory DIR [--project-id UUID] [--json] [--html NEW.html] +history show SCAN_UUID --directory DIR [--json] [--html NEW.html] +history delete SCAN_UUID --directory DIR [--apply] [--json] +history prune --directory DIR --project-id UUID [--keep-last N] [--older-than-days N] [--apply] [--json] + +No implicit history location. Init creates one project UUID per private directory. +Delete/prune preview by default; --apply explicitly authorizes local unlink. +Age AND keep-last criteria are combined. No automatic pruning, sync or rescan. +HTML is generated only to an explicitly requested NEW destination; no browser opens. +Records and reports can retain sensitive root-relative labels; inspect privately.`; + +export async function runHistory(args: string[], output: CliOutput): Promise { + const command = args[1]; + if (!command || !['init', 'list', 'show', 'delete', 'prune'].includes(command)) throw new Error('Unknown history command. Use patchwrk history --help.'); + const values = new Map(); let scanId: string | undefined; let json = false; let apply = false; + const paths = new Set(['--directory', '--project-id', '--html', '--keep-last', '--older-than-days']); + for (let index = 2; index < args.length; index++) { + const arg = args[index]!; + if (arg === '--json' && !json) json = true; + else if (arg === '--apply' && !apply) apply = true; + else if (paths.has(arg) && !values.has(arg)) { + const value = args[++index]; if (!value || value.startsWith('-')) throw new Error('Missing history option value.'); values.set(arg, value); + } else if (!arg.startsWith('-') && scanId === undefined && ['show', 'delete'].includes(command)) scanId = uuid(arg); + else throw new Error('Unknown or duplicate history argument.'); + } + const allowed: Record = { init: ['--directory'], list: ['--directory', '--project-id', '--html'], show: ['--directory', '--html'], delete: ['--directory'], prune: ['--directory', '--project-id', '--keep-last', '--older-than-days'] }; + if ([...values.keys()].some((key) => !allowed[command]!.includes(key)) || (apply && !['delete', 'prune'].includes(command))) throw new Error('Option not supported for this history command.'); + const dir = values.get('--directory'); if (!dir) throw new Error('History requires an explicit --directory.'); + if (command === 'init') { + const project = await LocalScanHistoryRepository.init(dir); output.stdout(json ? JSON.stringify(project, null, 2) : `History initialized. Project UUID: ${project.projectId}`); return 0; + } + const repo = await LocalScanHistoryRepository.open(dir, values.get('--project-id')); + if (command === 'list') { + const listing = await repo.list(); const html = values.get('--html'); + if (html) await writeNewArtifact(html, formatHistoryHtml(listing)); + output.stdout(json ? JSON.stringify({ ...listing, records: listing.records.map(historySummary) }, null, 2) : formatHistoryTerminal(listing)); return 0; + } + if (command === 'show') { + if (!scanId) throw new Error('History show requires a scan UUID.'); + const record = await repo.get(scanId); const html = values.get('--html'); + if (html) await writeNewArtifact(html, formatHtml(record.snapshot, { scanId: record.scanId, savedAt: record.savedAt })); + output.stdout(json ? JSON.stringify(record, null, 2) : formatSavedTerminal(record)); return 0; + } + if (command === 'delete' && !scanId) throw new Error('History delete requires a scan UUID.'); + const policy: { keepLast?: number; olderThanDays?: number } = {}; + for (const [flag, key] of [['--keep-last', 'keepLast'], ['--older-than-days', 'olderThanDays']] as const) { + const value = values.get(flag); if (value !== undefined) { if (!/^(?:0|[1-9]\d*)$/u.test(value)) throw new Error('Retention values must be nonnegative integers.'); policy[key] = Number(value); } + } + if (command === 'prune' && !values.has('--project-id')) throw new Error('History prune requires an explicit --project-id.'); + const outcome = command === 'delete' ? await repo.remove(scanId!, apply) : await repo.prune(policy, apply); + output.stdout(json ? JSON.stringify(outcome, null, 2) : `${outcome.applied ? 'Apply' : 'Preview only'}: ${outcome.selected.length} selected; ${outcome.deleted.length} deleted; ${outcome.failed.length} failed.\n${outcome.selected.map((id) => `Selected: ${id}`).join('\n')}\n${outcome.deleted.map((id) => `Deleted: ${id}`).join('\n')}\n${outcome.warnings.join('\n')}`); + return outcome.failed.length || outcome.lockCleanupFailed ? 1 : 0; +} diff --git a/src/cli/run.ts b/src/cli/run.ts index dc4c49c..6cb4045 100644 --- a/src/cli/run.ts +++ b/src/cli/run.ts @@ -9,6 +9,9 @@ import { captureReportContext } from '../reports/context.ts'; import { buildReportView } from '../reports/view.ts'; import { formatHtml } from '../reports/html.ts'; import { writeNewArtifact } from '../artifacts/filesystem.ts'; +import { runHistory, historyHelp } from './history.ts'; +import { LocalScanHistoryRepository } from '../history/repository.ts'; +import { buildSavedScan, uuid } from '../history/validation.ts'; export const help = `PatchWRK Scanner @@ -19,11 +22,14 @@ Usage: patchwrk scan [directory] [--json] Commands: scan Discover supported API SDK dependencies and source evidence changes List the local offline change catalog (no live updates) + history Explicit local snapshots; use history --help (no automatic history) Options: --help, -h Show this help --json Print a JSON report --html PATH Save a new self-contained local HTML report (never overwrite) + --history-dir DIR Explicit initialized private history directory + --project-id UUID Required with --history-dir; returned by history init --provider NAME Filter changes by provider --max-files N Maximum files read (default: 2000) --max-file-bytes N Maximum bytes per file (default: 1048576) @@ -59,7 +65,7 @@ Local offline catalog only; no live retrieval or continuous updates.`; export async function runCli(args: string[], output: CliOutput, services: { catalog?: ChangeCatalogRepository } = {}): Promise { try { if (args.length === 0 || args.includes('--help') || args.includes('-h')) { - output.stdout(args[0] === 'changes' ? changesHelp : help); + output.stdout(args[0] === 'changes' ? changesHelp : args[0] === 'history' ? historyHelp : help); return 0; } if (args[0] === 'changes') { @@ -79,11 +85,14 @@ export async function runCli(args: string[], output: CliOutput, services: { cata output.stdout(json ? formatChangesJson(report) : formatChangesTerminal(report, coverageOnly)); return 0; } + if (args[0] === 'history') return await runHistory(args, output); if (args[0] !== 'scan') throw new Error('Unknown command. Use patchwrk --help.'); let directory: string | undefined; let json = false; let positionalOnly = false; let htmlPath: string | undefined; + let historyDir: string | undefined; + let projectId: string | undefined; const limits: Partial = {}; const analysisLimits: Partial = {}; const moduleOptions: Record = { @@ -99,6 +108,11 @@ export async function runCli(args: string[], output: CliOutput, services: { cata const arg = args[index]!; if (!positionalOnly && arg === '--') positionalOnly = true; else if (!positionalOnly && arg === '--json') json = true; + else if (!positionalOnly && (arg === '--history-dir' || arg === '--project-id')) { + if ((arg === '--history-dir' && historyDir !== undefined) || (arg === '--project-id' && projectId !== undefined)) throw new Error('Duplicate history option.'); + const value = args[++index]; if (!value || value.startsWith('-')) throw new Error('Missing history option value.'); + if (arg === '--history-dir') historyDir = value; else projectId = uuid(value); + } else if (!positionalOnly && arg === '--html') { if (htmlPath !== undefined) throw new Error('Duplicate --html option.'); const value = args[++index]; @@ -129,10 +143,21 @@ export async function runCli(args: string[], output: CliOutput, services: { cata else directory = arg; } const options = { scan: { limits, analysisLimits }, apiVersionInputs, ...services }; + if ((historyDir === undefined) !== (projectId === undefined)) throw new Error('History saving requires both --history-dir and --project-id.'); + const history = historyDir === undefined ? undefined : await LocalScanHistoryRepository.open(historyDir, projectId); const target = directory ?? process.cwd(); - const context = htmlPath === undefined ? undefined : await captureReportContext(target, options); + const context = htmlPath === undefined && !history ? undefined : await captureReportContext(target, options); const report = context?.scan ?? await scanWithCompatibility(target, options); - if (context && htmlPath !== undefined) await writeNewArtifact(htmlPath, formatHtml(buildReportView(report, context.metadata))); + if (context) { + const view = buildReportView(report, context.metadata); + const saved = history ? buildSavedScan(report, view, projectId!) : undefined; + const html = htmlPath === undefined ? undefined : formatHtml(view); + if (html !== undefined) await writeNewArtifact(htmlPath!, html); + if (history && saved) { + try { const id = await history.save(saved); output.stderr(`History saved: ${id}`); } + catch { throw new Error(`History save failed; ${html !== undefined ? 'the requested HTML report was saved' : 'no HTML was requested'}. A record may be incomplete; inspect the private directory. No overwrite or automatic cleanup occurred.`); } + } + } output.stdout(json ? formatJson(report) : formatTerminal(report)); return 0; } catch (error) { diff --git a/src/history/repository.ts b/src/history/repository.ts new file mode 100644 index 0000000..521bfd9 --- /dev/null +++ b/src/history/repository.ts @@ -0,0 +1,201 @@ +import { constants } from 'node:fs'; +import type { Stats } from 'node:fs'; +import { lstat, mkdir, open, opendir, rmdir, unlink } from 'node:fs/promises'; +import { resolve, dirname, parse, sep } from 'node:path'; +import { randomUUID, createHash } from 'node:crypto'; +import { historyLimits, parseHistory, validateProject, validateSavedScan, uuid } from './validation.ts'; +import type { HistoryProject, HistoryListing, SavedScan, HistoryDeletion } from '../types/history.ts'; + +const marker = 'project.json'; +const mutex = '.history-lock'; +const failure = () => new Error('History operation failed: use an initialized private, stable, non-symlink directory and valid record/project IDs. Records may be missing, corrupt, busy, unsafe or over budget. Inspect privately; no automatic cleanup or overwrite is performed.'); +const identity = (stat: Stats) => `${stat.dev}:${stat.ino}`; +const stamp = (stat: Stats) => `${identity(stat)}:${stat.size}:${stat.mtimeMs}:${stat.ctimeMs}`; +function privateEntry(stat: Stats, mode: number): void { + if ((stat.mode & 0o777) !== mode || (typeof process.getuid === 'function' && stat.uid !== process.getuid())) throw failure(); +} +async function ancestors(path: string): Promise { + let current = parse(path).root; const entries: string[] = []; + for (const component of path.slice(current.length).split(sep).filter(Boolean)) { + current = resolve(current, component); const stat = await lstat(current); + if (!stat.isDirectory() || stat.isSymbolicLink()) throw failure(); + entries.push(identity(stat)); + } + return entries.join('/'); +} +function directory(input: string): string { + if (!input || input.includes('\0') || input.split(/[\\/]/u).includes('..')) throw failure(); + const path = resolve(input); if (path === parse(path).root || dirname(path) === parse(path).root) throw failure(); + return path; +} + +/** Dedicated private directory, immutable records, bounded reads; not a hostile-filesystem sandbox. */ +export class LocalScanHistoryRepository { + private readonly root: string; + private readonly rootIdentity: string; + readonly project: HistoryProject; + private readonly markerStamp: string | null; + private constructor(root: string, rootIdentity: string, project: HistoryProject, markerStamp: string | null = null) { + this.root = root; this.rootIdentity = rootIdentity; this.project = Object.freeze({ ...project }); this.markerStamp = markerStamp; + } + + static async init(input: string): Promise { + try { + const root = directory(input); await ancestors(dirname(root)); + try { await mkdir(root, { mode: 0o700 }); } catch (error) { if ((error as NodeJS.ErrnoException).code !== 'EEXIST') throw error; } + const stat = await lstat(root); if (!stat.isDirectory() || stat.isSymbolicLink()) throw failure(); privateEntry(stat, 0o700); + const repo = new LocalScanHistoryRepository(root, await ancestors(root), { schemaVersion: 1, projectId: randomUUID() }); + await repo.lock(async () => { + const handle = await opendir(root); let count = 0; + try { for await (const entry of handle) { if (++count > 1 || entry.name !== mutex) throw failure(); } } finally { await handle.close().catch(() => undefined); } + await repo.write(marker, JSON.stringify(repo.project)); + }); + return repo.project; + } catch { throw failure(); } + } + static async open(input: string, projectId?: string): Promise { + try { + const root = directory(input); const stat = await lstat(root); + if (!stat.isDirectory() || stat.isSymbolicLink()) throw failure(); privateEntry(stat, 0o700); + const repo = new LocalScanHistoryRepository(root, await ancestors(root), { schemaVersion: 1, projectId: '' }); + const read = await repo.read(marker, 4096); + const project = validateProject(parseHistory(read.text)); + if (projectId !== undefined && uuid(projectId) !== project.projectId) throw failure(); + return new LocalScanHistoryRepository(root, repo.rootIdentity, project, stamp(read.stat)); + } catch { throw failure(); } + } + private async guard(): Promise { + if (await ancestors(this.root) !== this.rootIdentity) throw failure(); + const stat = await lstat(this.root); privateEntry(stat, 0o700); + if (this.markerStamp !== null) { + const project = await lstat(resolve(this.root, marker)); + if (!project.isFile() || project.isSymbolicLink() || project.nlink !== 1 || stamp(project) !== this.markerStamp) throw failure(); + privateEntry(project, 0o600); + } + } + private async read(name: string, maxBytes = historyLimits.maxRecordBytes): Promise<{ text: string; stat: Stats; hash: string }> { + await this.guard(); const path = resolve(this.root, name); const before = await lstat(path); + if (!before.isFile() || before.isSymbolicLink() || before.nlink !== 1 || before.size > maxBytes) throw failure(); privateEntry(before, 0o600); + const handle = await open(path, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK); + try { + const opened = await handle.stat(); await this.guard(); + if (!opened.isFile() || opened.nlink !== 1 || identity(opened) !== identity(before) || opened.size !== before.size || opened.mtimeMs !== before.mtimeMs || opened.ctimeMs !== before.ctimeMs) throw failure(); privateEntry(opened, 0o600); + const data = Buffer.alloc(opened.size); let offset = 0; + while (offset < data.length) { const result = await handle.read(data, offset, data.length - offset, offset); if (!result.bytesRead) break; offset += result.bytesRead; } + const after = await handle.stat(); const current = await lstat(path); await this.guard(); + if (offset !== opened.size || identity(current) !== identity(opened) || current.isSymbolicLink() || current.nlink !== 1 || after.nlink !== 1 || after.size !== opened.size || after.mtimeMs !== opened.mtimeMs || after.ctimeMs !== opened.ctimeMs || current.mode !== opened.mode || stamp(current) !== stamp(after)) throw failure(); + privateEntry(current, 0o600); + return { text: new TextDecoder('utf-8', { fatal: true }).decode(data), stat: after, hash: createHash('sha256').update(data).digest('hex') }; + } finally { await handle.close(); } + } + private async write(name: string, text: string): Promise { + if (Buffer.byteLength(text, 'utf8') > historyLimits.maxRecordBytes) throw failure(); + await this.guard(); const path = resolve(this.root, name); + const handle = await open(path, constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | constants.O_NOFOLLOW | constants.O_NONBLOCK, 0o600); + try { + const opened = await handle.stat(); + const verify = async () => { await this.guard(); const stat = await lstat(path); const fd = await handle.stat(); if (identity(stat) !== identity(opened) || stat.isSymbolicLink() || !stat.isFile() || stat.nlink !== 1 || fd.nlink !== 1) throw failure(); privateEntry(stat, 0o600); }; + await verify(); await handle.writeFile(text); await handle.sync(); await verify(); + if ((await handle.stat()).size !== Buffer.byteLength(text, 'utf8')) throw failure(); + } finally { await handle.close(); } + } + private async lock(operation: () => Promise): Promise { + await this.guard(); const lockPath = resolve(this.root, mutex); + await mkdir(lockPath, { mode: 0o700 }); const lockStat = await lstat(lockPath); + const release = async () => { + // Never remove an unrelated replacement or auto-clear an interrupted lock. + await this.guard(); const current = await lstat(lockPath); + if (identity(current) !== identity(lockStat) || !current.isDirectory() || current.isSymbolicLink()) throw failure(); + await rmdir(lockPath); + }; + let result: T; + try { await this.guard(); privateEntry(lockStat, 0o700); result = await operation(); } + catch (error) { await release().catch(() => undefined); throw error; } + try { await release(); } catch { + if (result && typeof result === 'object' && 'deleted' in result && 'warnings' in result) { + const deletion = result as unknown as HistoryDeletion; + deletion.lockCleanupFailed = true; + deletion.warnings.push('Write lock cleanup failed; actual deletion results above remain valid. Inspect the private directory before retrying.'); + } else throw failure(); + } + return result; + } + async get(scanId: string): Promise { + uuid(scanId); + try { + const saved = validateSavedScan(parseHistory((await this.read(`${uuid(scanId)}.json`)).text)); + if (saved.scanId !== scanId || saved.projectId !== this.project.projectId) throw failure(); return saved; + } catch (error) { + if ((error as NodeJS.ErrnoException).code === 'ENOENT') throw new Error('History record is missing or its directory is unavailable.', { cause: error }); + throw new Error('History record is corrupt, unsupported or unsafe; no record content was displayed.', { cause: error }); + } + } + private async listing(locked = false): Promise { + await this.guard(); const records: SavedScan[] = []; const warnings: string[] = []; let bytes = 0; let count = 0; let unknown = 0; + const before = stamp(await lstat(this.root)); + const handle = await opendir(this.root); + try { + for await (const entry of handle) { + if (++count > historyLimits.maxEntries) throw failure(); + if (entry.name === mutex) { if (!locked) throw failure(); continue; } + await this.guard(); const stat = await lstat(resolve(this.root, entry.name)); + if (stat.isFile()) bytes += stat.size; + if (bytes > historyLimits.maxTotalBytes) throw failure(); + if (entry.name === marker) continue; + const match = /^([a-f0-9-]{36})\.json$/u.exec(entry.name); + if (!match) { unknown++; continue; } + try { uuid(match[1]); records.push(await this.get(match[1]!)); } + catch { warnings.push(`Record ${match[1]} is unreadable, corrupt, unsafe or unsupported; left untouched.`); } + } + } finally { await handle.close().catch(() => undefined); } + await this.guard(); + if (stamp(await lstat(this.root)) !== before) throw failure(); + if (unknown) warnings.push(`${unknown} unrecognized entries left untouched; filenames withheld.`); + records.sort((a, b) => b.savedAt.localeCompare(a.savedAt) || a.scanId.localeCompare(b.scanId)); + return { schemaVersion: 1, projectId: this.project.projectId, records, warnings, bytes }; + } + async list(): Promise { try { return await this.listing(); } catch { throw failure(); } } + async save(input: SavedScan): Promise { + try { + const record = validateSavedScan(input); if (record.projectId !== this.project.projectId) throw failure(); + await this.lock(async () => { + const listing = await this.listing(true); const text = JSON.stringify(record); + // Entry bound reserves space for marker, lock and the new immutable file. + const handle = await opendir(this.root); let count = 0; + try { for await (const entry of handle) { count++; if (!entry.name) throw failure(); } } finally { await handle.close().catch(() => undefined); } + if (count >= historyLimits.maxEntries || listing.bytes + Buffer.byteLength(text) > historyLimits.maxTotalBytes) throw failure(); + await this.write(`${record.scanId}.json`, text); + }); return record.scanId; + } catch { throw failure(); } + } + async remove(scanId: string, apply: boolean): Promise { + try { uuid(scanId); return await this.lock(async () => { await this.get(scanId); return this.removeSelected([scanId], apply); }); } catch { throw failure(); } + } + async prune(policy: { keepLast?: number; olderThanDays?: number }, apply: boolean, now = Date.now()): Promise { + try { + for (const value of Object.values(policy)) if (!Number.isSafeInteger(value) || value! < 0 || value! > 1_000_000) throw failure(); + if (!Object.keys(policy).length || !Number.isFinite(now)) throw failure(); + return await this.lock(async () => { + const listing = await this.listing(true); + if (listing.records.some((record) => Date.parse(record.savedAt) > now)) throw failure(); + const ids = listing.records.filter((record, index) => (policy.keepLast === undefined || index >= policy.keepLast) + && (policy.olderThanDays === undefined || Date.parse(record.savedAt) < now - policy.olderThanDays * 86400_000)).map((record) => record.scanId); + const outcome = await this.removeSelected(ids, apply); outcome.warnings.push(...listing.warnings); return outcome; + }); + } catch { throw failure(); } + } + private async removeSelected(ids: string[], apply: boolean): Promise { + const outcome: HistoryDeletion = { schemaVersion: 1, applied: apply, selected: ids, deleted: [], failed: [], lockCleanupFailed: false, warnings: ['Local unlink is not secure erasure or deletion from independent backups. Preview is not a reservation; apply validates current records.'] }; + if (!apply) return outcome; + for (const id of ids) { + try { + const file = `${uuid(id)}.json`; const first = await this.read(file); const record = validateSavedScan(parseHistory(first.text)); + if (record.scanId !== id || record.projectId !== this.project.projectId) throw failure(); + const current = await this.read(file); + if (identity(first.stat) !== identity(current.stat) || first.hash !== current.hash) throw failure(); + await this.guard(); await unlink(resolve(this.root, file)); outcome.deleted.push(id); await this.guard(); + } catch { outcome.failed.push(id); break; } + } + return outcome; + } +} diff --git a/src/history/validation.ts b/src/history/validation.ts new file mode 100644 index 0000000..b764813 --- /dev/null +++ b/src/history/validation.ts @@ -0,0 +1,155 @@ +import ts from 'typescript'; +import { createHash, randomUUID } from 'node:crypto'; +import type { SavedScan, HistoryProject } from '../types/history.ts'; +import type { ReportView } from '../types/reporting.ts'; +import type { CompatibilityScanReport } from '../types/compatibility.ts'; +import { canonicalJson } from '../reports/context.ts'; +import { isOfficialUrl } from '../changes/sources.ts'; +import { providerIds, changeCategories } from '../types/changes.ts'; + +export const historyLimits = { maxRecordBytes: 32 * 1024 * 1024, maxTotalBytes: 256 * 1024 * 1024, maxEntries: 10_000, maxNodes: 100_000, maxDepth: 64 } as const; +export function invalid(): never { throw new Error('Invalid, corrupt, unsafe or unsupported history data.'); } +export function uuid(value: unknown): string { + if (typeof value !== 'string' || !/^[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/u.test(value)) invalid(); + return value; +} +function obj(value: unknown, required: string[], optional: string[] = []): Record { + if (!value || typeof value !== 'object' || Array.isArray(value) || Object.getPrototypeOf(value) !== Object.prototype) invalid(); + const descriptors = Object.getOwnPropertyDescriptors(value); + if (required.some((key) => !Object.hasOwn(descriptors, key)) || Object.keys(descriptors).some((key) => ![...required, ...optional].includes(key) || !('value' in descriptors[key]!))) invalid(); + return value as Record; +} +function str(value: unknown): string { + if (typeof value !== 'string' || value.length > 10_000 || /(?:(?:^|[\s"'(])\/(?![\s/])|[A-Za-z]:\\|\\\\|-----BEGIN .*PRIVATE KEY-----|\bAKIA[0-9A-Z]{16}|\bgh[pousr]_[A-Za-z0-9]{25,}|\bsk-(?:proj-)?[A-Za-z0-9_-]{12,}|\bsk_(?:live|test)_[A-Za-z0-9]+)/u.test(value) + || [...value].some((c) => (c.charCodeAt(0) < 32 && c !== '\n' && c !== '\t') || c.charCodeAt(0) === 127)) invalid(); + return value; +} +function number(value: unknown): number { if (!Number.isSafeInteger(value) || (value as number) < 0) invalid(); return value as number; } +function bool(value: unknown): void { if (typeof value !== 'boolean') invalid(); } +function hash(value: unknown): void { if (typeof value !== 'string' || !/^[a-f0-9]{64}$/u.test(value)) invalid(); } +function timestamp(value: unknown): void { const text = str(value); if (!/^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z$/u.test(text) || !Number.isFinite(Date.parse(text)) || new Date(text).toISOString() !== text) invalid(); } +function array(value: unknown): unknown[] { if (!Array.isArray(value) || value.length > 10_000 || Object.keys(value).length !== value.length || Object.keys(value).some((key) => !/^(?:0|[1-9]\d*)$/u.test(key))) invalid(); return value; } +function strings(value: unknown): void { array(value).forEach(str); } +function rows(value: unknown): void { array(value).forEach((item) => { const row = obj(item, ['label', 'value']); str(row.label); str(row.value); }); } +function choice(value: unknown, values: readonly string[]): string { const text = str(value); if (!values.includes(text)) invalid(); return text; } +function structure(value: unknown): void { + const pending: [unknown, number][] = [[value, 0]]; let nodes = 0; + while (pending.length) { + const [item, depth] = pending.pop()!; + if (++nodes > historyLimits.maxNodes || depth > historyLimits.maxDepth) invalid(); + if (typeof item === 'string') str(item); + else if (item !== null && typeof item === 'object') { + const proto = Object.getPrototypeOf(item); + if (proto !== Object.prototype && !(Array.isArray(item) && proto === Array.prototype)) invalid(); + if (Object.getOwnPropertySymbols(item).length) invalid(); + for (const [key, descriptor] of Object.entries(Object.getOwnPropertyDescriptors(item))) { + if (['__proto__', 'constructor', 'prototype'].includes(key) || !('value' in descriptor)) invalid(); + if (Array.isArray(item) && key === 'length') continue; + pending.push([descriptor.value, depth + 1]); + if (pending.length > historyLimits.maxNodes) invalid(); + } + } else if (item !== null && typeof item !== 'boolean' && (typeof item !== 'number' || !Number.isFinite(item))) invalid(); + } +} + +/** Independent bounded parser: never imports/evaluates records or silently accepts duplicate keys. */ +export function parseHistory(text: string): unknown { + if (Buffer.byteLength(text, 'utf8') > historyLimits.maxRecordBytes) invalid(); + let depth = 0; let quoted = false; let escaped = false; + for (const c of text) { + if (quoted) { if (escaped) escaped = false; else if (c === '\\') escaped = true; else if (c === '"') quoted = false; } + else if (c === '"') quoted = true; + else if (c === '{' || c === '[') { if (++depth > historyLimits.maxDepth) invalid(); } + else if (c === '}' || c === ']') depth--; + } + let value: unknown; try { value = JSON.parse(text); } catch { invalid(); } + structure(value); + const pending: ts.Node[] = [ts.parseJsonText('history.json', text)]; let visited = 0; + while (pending.length) { + const node = pending.pop()!; if (++visited > 400_000) invalid(); + if (ts.isObjectLiteralExpression(node)) { + const keys = new Set(); + for (const property of node.properties) { + if (!ts.isPropertyAssignment(property) || !ts.isStringLiteral(property.name) || keys.has(property.name.text)) invalid(); + keys.add(property.name.text); + } + } + ts.forEachChild(node, (child) => { pending.push(child); }); + } + return value; +} +export function validateProject(value: unknown): HistoryProject { + structure(value); const project = obj(value, ['schemaVersion', 'projectId']); + if (project.schemaVersion !== 1) invalid(); uuid(project.projectId); + return structuredClone(project) as unknown as HistoryProject; +} +const labels = ['Affected', 'Potential risk', 'Unknown', 'Not affected by this change', 'No known finding within supported coverage', 'Unknown / partial coverage']; +function block(value: unknown): void { + const b = obj(value, ['title', 'rows', 'notes'], ['status', 'sourceUrl', 'finding']); str(b.title); rows(b.rows); strings(b.notes); + if (b.status !== undefined) choice(b.status, labels); + if (b.sourceUrl !== undefined) { + const url = str(b.sourceUrl); + if (!providerIds.some((id) => isOfficialUrl(id, url)) || new URL(url).search) invalid(); + } + if (b.finding !== undefined) { + const f = obj(b.finding, ['provider', 'title', 'explanation', 'references', 'moreReferences', 'uncertainty', 'actions']); + str(f.provider); str(f.title); str(f.explanation); strings(f.references); number(f.moreReferences); strings(f.uncertainty); strings(f.actions); + } +} +function snapshot(value: unknown): ReportView { + const view = obj(value, ['schemaVersion', 'metadata', 'complete', 'stats', 'notices', 'coverage', 'sections']); + if (view.schemaVersion !== 1) invalid(); bool(view.complete); strings(view.notices); rows(view.coverage); + const m = obj(view.metadata, ['scannerVersion', 'buildFingerprint', 'catalogVersion', 'catalogFingerprint', 'catalogRecordCount', 'providers', 'startedAt', 'finishedAt', 'generatedAt']); + if (m.scannerVersion !== null && !/^\d+\.\d+\.\d+(?:-[a-zA-Z0-9.-]+)?$/u.test(str(m.scannerVersion))) invalid(); + if (m.buildFingerprint !== null) hash(m.buildFingerprint); hash(m.catalogFingerprint); + if (!/^[1-9]\d{0,8}$/u.test(str(m.catalogVersion))) invalid(); number(m.catalogRecordCount); + [m.startedAt, m.finishedAt, m.generatedAt].forEach(timestamp); + const ids = new Set(); + array(m.providers).forEach((item) => { const p = obj(item, ['id', 'name', 'records']); const id = choice(p.id, providerIds); if (ids.has(id)) invalid(); ids.add(id); str(p.name); number(p.records); }); + if (ids.size !== providerIds.length) invalid(); + array(view.stats).forEach((item) => { const stat = obj(item, ['label', 'value']); str(stat.label); number(stat.value); }); + const kinds = ['scope', 'integrations', 'findings', 'attention', 'actions', 'limitations']; const seen = new Set(); + array(view.sections).forEach((item) => { const section = obj(item, ['kind', 'title', 'blocks']); const kind = choice(section.kind, kinds); if (seen.has(kind)) invalid(); seen.add(kind); str(section.title); array(section.blocks).forEach(block); }); + if (seen.size !== kinds.length) invalid(); + return view as unknown as ReportView; +} +export function validateSavedScan(value: unknown): SavedScan { + structure(value); + const record = obj(value, ['schemaVersion', 'scanId', 'projectId', 'savedAt', 'identities', 'snapshot']); + if (record.schemaVersion !== 1) invalid(); uuid(record.scanId); uuid(record.projectId); timestamp(record.savedAt); + const identities = obj(record.identities, ['integrations', 'findings']); const seen = new Set(); + const packages = ['stripe', 'openai', '@anthropic-ai/sdk', '@shopify/shopify-api', 'twilio']; + array(identities.integrations).forEach((item) => { const i = obj(item, ['provider', 'packageName']); const provider = choice(i.provider, providerIds); const name = choice(i.packageName, packages); if (packages[providerIds.indexOf(provider as typeof providerIds[number])] !== name) invalid(); const key = `${provider}:${name}`; if (seen.has(key)) invalid(); seen.add(key); }); + const references = new Set(); + array(identities.findings).forEach((item) => { + const f = obj(item, ['provider', 'packageName', 'changeId', 'reference', 'clientReference', 'recordFingerprint', 'status', 'scope', 'category']); + choice(f.provider, providerIds); choice(f.packageName, packages); if (!seen.has(`${str(f.provider)}:${str(f.packageName)}`)) invalid(); + if (!/^[a-z0-9][a-z0-9.-]{0,159}$/u.test(str(f.changeId))) invalid(); hash(f.reference); hash(f.recordFingerprint); if (f.clientReference !== null) hash(f.clientReference); + if (references.has(f.reference as string)) invalid(); references.add(f.reference as string); + choice(f.status, ['affected', 'potentially-affected', 'unknown', 'not-affected-by-this-change']); + choice(f.scope, ['documented-version-applicability', 'documented-configuration-applicability']); choice(f.category, changeCategories); + }); + const view = snapshot(record.snapshot); + const findings = identities.findings as SavedScan['identities']['findings']; + const stats = [view.stats[0]?.label === 'Files scanned' ? view.stats[0].value : -1, seen.size, findings.filter((f) => ['affected', 'potentially-affected'].includes(f.status)).length, findings.filter((f) => f.status === 'unknown').length]; + if (view.stats.length !== 4 || view.stats.some((item, index) => item.value !== stats[index] || item.label !== ['Files scanned', 'Supported integrations', 'Assessments requiring attention', 'Unknown assessments'][index])) invalid(); + const assessmentBlocks = view.sections.find((s) => s.kind === 'findings')!.blocks.filter((b) => b.finding !== undefined); + if (assessmentBlocks.length !== findings.length) invalid(); + const statusLabels = { affected: 'Affected', 'potentially-affected': 'Potential risk', unknown: 'Unknown', 'not-affected-by-this-change': 'Not affected by this change' }; + for (const f of findings) { + const b = assessmentBlocks.find((b) => b.rows.some((r) => r.label === 'Finding reference' && r.value === f.reference)); + if (!b || b.status !== statusLabels[f.status]) invalid(); + if (b.sourceUrl && !isOfficialUrl(f.provider as typeof providerIds[number], b.sourceUrl)) invalid(); + } + if (Buffer.byteLength(JSON.stringify(value), 'utf8') > historyLimits.maxRecordBytes) invalid(); + return structuredClone(record) as unknown as SavedScan; +} +export function buildSavedScan(scan: CompatibilityScanReport, view: ReportView, projectId: string): SavedScan { + const digest = (value: unknown) => createHash('sha256').update(canonicalJson(value)).digest('hex'); + const findings = view.sections.find((s) => s.kind === 'findings')!.blocks; + return validateSavedScan({ schemaVersion: 1, scanId: randomUUID(), projectId: uuid(projectId), savedAt: new Date().toISOString(), + identities: { integrations: scan.integrations.map((i) => ({ provider: i.provider, packageName: i.packageName })), + findings: scan.compatibility.assessments.map((a, index) => ({ provider: a.provider, packageName: a.packageName, changeId: a.changeId, + reference: findings[index]!.rows.find((r) => r.label === 'Finding reference')!.value, + clientReference: a.client ? digest(a.client.definition.id) : null, recordFingerprint: digest(a.changeRecord), status: a.status, scope: a.scope, category: a.category })) }, snapshot: view }); +} diff --git a/src/reports/history.ts b/src/reports/history.ts new file mode 100644 index 0000000..7950e9d --- /dev/null +++ b/src/reports/history.ts @@ -0,0 +1,53 @@ +import { createHash } from 'node:crypto'; +import type { HistoryListing, SavedScan } from '../types/history.ts'; +import { escapeHtml } from './html.ts'; +import { maxHtmlBytes } from '../artifacts/filesystem.ts'; + +export function historySummary(record: SavedScan) { + return { scanId: record.scanId, projectId: record.projectId, savedAt: record.savedAt, scanStartedAt: record.snapshot.metadata.startedAt, + scannerVersion: record.snapshot.metadata.scannerVersion, catalogVersion: record.snapshot.metadata.catalogVersion, + catalogFingerprint: record.snapshot.metadata.catalogFingerprint, complete: record.snapshot.complete, readable: true, + integrations: record.identities.integrations, stats: record.snapshot.stats }; +} +export function formatHistoryTerminal(list: HistoryListing): string { + return ['PatchWRK — local scan history (historical snapshots, not current compatibility)', `Project: ${list.projectId}`, `Records: ${list.records.length}; stored file bytes: ${list.bytes}`, + ...list.records.map((record) => `${record.scanId} | ${record.snapshot.metadata.startedAt} | integrations ${record.identities.integrations.length} | attention ${record.snapshot.stats[2]!.value} | unknown ${record.snapshot.stats[3]!.value} | ${record.snapshot.complete ? 'recorded scope complete' : 'partial'} | readable snapshot`), + ...list.warnings.map((warning) => `Warning: ${warning}`), 'Schema validation does not authenticate stored findings. No rescan, upload or monitoring.'].join('\n'); +} +export function formatSavedTerminal(record: SavedScan): string { + return ['PatchWRK — historical snapshot (not current compatibility)', `Scan: ${record.scanId}; project: ${record.projectId}`, `Original scan: ${record.snapshot.metadata.startedAt}; saved: ${record.savedAt}`, + `Scanner: ${record.snapshot.metadata.scannerVersion ?? 'unknown'}; catalog: ${record.snapshot.metadata.catalogVersion}; fingerprint: ${record.snapshot.metadata.catalogFingerprint}`, + ...record.snapshot.stats.map((stat) => `${stat.label}: ${stat.value}`), + ...record.snapshot.sections.find((s) => s.kind === 'findings')!.blocks.flatMap((block) => [ + `${JSON.stringify(block.title)}: ${block.status ?? 'No assessment'}`, ...(block.finding?.uncertainty ?? block.notes).map((note) => ` ${JSON.stringify(note)}`), + ]), 'Use --json for the complete stored privacy-filtered snapshot or --html NEW.html for a historical report. No rescan was performed.'].join('\n'); +} +const historyCss = `*{box-sizing:border-box}body{font:16px/1.6 system-ui,sans-serif;background:#f6f5f1;color:#191a1c;margin:0}main{max-width:1120px;margin:auto;padding:40px 24px}.brand{font-size:18px;font-weight:750;margin:0}h1{font-size:clamp(30px,4.2vw,44px);line-height:1.15;margin:28px 0 12px}h2{font-size:21px;margin:0}p{margin:8px 0}.muted,dt{color:#55585d}.notice{border-left:3px solid #191a1c;padding:12px 18px;margin:24px 0;background:#fff}.stats{display:grid;grid-template-columns:repeat(4,minmax(0,1fr));gap:12px;margin:24px 0}.stat,article,.technical{background:#fff;border:1px solid #deded7;border-radius:12px;padding:20px}.stat strong{display:block;font-size:28px;line-height:1.3}.stat span{font-size:13px;color:#55585d}.latest{margin:20px 0}.timeline{display:grid;gap:18px;margin:24px 0}article header{display:flex;justify-content:space-between;align-items:start;gap:16px}.badges{display:flex;flex-wrap:wrap;gap:8px}.badge{display:inline-block;font-size:12px;font-weight:650;border-radius:20px;border:1px solid #d7dade;background:#f1f2f3;color:#4a5058;padding:4px 10px}.affected{color:#9d2929;background:#fff0f0;border-color:#efc8c8}.review{color:#805000;background:#fff6e6;border-color:#ead4a9}dl{display:grid;grid-template-columns:minmax(160px,1fr) 2fr;gap:8px 20px;margin:18px 0}dd{margin:0}p,dd,code,li{overflow-wrap:anywhere}code{font-size:13px}details{border-top:1px solid #deded7;margin-top:18px;padding-top:14px}summary{cursor:pointer;font-weight:650}summary:focus-visible{outline:3px solid #151515;outline-offset:4px}.technical details{border:0;margin:0;padding:0}footer{font-size:13px;color:#55585d;margin:28px 0}.empty{padding:24px 0}@media(max-width:700px){main{padding:24px 16px}.stats{grid-template-columns:repeat(2,minmax(0,1fr))}article header{display:block}.badges{margin-top:12px}dl{grid-template-columns:1fr;gap:4px}dd{margin-bottom:10px}.stat,article,.technical{padding:16px}}`; + +function displayDate(value: string): string { + return new Intl.DateTimeFormat('en-US', { dateStyle: 'long', timeStyle: 'short', timeZone: 'UTC' }).format(new Date(value)) + ' UTC'; +} + +function historyBadges(record: SavedScan): string { + const statuses = record.identities.findings.map((finding) => finding.status); + const badges: string[] = []; + if (statuses.includes('affected')) badges.push('Affected'); + if (statuses.includes('potentially-affected')) badges.push('Review needed'); + if (record.snapshot.stats[3]!.value > 0 || !record.snapshot.complete || record.identities.integrations.length === 0) badges.push('Unknown'); + if (badges.length === 0) badges.push('No known finding within coverage'); + return badges.join(''); +} + +export function formatHistoryHtml(list: HistoryListing): string { + const hash = createHash('sha256').update(historyCss).digest('base64'); const e = escapeHtml; + // Repository order is newest saved snapshot first; rendering does not mutate records. + const latest = list.records[0]; + const stat = (label: string, value: number | undefined) => `
${value ?? '—'}${label}
`; + const cards = list.records.map((record, index) => { + const title = index === 0 ? 'Latest scan' : index === 1 ? 'Previous scan' : `Scan ${index + 1} — ${displayDate(record.snapshot.metadata.startedAt)}`; + return `

${e(title)}

${e(displayDate(record.snapshot.metadata.startedAt))}

${historyBadges(record)}
Supported integrations
${e(record.identities.integrations.map((i) => `${i.provider} (${i.packageName})`).join(', ') || 'None detected within supported scope')}
Assessments requiring attention
${record.snapshot.stats[2]!.value}
Unknown assessments
${record.snapshot.stats[3]!.value}
Recorded scope
${record.snapshot.complete ? 'Complete within supported scope' : 'Partial — review coverage limits'}
Snapshot readability
Validated snapshot (not authenticated)
Technical details
Full scan UUID
${e(record.scanId)}
Snapshot filename
${e(record.scanId)}.json
Original UTC timestamp
${e(record.snapshot.metadata.startedAt)}
Saved UTC timestamp
${e(record.savedAt)}
Snapshot schema version
${record.schemaVersion}
Validation / readability
Schema validated and readable; local contents are not authenticated.

Reopen explicitly to a new destination; redacted equivalent:

patchwrk history show ${e(record.scanId)} --directory <selected-history> --html <new-report.html>
`; + }).join(''); + const html = `PatchWRK — Scan History

PatchWRK Scanner

Scan History

Review previous local scans and their recorded findings.

${stat('Saved scans (readable)', list.records.length)}${stat('Latest scan: integrations', latest?.snapshot.stats[1]!.value)}${stat('Latest scan: requiring attention', latest?.snapshot.stats[2]!.value)}${stat('Latest scan: unknown assessments', latest?.snapshot.stats[3]!.value)}

Latest original scan: ${latest ? e(displayDate(latest.snapshot.metadata.startedAt)) : 'No saved scan available'}

Newest saved snapshot first. Counts include validated readable records only; unreadable or unknown entries are excluded.

${list.warnings.length ? `` : ''}
${cards || '

No readable saved scans. Save a scan explicitly through the CLI to add a snapshot.

'}
History directory metadata
Project UUID
${e(list.projectId)}
Listing schema version
${list.schemaVersion}
Stored file bytes
${list.bytes}

Attention = Affected + Potential risk; unknowns also require review. No known finding within coverage does not establish application health or guarantee compatibility.

History is managed through explicit CLI commands. No automatic deletion, telemetry, Cloud synchronization or repair. A schema-valid local file does not prove genuine Scanner execution. Review privately before sharing.

`; + if (Buffer.byteLength(html, 'utf8') > maxHtmlBytes) throw new Error('History HTML exceeds the 32 MiB output limit.'); + return html; +} diff --git a/src/reports/html.ts b/src/reports/html.ts index be6b6ce..bfa173f 100644 --- a/src/reports/html.ts +++ b/src/reports/html.ts @@ -15,7 +15,7 @@ const css = ` @media print{body{background:white}main{padding:0;max-width:none}nav,.skip{display:none}.block,.finding,.stat,.technical{border-color:#999;break-inside:avoid}a{text-decoration:underline}.stats{grid-template-columns:repeat(4,minmax(0,1fr))}.integrations{grid-template-columns:1fr}h2{break-after:avoid}details>*{display:block!important}details::details-content{content-visibility:visible;height:auto}} `; -export function formatHtml(view: ReportView): string { +export function formatHtml(view: ReportView, historical?: { scanId: string; savedAt: string }): string { const text = escapeHtml; const rows = (values: { label: string; value: string }[]) => `
${values.map((value) => `
${text(value.label)}
${text(value.value)}
`).join('')}
`; const notes = (values: string[]) => values.length ? `
    ${values.map((note) => `
  • ${text(note)}
    • `).join('')}
` : ''; @@ -38,7 +38,7 @@ export function formatHtml(view: ReportView): string { const html = ` PatchWRK Scanner — Local report
PatchWRK Scanner
Experimental beta

API Compatibility Report

Local, read-only analysis of supported third-party API integrations.

Report generatedScanner ${text(metadata.scannerVersion ?? 'version unknown (source execution)')}Manual scan · no continuous monitoring
-

Overview

A snapshot of supported evidence

${view.stats.map((stat) => `
${text(String(stat.value))}${text(stat.label)}
`).join('')}

Attention = Affected + Potential risk assessments. Unknown assessments also require review. Counts describe recorded findings, not application health.

+${historical ? `

Historical snapshot — not current compatibility
Scan ${text(historical.scanId)} · saved ${text(historical.savedAt)}. All scan timestamps and catalog identity below are original. Rendered from stored evidence without rescanning; schema validation does not authenticate findings.

` : ''}

Overview

A snapshot of supported evidence

${view.stats.map((stat) => `
${text(String(stat.value))}${text(stat.label)}
`).join('')}

Attention = Affected + Potential risk assessments. Unknown assessments also require review. Counts describe recorded findings, not application health.

Scan status${view.complete ? 'Eligible supported scope analyzed' : 'Report generated · partial coverage — investigate unknowns'}

Manual scan · Offline execution · Read-only static analysis

Selected local project; absolute path omitted. Supported scope only. Successful report generation does not establish application compatibility.

${view.coverage.map((item) => `${text(item.label)}: ${text(item.value)}`).join('')}
Invocation — redacted equivalentpatchwrk scan <selected-project> --html <new-report.html>

Raw arguments, target/output paths and API-version values are omitted. Full scope and resource budgets are available in technical details.

${view.notices.length ? `
${notes(view.notices)}
` : ''}

Findings

Historical records · evidence-backed assessments

${findings.map(finding).join('')}

Recommended actions

Manual investigation · no automatic changes

${blocks('actions')}
diff --git a/src/reports/view.ts b/src/reports/view.ts index 3177421..2b4e298 100644 --- a/src/reports/view.ts +++ b/src/reports/view.ts @@ -168,7 +168,7 @@ export function buildReportView(scan: CompatibilityScanReport, metadata: ReportM 'Experimental beta. Current bundled production catalog has four historical Stripe/OpenAI records; other supported providers have no curated records.', 'Static evidence does not establish runtime reachability, production breakage, compatibility or application health. No automatic repair or Cloud monitoring.', 'Report paths are root-relative and may still identify private code. Unsafe path labels and configuration values are withheld. No source excerpts, absolute target paths or raw arguments are stored in HTML. Review privately before sharing.', - 'This visual projection intentionally omits sensitive values and quoted warning details; the original local terminal/JSON preserves existing evidence. No history, comparison, telemetry or upload occurs.', + 'This visual projection intentionally omits sensitive values and quoted warning details; the original local terminal/JSON preserves existing evidence. History requires explicit opt-in. No comparison, telemetry or upload occurs.', 'Fingerprints identify content, not authenticity. Review dates describe manual curation, not continuous freshness. Source-mode version/build identity may be unavailable.', ...scan.warnings, ...scan.coverage.limitations, ...(scan.analysis?.warnings ?? []), ...(scan.analysis?.limitations ?? []), ...scan.compatibility.warnings, ...scan.compatibility.limitations, ].map(narrative) }] }, diff --git a/src/types/history.ts b/src/types/history.ts new file mode 100644 index 0000000..beb68ef --- /dev/null +++ b/src/types/history.ts @@ -0,0 +1,31 @@ +import type { ReportView } from './reporting.ts'; +import type { CompatibilityStatus } from './compatibility.ts'; + +export interface HistoryProject { schemaVersion: 1; projectId: string } +export interface SavedScan { + schemaVersion: 1; + scanId: string; + projectId: string; + savedAt: string; + identities: { + integrations: { provider: string; packageName: string }[]; + findings: { provider: string; packageName: string; changeId: string; reference: string; clientReference: string | null; recordFingerprint: string; status: CompatibilityStatus; scope: string; category: string }[]; + }; + snapshot: ReportView; +} +export interface HistoryListing { + schemaVersion: 1; + projectId: string; + records: SavedScan[]; + warnings: string[]; + bytes: number; +} +export interface HistoryDeletion { + schemaVersion: 1; + applied: boolean; + selected: string[]; + deleted: string[]; + failed: string[]; + warnings: string[]; + lockCleanupFailed: boolean; +} diff --git a/tests/history-dashboard.test.ts b/tests/history-dashboard.test.ts new file mode 100644 index 0000000..50dafa5 --- /dev/null +++ b/tests/history-dashboard.test.ts @@ -0,0 +1,55 @@ +import { createHash } from 'node:crypto'; +import { expect, it } from 'vitest'; +import { formatHistoryHtml } from '../src/reports/history.ts'; +import type { HistoryListing, SavedScan } from '../src/types/history.ts'; + +function record(): SavedScan { + return { + schemaVersion: 1, scanId: '11111111-1111-4111-8111-111111111111', projectId: '22222222-2222-4222-8222-222222222222', savedAt: '2026-10-20T12:01:00.000Z', + identities: { integrations: [{ provider: 'openai', packageName: 'openai' }], findings: [] }, + snapshot: { schemaVersion: 1, metadata: { startedAt: '2026-10-20T12:00:00.000Z' }, complete: true, + stats: [{ label: 'Files scanned', value: 2 }, { label: 'Supported integrations', value: 1 }, { label: 'Assessments requiring attention', value: 0 }, { label: 'Unknown assessments', value: 0 }] } as SavedScan['snapshot'], + }; +} +function listing(records = [record()]): HistoryListing { return { schemaVersion: 1, projectId: 'project', records, warnings: [], bytes: 123 }; } + +it('renders real recorded counts and UTC dates without mutating the listing', () => { + const list = listing(); const before = structuredClone(list); const html = formatHistoryHtml(list); + expect(list).toEqual(before); + expect(html).toContain('

Scan History

'); expect(html).toContain('

Latest scan

'); + expect(html).toContain('October 20, 2026 at 12:00 PM UTC'); + expect(html).toContain('No known finding within coverage'); expect(html).toContain('does not establish application health'); + expect(html).toContain('Newest saved snapshot first'); +}); + +it('preserves reopening metadata in native collapsed details', () => { + const html = formatHistoryHtml(listing()); + const details = html.match(/
Technical details<\/summary>(.*?)<\/details>/su)![1]!; + for (const value of ['11111111-1111-4111-8111-111111111111.json', '2026-10-20T12:00:00.000Z', '2026-10-20T12:01:00.000Z', 'Snapshot schema version', 'not authenticated', 'redacted equivalent:', 'patchwrk history show']) expect(details).toContain(value); + expect(html).not.toMatch(/]*\bopen/u); + expect(html).not.toMatch(/ { + const list = listing([]); list.warnings = ['']; + const html = formatHistoryHtml(list); + expect(html).toContain('No saved scan available'); expect(html).toContain('No readable saved scans'); + expect(html).toContain('—'); expect(html).toContain('<img'); expect(html).not.toContain(' { + const first = record(); first.snapshot.stats[3]!.value = 2; + first.identities.findings = [{ status: 'affected' } as SavedScan['identities']['findings'][number]]; + const html = formatHistoryHtml(listing([first, record(), record()])); + expect(html).toContain('badge affected">Affected'); expect(html).toContain('badge">Unknown'); + expect(html).toContain('

Previous scan

'); expect(html).toContain('

Scan 3 — October 20, 2026'); +}); + +it('hashes only its static stylesheet for CSP and retains mobile layout', () => { + const html = formatHistoryHtml(listing()); const css = html.match(/