diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 6b34f6265..70f83ee43 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -825,14 +825,14 @@ to **six flavors**. | eco / flavor | vendored artifact | committed wiring | consumption proof | |---|---|---|---| -| npm (package-lock) | deterministic patched tarball `[@scope/]-.tgz`, plus `/.gitignore` (re-includes the tarball against the project's ignores, such as Node.gitignore's `*.tgz`) and `/.gitattributes` (`-text`); every tarball flavor below writes the same pair and refuses `vendor_artifact_gitignored` when git would still drop the tarball | `package-lock.json` only (`npm-shrinkwrap.json` wins when present): every entry matching name+version gets `resolved: "file:…"` + recomputed `integrity`. `package.json` untouched | `npm ci` (integrity-verified). Plain `npm install` preserves the entry; `npm update ` re-resolves and drops it | +| npm (package-lock) | deterministic patched tarball `[@scope/]-.tgz`, plus `/.gitignore` (re-includes the tarball against the project's ignores, such as Node.gitignore's `*.tgz`) and `/.gitattributes` (`* -text -filter -ident -working-tree-encoding`, so neither EOL conversion nor a project filter such as a Git LFS `*.tgz` rule changes the committed bytes; a pre-#1541 `* -text` file is rewritten by the next vendored run); every tarball flavor below writes the same pair and refuses `vendor_artifact_gitignored` when git would still drop the tarball | `package-lock.json` only (`npm-shrinkwrap.json` wins when present): every entry matching name+version gets `resolved: "file:…"` + recomputed `integrity`. `package.json` untouched | `npm ci` (integrity-verified). Plain `npm install` preserves the entry; `npm update ` re-resolves and drops it | | npm / yarn classic | (same tarball) | `yarn.lock` only: matching blocks get `resolved "file:./…#"` + `integrity` (both checksums recomputed; merged-key & `npm:`-alias blocks covered); a patch that rewrites the package's `package.json` gets the blocks' `dependencies:` / `optionalDependencies:` sub-maps recomputed, and is refused `vendor_dep_manifest_unlocked` before any write when a dependency it adds or a range it changes to has no lock block of its own (#591) | `yarn install --frozen-lockfile --offline` (sha1 fragment + sha512 SRI both enforced; byte-stable lock) | | npm / yarn berry (node-modules linker) | (same tarball) | root `package.json` `resolutions` + `yarn.lock` entry with `checksum: 10c0/` of the berry cache-zip (reproduced from the tarball offline). **PnP is refused** (`.pnp.*` → different artifact pipeline) | `yarn install --immutable --check-cache`, cold cache. Refused if `__metadata.cacheKey ≠ 10c0` or a non-default `compressionLevel`. Both files keep their own layout — a CRLF lock (yarn's output on Windows) is spliced in CRLF, `package.json` is re-serialized with its BOM, indent, line ending and trailing-newline shape — so vendor + `--revert` round-trip byte-exactly; a lock or `package.json` MIXING CRLF and LF is refused before any write (`vendor_yarn_berry_mixed_line_endings`) | | npm / pnpm (lockfileVersion 9) | (same tarball) | root `package.json` `pnpm.overrides` (versioned selector) **+** `pnpm-lock.yaml` surgery (overrides / importer version / packages `resolution.integrity` / snapshots) **+** the same override in `pnpm-workspace.yaml` (pnpm >= 10.5 reads it there; created with a root-only `packages:` scaffold when absent, and given that scaffold too when the existing file has no keys at all, since pnpm 8.x–10.4 refuse a workspace file holding a key but no `packages` (#1096); revert removes it again). A project with no `pnpm-workspace.yaml` (or a keyless one) pinned to pnpm 9.0–10.4 (every pin, read as for the hosted trust config) gets no file: those read package.json, and a root-only workspace makes `pnpm add` fail there (#734); a later vendor on pnpm >= 10.5 adds it. Residual: an unpinned project with no install record still gets the scaffold, so on pnpm 9.0–10.4 it needs `pnpm add -w ` or a `packageManager` pin | `pnpm install --frozen-lockfile --offline`, cold store (integrity-verified; byte-stable on pnpm 9 & 10). Other lockfileVersions: 5.4/6.0 route to the legacy backend below; anything else refused | | npm / pnpm LEGACY (lockfileVersion 5.4 = pnpm 7, 6.0 = pnpm 8; flavor `pnpm-legacy`) | (same tarball) | root `package.json` `pnpm.overrides` **+** legacy lock surgery (overrides / root dep + specifiers / packages rekey to a bare `file:` key with recomputed integrity / in-package dep refs). **No `pnpm-workspace.yaml` is written** (pnpm ≤ 8 reads overrides only from package.json). The lock's SPECIFIER is machine-ABSOLUTE — pnpm ≤ 8 absolutizes `file:` overrides itself — surfaced as `vendor_pnpm_legacy_absolute_specifier`. Legacy WORKSPACE locks (`importers:`) refused | same-path `pnpm install --frozen-lockfile --offline`, cold store (byte-stable on pnpm 7.33.5 / 8.15.9). A checkout at a DIFFERENT path fails the frozen check (path-bound specifier) and must run `pnpm install --offline --no-frozen-lockfile` once (the flag matters on CI, where pnpm defaults frozen on), which installs the vendored tarball and re-resolves only the specifier line | | npm / bun (`bun.lock`, lockfileVersion 0, 1 or 2 — `vendor_lockfile_version_unsupported` otherwise) | (same tarball) | `bun.lock` only: the packages entry's registry 4-tuple → local 3-tuple with recomputed `sha512`; the entry's `{deps}` meta, the lock's version line and its line endings are preserved. A lock holding `workspace:` packages is refused `vendor_bun_workspace_unsupported` unless lockfileVersion is 2 — Bun 1.2–1.3 resolve a workspace member's local-tarball path relative to the MEMBER (ENOENT on our root-relative path), 1.4 relative to the lockfile, and a committed version-2 lock is the only proof every consumer runs Bun ≥ 1.4 (a deliberate over-approximation: a package declared only by the workspace root would install on version 1 too). The gate fires only on a run that would WRITE a new local tuple, so in-sync re-runs, `already_vendored` skips and `repair` redownloads on such a lock pass. The detail names the version and the remedy: delete `bun.lock` and re-lock with Bun ≥ 1.4 (an in-place `bun install` keeps the existing lockfileVersion), or `--mode hosted`. Native binary support is described in the next row. `scan`/`get --mode vendored` apply all four refusals BEFORE downloading (see the `get --mode vendored` bullet). Bun 1.1.39–1.3.9 re-save the local tuple WITHOUT its `sha512` on any later lock re-save (`bun add`, `bun install` after a manifest change); the digest-less 2-tuple is recognised as the same wiring — an in-sync re-run stays `already_vendored` and re-pins the digest on disk (no new wiring record) when the committed artifact still holds the bytes the lock was written from — otherwise, as for any stale tuple of ours, the line is re-pinned and the fresh entry carries the new fingerprint — `repair` redownloads through it, and `vendor --revert` / `rollback` restore the registry line over it (a 2-tuple at ANOTHER uuid is still `vendor_lock_entry_drifted`) | `bun install --frozen-lockfile`, cold cache (the local tarball's sha512 is enforced by Bun ≥ 1.3.10; 1.1.39–1.3.9 install it unverified — the committed artifact is the protection there) | | npm / bun binary (`bun.lockb`, native binary format 1, 2 or 3) | (same tarball) | Rewrite matching binary package resolutions and integrity in place; preserve topology and unrelated metadata, update binary offsets and the package metadata hash. A second record of the same `name@version` (Bun writes one for a dependent added after vendoring) is folded into the tarball record — its dependents re-pointed, the record and its own dependency edges dropped and the hoisting trees re-derived with Bun's hoister — so no two records share a tarball resolution, which Bun's isolated linker fails to install (`EEXIST`); where that is not exact (the records' dependencies resolve to different packages, or the lock's hoisting is not one the codec reproduces) both are rewired with `vendor_bun_lockb_duplicate_records`. In a workspace the tarball is also written to each member's `.socket/vendor/npm//` (Bun 0.5.9–1.3 resolve a local tarball relative to the declaring member), with the same `/.gitignore` and `/.gitattributes`; a member uuid dir git ignores as a directory (a member's `.socket/` rule) refuses `vendor_artifact_gitignored` before any write (#1116). Text `bun.lock` takes precedence. `bun_lockb_package` wiring snapshots recover pristine registry metadata for repair and support per-package revert and hosted ↔ vendored migration. Binary discovery and rewrites require no installed Bun runtime. Malformed or unsupported content refuses `vendor_bun_lockb_invalid` before download or takeover. | Frozen installs with the original compatible Bun reader; see `docs/testing/bun-compatibility.md` for the release matrix and historical runtime integrity limits. | -| npm / vlt (`vlt-lock.json`, lockfileVersion 0 or 1 — A0 locks without a version and every other version refuse `vendor_lockfile_version_unsupported`; flavor `vlt`) | patched package **directory** `.socket/vendor/npm//[@scope/]-/node_modules//` (the extra `node_modules/` level lets a package `require()` its own name), its `package.json` without `devDependencies`, plus `/.gitignore` (re-includes the payload against the project's ignores, ignores vlt's links inside it) and `/.gitattributes` (`-text`) | direct dependencies of the root or a workspace member only: the lock node becomes a `file` node for the directory, its importer edges and outgoing edges are re-keyed, and each importer's `package.json` spec becomes `file:`; every moved entry lands where vlt's serializer puts it. A node whose only extra is one peer context (`ṗ:N`, `peer.N`, `peer.<16 hex>`: from vlt 1.0.8 a root dependency with resolved peers, from rc.15 a workspace member's) becomes a `file` node without the extra, as vlt writes `file:` dependencies, keeping its peer edges; revert restores the extra-bearing DepID. Refused before any write: transitive targets (`vendor_vlt_transitive_unsupported`), two or more instances of one `name@version` or a modifier extra, importer `peer` edges, foreign registries, a git, remote-tarball or local-directory node of the same package name (vlt records no version for it), a package `vlt build` would build in place (`vendor_vlt_build_scripts_unsupported`), a name declared in several dependency fields (`vendor_lock_entry_unsupported`), a spec that disagrees with the lock (`vendor_vlt_lock_out_of_sync`), a payload git would ignore (`vendor_artifact_gitignored`), a purl vendored under another flavor (`vendor_flavor_changed`); era-A locks warn `vendor_vlt_legacy_lockfile`; an optional dependency (or any dependency node_modules still links to its installed upstream copy) gets `vendor_vlt_reinstall_required` | fresh checkout, `vlt ci` with cold caches: the patched bytes load and `vlt-lock.json` stays byte-identical, also through a warm and a cold `vlt install --frozen-lockfile` (checked on 1.2.0, 1.0.10, 1.0.4, 1.0.0-rc.32 and 1.0.0-rc.14, and on every release by `docs/testing/vlt-compatibility.md`); no-op installs, `vlt install `, `uninstall` and `vlt update` keep the direct dependency vendored. `vendor --revert` restores the registry node, edges and specs, keeping what vlt re-laid since, and refuses on drift | +| npm / vlt (`vlt-lock.json`, lockfileVersion 0 or 1 — A0 locks without a version and every other version refuse `vendor_lockfile_version_unsupported`; flavor `vlt`) | patched package **directory** `.socket/vendor/npm//[@scope/]-/node_modules//` (the extra `node_modules/` level lets a package `require()` its own name), its `package.json` without `devDependencies`, plus `/.gitignore` (re-includes the payload against the project's ignores, ignores vlt's links inside it) and `/.gitattributes` (the same `* -text -filter -ident -working-tree-encoding` as the tarball flavors) | direct dependencies of the root or a workspace member only: the lock node becomes a `file` node for the directory, its importer edges and outgoing edges are re-keyed, and each importer's `package.json` spec becomes `file:`; every moved entry lands where vlt's serializer puts it. A node whose only extra is one peer context (`ṗ:N`, `peer.N`, `peer.<16 hex>`: from vlt 1.0.8 a root dependency with resolved peers, from rc.15 a workspace member's) becomes a `file` node without the extra, as vlt writes `file:` dependencies, keeping its peer edges; revert restores the extra-bearing DepID. Refused before any write: transitive targets (`vendor_vlt_transitive_unsupported`), two or more instances of one `name@version` or a modifier extra, importer `peer` edges, foreign registries, a git, remote-tarball or local-directory node of the same package name (vlt records no version for it), a package `vlt build` would build in place (`vendor_vlt_build_scripts_unsupported`), a name declared in several dependency fields (`vendor_lock_entry_unsupported`), a spec that disagrees with the lock (`vendor_vlt_lock_out_of_sync`), a payload git would ignore (`vendor_artifact_gitignored`), a purl vendored under another flavor (`vendor_flavor_changed`); era-A locks warn `vendor_vlt_legacy_lockfile`; an optional dependency (or any dependency node_modules still links to its installed upstream copy) gets `vendor_vlt_reinstall_required` | fresh checkout, `vlt ci` with cold caches: the patched bytes load and `vlt-lock.json` stays byte-identical, also through a warm and a cold `vlt install --frozen-lockfile` (checked on 1.2.0, 1.0.10, 1.0.4, 1.0.0-rc.32 and 1.0.0-rc.14, and on every release by `docs/testing/vlt-compatibility.md`); no-op installs, `vlt install `, `uninstall` and `vlt update` keep the direct dependency vendored. `vendor --revert` restores the registry node, edges and specs, keeping what vlt re-laid since, and refuses on drift | | cargo | crate dir `-/` (no `.cargo-checksum.json`) | (v5.0) `[patch.crates-io]` path entry in the **workspace-root `Cargo.toml`** (the manifest beside the `Cargo.lock` it detaches — never `.cargo/config*`) **+** Cargo.lock surgery (the `[[package]]` entry's `source`/`checksum` removed and its `version` set to the copy's TAGGED version `+socket.` — `+.socket.` when the version already has build metadata — with every lock reference that spells the old version rewritten, formats v1–v4; the copy's own `Cargo.toml` version carries the same tag, so the patched crate sees it in `CARGO_PKG_VERSION`; revert restores the lock byte for byte). Key: always the Socket-owned `-socket-` with `package = ""` (the full uuid hex when that key is taken), never the bare crate name — cargo lets a config-file `[patch]` item (project, ancestor directory or `$CARGO_HOME`) replace the manifest item with the same key whatever its version, so keys any of those configs use are avoided and a re-run moves an entry off a now-shadowed key; two versions of one crate are wired side by side. Pre-v5 wiring in `.cargo/config.toml` / `.cargo/config` is moved into `Cargo.toml` by a re-run (`vendor`, `scan`/`get --mode vendored`) or `repair` (`cargo_wiring_migrated` note; the ledger's `cargo_patch_entry` record then names `Cargo.toml`); a detached lock entry left unwired by the pre-v5 multi-version overwrite is re-wired the same way (`cargo_wiring_restored`); every revert removes both spellings | `cargo build --locked --offline` on a fresh checkout — single-version manifest `[patch]` also builds with no network on cargo older than 1.56 (the old config-file wiring's floor); two vendored versions of ONE crate need cargo 1.45 or newer (`--offline` from an empty CARGO_HOME is enough there); older cargo fails closed whatever the index state, and a project that does not pin cargo ≥ 1.45 (`rust-version` or toolchain file) gets the `cargo_multi_version_old_cargo` warning. Note: path deps build **without** `--cap-lints allow` | | golang | module dir `@/` | `go.mod` `replace => ./.socket/vendor/golang//@` | `go build` with `GOPROXY=off` + empty `GOMODCACHE` (directory replaces bypass go.sum entirely; survives `go mod tidy`) | | composer | package dir `/@/`; the copy's `.gitignore` / `.hgignore` are emptied and its `.gitattributes` `export-ignore` rules dropped, because Composer's path mirror skips the files they match (`vendor_composer_mirror_filters_neutralized`; a patch that rewrites one of them is refused `vendor_composer_mirror_filter_conflict`). Re-runs heal copies vendored before this | `composer.lock` only: entry's `dist` → `{type: "path", url, reference: ""}`, `source` removed, `transport-options: {symlink: false}` added. `content-hash` unaffected; `composer.json` untouched | `composer install` (from the lock alone, real copy not symlink, works under `--network none`). Composer 1 does not reinstall an already-installed package whose dist changed: remove `//` first (the hint names the project's resolved vendor dir: `COMPOSER_VENDOR_DIR`, else `config.vendor-dir`, else `vendor`). `composer update ` reverts it. See `docs/testing/composer-compatibility.md` | diff --git a/crates/socket-patch-cli/tests/e2e_vendor_bun_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_bun_build.rs index 3a2b01ccc..a76f1a1a6 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_bun_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_bun_build.rs @@ -1669,3 +1669,115 @@ fn bun_vendor_survives_a_digest_dropping_lock_resave() { ); eprintln!("REVERT AFTER RESAVE OK (bun {})", fx.bun_raw); } + +// ── #1541: a repository that hands *.tgz to Git LFS ─────────────────── + +/// `git` in `cwd` with no user or system config, so an ambient +/// `filter.lfs` (or its absence) can't decide the outcome. +fn git(cwd: &Path, args: &[&str], env: &[(&str, &str)]) -> Output { + let mut cmd = Command::new("git"); + cmd.args([ + "-c", + "user.name=t", + "-c", + "user.email=t@example.com", + "-c", + "commit.gpgsign=false", + "-c", + "init.defaultBranch=main", + ]) + .args(args) + .current_dir(cwd) + .env("GIT_CONFIG_NOSYSTEM", "1") + .env( + "GIT_CONFIG_GLOBAL", + if cfg!(windows) { "NUL" } else { "/dev/null" }, + ); + for (k, v) in env { + cmd.env(k, v); + } + let out = cmd.output().expect("failed to run git"); + assert!( + out.status.success(), + "git {args:?} failed:\n{}", + String::from_utf8_lossy(&out.stderr) + ); + out +} + +/// #1541: the project's `.gitattributes` tracks `*.tgz` with Git LFS. The +/// vendored tarball must still be committed as its own bytes, so a clone +/// that does not smudge LFS (`actions/checkout` defaults, no git-lfs, +/// `GIT_LFS_SKIP_SMUDGE=1`) frozen-installs the patched package. +#[test] +#[serial_test::serial] +fn bun_vendor_tarball_is_not_committed_to_git_lfs() { + let lfs_installed = Command::new("git") + .args(["lfs", "version"]) + .output() + .is_ok_and(|o| o.status.success()); + if !lfs_installed { + println!("SKIP bun_vendor_tarball_is_not_committed_to_git_lfs: git-lfs is not installed"); + return; + } + let Some(fx) = bun_project("vendor-lfs", Target::LeftPad, None) else { + return; + }; + let proj = &fx.proj; + git(proj, &["init", "-q"], &[]); + git(proj, &["lfs", "install", "--local"], &[]); + std::fs::write( + proj.join(".gitattributes"), + "*.tgz filter=lfs diff=lfs merge=lfs -text\n", + ) + .unwrap(); + std::fs::write(proj.join(".gitignore"), "node_modules/\n").unwrap(); + stage_and_vendor(&fx); + git(proj, &["add", "-A"], &[]); + git(proj, &["commit", "-qm", "vendored"], &[]); + + let tgz_rel = vendored_tgz(&fx) + .strip_prefix(proj) + .unwrap() + .to_string_lossy() + .replace('\\', "/"); + let lfs_files = git(proj, &["lfs", "ls-files", "--name-only"], &[]); + assert!( + !String::from_utf8_lossy(&lfs_files.stdout).contains(&tgz_rel), + "the vendored tarball went into LFS: {}", + String::from_utf8_lossy(&lfs_files.stdout) + ); + + let clone = fx.tmp.path().join("lfs-less-clone"); + git( + fx.tmp.path(), + &[ + "clone", + "-q", + proj.to_str().unwrap(), + clone.to_str().unwrap(), + ], + &[("GIT_LFS_SKIP_SMUDGE", "1")], + ); + assert_eq!( + std::fs::read(clone.join(&tgz_rel)).unwrap(), + std::fs::read(vendored_tgz(&fx)).unwrap(), + "the clone must hold the tarball's bytes, not an LFS pointer" + ); + let ci = bun( + &clone, + &["install", "--frozen-lockfile", "--ignore-scripts"], + &fx.tmp.path().join("lfs-less-bun-cache"), + ); + assert!( + ci.status.success(), + "frozen install from the LFS-less clone failed.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&ci.stdout), + String::from_utf8_lossy(&ci.stderr), + ); + assert_eq!( + std::fs::read(fx.target.installed_dir(&clone).join("index.js")).unwrap(), + fx.patched, + "the clone installs the patched bytes" + ); +} diff --git a/crates/socket-patch-cli/tests/e2e_vendor_vlt_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_vlt_build.rs index 0806235cf..bd4cb9263 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_vlt_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_vlt_build.rs @@ -245,7 +245,7 @@ async fn vlt_pinned_matrix_vendored_scan_fresh_ci() { ); assert_eq!( read(&uuid_dir(&fx.proj, fx.t()).join(".gitattributes")), - "* -text\n" + "* -text -filter -ident -working-tree-encoding\n" ); assert_fresh_vendored(&fx, fx.t(), "fresh-scan"); fx.leg.ran(); diff --git a/crates/socket-patch-cli/tests/repair_vendor_flavors_e2e/vlt.rs b/crates/socket-patch-cli/tests/repair_vendor_flavors_e2e/vlt.rs index 7ce0b9331..75d3bb4a0 100644 --- a/crates/socket-patch-cli/tests/repair_vendor_flavors_e2e/vlt.rs +++ b/crates/socket-patch-cli/tests/repair_vendor_flavors_e2e/vlt.rs @@ -281,7 +281,7 @@ async fn vlt_repair_restores_the_uuid_gitignore() { assert_eq!(std::fs::read(uuid.join(".gitignore")).unwrap(), ignore); assert_eq!( std::fs::read_to_string(uuid.join(".gitattributes")).unwrap(), - "* -text\n" + "* -text -filter -ident -working-tree-encoding\n" ); } } diff --git a/crates/socket-patch-cli/tests/vlt_vendor_common/mod.rs b/crates/socket-patch-cli/tests/vlt_vendor_common/mod.rs index a178f1db4..652d4bc4f 100644 --- a/crates/socket-patch-cli/tests/vlt_vendor_common/mod.rs +++ b/crates/socket-patch-cli/tests/vlt_vendor_common/mod.rs @@ -457,7 +457,7 @@ pub fn ledger_entry(root: &Path, purl: &str) -> Value { /// The exact `/.gitignore` and `.gitattributes` bytes (DESIGN §4.2). pub const UUID_GITIGNORE: &str = "!*\n**/node_modules/*/node_modules/\n**/node_modules/@*/*/node_modules/\n"; -pub const UUID_GITATTRIBUTES: &str = "* -text\n"; +pub const UUID_GITATTRIBUTES: &str = "* -text -filter -ident -working-tree-encoding\n"; pub fn git() -> Option { socket_patch_core::utils::process::resolve_tool("git") diff --git a/crates/socket-patch-core/src/vendor/bun_binary.rs b/crates/socket-patch-core/src/vendor/bun_binary.rs index a50d1ea6d..33952f663 100644 --- a/crates/socket-patch-core/src/vendor/bun_binary.rs +++ b/crates/socket-patch-core/src/vendor/bun_binary.rs @@ -1065,10 +1065,6 @@ fn mirror_metadata(path: &Path) -> Vec { }) else { return Vec::new(); }; - let ours = [ - (".gitignore", super::npm_dir::UUID_GITIGNORE), - (".gitattributes", super::npm_dir::UUID_GITATTRIBUTES), - ]; let mut metadata = Vec::new(); let Ok(entries) = std::fs::read_dir(dir) else { return Vec::new(); @@ -1086,15 +1082,16 @@ fn mirror_metadata(path: &Path) -> Vec { } continue; } - let name = entry.file_name(); - match ours.iter().find(|(n, _)| name == *n) { - Some((_, want)) - if read_regular_to_bytes_sync(&child).is_ok_and(|b| b == want.as_bytes()) => - { - metadata.push(child) - } + let ours: fn(&[u8]) -> bool = match entry.file_name().to_str() { + Some(".gitignore") => |b| b == super::npm_dir::UUID_GITIGNORE.as_bytes(), + // Current or pre-#1541 bytes. + Some(".gitattributes") => super::npm_dir::is_uuid_gitattributes, _ => return Vec::new(), + }; + if !read_regular_to_bytes_sync(&child).is_ok_and(|b| ours(&b)) { + return Vec::new(); } + metadata.push(child); } metadata } @@ -1710,8 +1707,17 @@ mod rebuild_tests { "repair restores the mirrors' .gitignore" ); - // A mirror deleted by hand still takes its metadata on revert. + // A mirror deleted by hand still takes its metadata on revert, and + // so does a `.gitattributes` an older release wrote (#1541). std::fs::remove_file(root.join(&mirrors[0])).unwrap(); + for rel in &mirrors { + let attributes = Path::new(rel).parent().unwrap().join(".gitattributes"); + std::fs::write( + root.join(attributes), + super::super::npm_dir::LEGACY_UUID_GITATTRIBUTES, + ) + .unwrap(); + } ts::persist(root, PURL, entry.clone()).await; let outcome = revert(&entry, root, RevertOpts::new(false)).await; assert!(outcome.success, "{outcome:?}"); diff --git a/crates/socket-patch-core/src/vendor/npm_dir.rs b/crates/socket-patch-core/src/vendor/npm_dir.rs index c9bb01a7c..5088fe54b 100644 --- a/crates/socket-patch-core/src/vendor/npm_dir.rs +++ b/crates/socket-patch-core/src/vendor/npm_dir.rs @@ -42,8 +42,21 @@ use super::{VendorOutcome, VendorServiceConfig, VendorWarning}; /// `/.gitignore`, exactly. pub(crate) const UUID_GITIGNORE: &str = "!*\n**/node_modules/*/node_modules/\n**/node_modules/@*/*/node_modules/\n"; -/// `/.gitattributes`, exactly. -pub(crate) const UUID_GITATTRIBUTES: &str = "* -text\n"; +/// `/.gitattributes`, exactly. It unsets every attribute that lets +/// git store or check out other bytes than the artifact's: EOL conversion +/// (`text`), clean/smudge filters such as a repository's `*.tgz +/// filter=lfs` rule (#1541), `ident` expansion and re-encoding. It beats +/// every `.gitattributes` above the uuid dir, but not `.git/info/attributes` +/// or a `.gitattributes` deeper in a vendored directory payload. +pub(crate) const UUID_GITATTRIBUTES: &str = "* -text -filter -ident -working-tree-encoding\n"; +/// The `/.gitattributes` releases before #1541 wrote: still ours, +/// and rewritten to [`UUID_GITATTRIBUTES`] by the next vendored run. +pub(crate) const LEGACY_UUID_GITATTRIBUTES: &str = "* -text\n"; + +/// `bytes` is a `/.gitattributes` socket-patch wrote, current or legacy. +pub(crate) fn is_uuid_gitattributes(bytes: &[u8]) -> bool { + bytes == UUID_GITATTRIBUTES.as_bytes() || bytes == LEGACY_UUID_GITATTRIBUTES.as_bytes() +} const GITIGNORE: &str = ".gitignore"; const GITATTRIBUTES: &str = ".gitattributes"; @@ -1015,8 +1028,86 @@ mod tests { ); assert_eq!( std::fs::read_to_string(tmp.path().join(GITATTRIBUTES)).unwrap(), - "* -text\n" + "* -text -filter -ident -working-tree-encoding\n" + ); + } + + /// #1541: a vendored tree written by an older release holds the + /// legacy `* -text` file; the next run rewrites it in place. + #[tokio::test] + async fn legacy_uuid_gitattributes_is_upgraded() { + let tmp = tempfile::tempdir().unwrap(); + tokio::fs::write(tmp.path().join(GITATTRIBUTES), LEGACY_UUID_GITATTRIBUTES) + .await + .unwrap(); + restore_uuid_metadata(tmp.path()).await.unwrap(); + assert_eq!( + std::fs::read_to_string(tmp.path().join(GITATTRIBUTES)).unwrap(), + UUID_GITATTRIBUTES + ); + assert!(is_uuid_gitattributes(LEGACY_UUID_GITATTRIBUTES.as_bytes())); + assert!(is_uuid_gitattributes(UUID_GITATTRIBUTES.as_bytes())); + assert!(!is_uuid_gitattributes(b"* -text filter=lfs\n")); + } + + /// #1541: a repository rule that hands `*.tgz` (or every file) to Git + /// LFS, or re-encodes or expands it on checkout, does not reach the + /// vendored artifacts once the uuid `.gitattributes` is in place, so + /// the commit stores their exact bytes. + #[cfg(unix)] + #[tokio::test] + async fn the_uuid_gitattributes_overrides_repository_lfs_rules() { + let Some(git) = crate::utils::process::resolve_tool("git") else { + return; + }; + let tmp = tempfile::tempdir().unwrap(); + let root = tmp.path(); + let status = std::process::Command::new(&git) + .args(["init", "-q"]) + .current_dir(root) + .status() + .unwrap(); + assert!(status.success()); + std::fs::write( + root.join(".gitattributes"), + "*.tgz filter=lfs diff=lfs merge=lfs -text\n\ + *.js filter=lfs working-tree-encoding=UTF-16 ident text eol=crlf\n", + ) + .unwrap(); + let uuid = root.join(".socket/vendor/npm/u"); + std::fs::create_dir_all(&uuid).unwrap(); + let paths = [ + ".socket/vendor/npm/u/left-pad-1.3.0.tgz", + ".socket/vendor/npm/u/a-1.0.0/node_modules/a/index.js", + ]; + let attr = |path: &str| { + let out = std::process::Command::new(&git) + .args([ + "check-attr", + "filter", + "text", + "ident", + "working-tree-encoding", + "--", + ]) + .arg(path) + .current_dir(root) + .output() + .unwrap(); + assert!(out.status.success()); + String::from_utf8(out.stdout).unwrap() + }; + assert!( + attr(paths[0]).contains("filter: lfs"), + "the repo rule applies" ); + restore_uuid_metadata(&uuid).await.unwrap(); + for path in paths { + let out = attr(path); + for name in ["filter", "text", "ident", "working-tree-encoding"] { + assert!(out.contains(&format!("{path}: {name}: unset")), "{out}"); + } + } } fn tgz(entries: &[(&str, tar::EntryType, &[u8])]) -> Vec { diff --git a/crates/socket-patch-core/tests/vlt_locks.rs b/crates/socket-patch-core/tests/vlt_locks.rs index 2f9b694e1..b0ba99ddb 100644 --- a/crates/socket-patch-core/tests/vlt_locks.rs +++ b/crates/socket-patch-core/tests/vlt_locks.rs @@ -742,7 +742,7 @@ async fn vendored_wiring_matches_what_vlt_ci_writes_and_reverts_byte_exact() { ); assert_eq!( fs::read_to_string(uuid_dir.join(".gitattributes")).unwrap(), - "* -text\n" + "* -text -filter -ident -working-tree-encoding\n" ); let artifact = staged.root.join(&rel_dir); assert_eq!( diff --git a/docs/ecosystems.md b/docs/ecosystems.md index 6d1e65644..9d6762c2a 100644 --- a/docs/ecosystems.md +++ b/docs/ecosystems.md @@ -655,8 +655,11 @@ package that `require()`s its own name resolve itself. The payload's `devDepende are dropped from its `package.json` (vlt would otherwise try to install them). The uuid directory carries a `.gitignore` that re-includes the payload (`!*`) against the project's own ignores (`node_modules`, `dist/`, `*.map`, …) while ignoring the links vlt creates -inside it, and a `.gitattributes` that turns EOL conversion off so an `autocrlf` checkout -stays byte-exact. The lock's node becomes a `file` node, the importer edges and the +inside it, and a `.gitattributes` (`* -text -filter -ident -working-tree-encoding`) that +turns off EOL conversion and overrides the project's clean/smudge filters, such as a Git +LFS `*.tgz` rule, so an `autocrlf` or LFS-less checkout stays byte-exact (#1541). A tarball +an older release already committed as an LFS pointer stays one until it is re-added: +after the next vendored run, `git add --renormalize .socket/vendor/npm` and commit. The lock's node becomes a `file` node, the importer edges and the importers' `package.json` specs move to the `file:` path, and every moved entry is placed where vlt's own serializer puts it, so `vlt ci` keeps the lock byte-identical. Transitive targets are refused (`vendor_vlt_transitive_unsupported`: vlt silently reverts