Skip to content

0.48.2-RC1: #3832, #3858, #3857 on top of 0.48.1-RC4 - #3863

Merged
icanhasmath merged 4 commits into
version/0-48-2-RC1from
marcg/0-48-2-RC1-from-rc4
Oct 23, 2026
Merged

0.48.2-RC1: #3832, #3858, #3857 on top of 0.48.1-RC4#3863
icanhasmath merged 4 commits into
version/0-48-2-RC1from
marcg/0-48-2-RC1-from-rc4

Conversation

@icanhasmath

@icanhasmath icanhasmath commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator

0.48.2-RC1 (PROD-240), continuing from the last release: version/0-48-1-RC4 (0.48.1). version/0-48-2-RC1 is that tip plus mitchell's version.txt bump to 0.48.2-RC1. This PR adds only the changes merged to master since then, cherry-picked with -x, plus the changelog:

The older master-only work listed in #3862 (option A) is not included.

After this merges, the release is version/0-48-2-RC1 → beta.

Checked locally:

  • All three cherry-picks applied without conflicts.
  • After the constants-generator step, go build ./cmd/... passes.
  • The unit tests for the touched packages (smartlink, python/wheel, runners/publish, platform/model, runtime) pass with -vet=off, as state run test runs them.

Changelog

Added

  • state publish --build now reads [project].dependencies from pyproject.toml and publishes them as runtime
    dependencies of the encrypted private ingredient. Version ranges carry over, and OS and Python version markers become
    dependency conditions (unsupported markers are skipped with a notice).
  • Added state export ingredient-meta <namespace>/<name>[@version], which writes an existing ingredient's meta file for
    use with state publish --edit --meta.

Changed

  • Installing runtimes on Windows is significantly faster: link paths are resolved once per tree instead of once per
    file, and artifacts are linked concurrently.
  • A --meta file's name, version, and description now feed the built wheel (flags take precedence over the meta file,
    which takes precedence over pyproject.toml).

Fixed

  • state publish --edit no longer overwrites the meta file's dependencies and authors with the previous revision's,
    keeps inherited dependency types and conditions, and rejects author or description changes on an existing version
    up front.
  • state publish --edit --meta with an exported meta file now edits the version named in the file rather than the
    latest version.

Security

icanhasmath and others added 4 commits October 22, 2026 21:03
@icanhasmath @claude
Speed up file installation on Windows (#3832)
9d54ba5 
* Resolve link paths once per tree instead of once per file

smartlink.Link resolved both src and dest via ResolveUniquePath for every
entry it recursed into. On Windows ResolveUniquePath calls GetLongPathName,
which is a syscall, so installing a runtime with many files performed two
extra syscalls per file - a major contributor to slow Windows installs.

The tree root is already resolved by LinkContents (and now by the public
Link), and descendant paths are built by joining an already-resolved parent
with real (long) entry names, so they need no further resolution. Split the
recursion into an unexported link() worker that assumes resolved inputs and
does not re-resolve, keeping resolution only for the rare symlink-to-file
case where the pre-existing behavior links to the dereferenced target.

Also treat an "already exists" error from linkFile the same as the existing
TargetExists skip, so concurrent deploys of a shared file degrade to the
documented "first one wins" behavior rather than failing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Deploy linked artifacts concurrently instead of serializing them

DeployViaLink held the depot-wide fsMutex for the entire link operation, so
even though the install worker pool runs several artifacts at once, the
actual file linking was serialized to one artifact at a time. On Windows,
where each link is an individual (relatively slow) syscall, this made the
install phase far slower than it needed to be.

Linking touches no shared in-memory depot state (Exists guards its own map
via mapMutex and depotPath is immutable after construction), directory
creation is idempotent, and smartlink now treats an already-existing target
as a skip, so it is safe to run without the lock. Drop it from DeployViaLink
so linking parallelizes across the worker pool.

DeployViaCopy keeps the lock: two artifacts copying the same file would race
on os.Create and could corrupt the destination. That path is only used for
artifacts needing file transforms, when hard links are unsupported, or in
portable mode, so it remains the less common case.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Hold a read lock while linking so copies can't write through links

Linking without the depot lock let a concurrent copy deployment write
through a link created moments earlier: copyFiles checks the destination
is free, a link to another artifact's depot file appears there, and
os.Create then truncates and overwrites that depot file, corrupting the
shared cache. TestConcurrentCopyAndLinkKeepDepotIntact reproduces it in
every run without the lock.

Make fsMutex an RWMutex: DeployViaLink holds it for reading, so links
still run concurrently, while copies, Put and Undeploy hold it for
writing. Also fix the non-constant format string in DeployViaCopy, which
made go test's vet step fail the whole package.

Add smartlink tests for an existing target, concurrent links to the same
destination, and the Link refactor's handling of directories and
symlinks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Skip per-file metadata calls when linking

link() checked each file with IsDir, IsSymlink, MkdirUnlessExists on its
parent and TargetExists before linking it, plus another IsDir inside the
Windows linkFile. Each is a file open on Windows. Instead:

- take the entry type from the directory listing (os.ReadDir already
  has it), so only symlinks need a Stat;
- rely on the parent directory existing, since the recursion creates it
  (Link creates it for its single entry);
- drop the TargetExists pre-check and treat the link call's "already
  exists" error as the skip, which already covered concurrent links;
- check for a directory only on the symlink-to-directory path, which is
  the only one that can pass one to linkFile.

A regular file now costs one syscall: the link itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 8f473b3)
@icanhasmath @claude
Remediate CVE-2026-63209 in klauspost/compress (#3858)
4edce2e 
Bump github.com/klauspost/compress (indirect, via mholt/archives) from
v1.18.0 to v1.18.7, the first patched release for GHSA-259r-337f-4rfw
(integer overflow and out-of-bounds write in s2). Re-vendored.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit b09f7ee)
@icanhasmath @claude
Support dependencies for encrypted private ingredients (#3857)
cca44aa 
* Support dependencies for encrypted private ingredients

`state publish --build` can now declare and edit the dependencies of an
encrypted private ingredient:

- Read [project].dependencies from pyproject.toml and publish them as
  runtime dependencies on language/python packages, with names normalized
  by the inventory API. Version ranges carry over (~= and ==X.* are
  expanded). OS and Python version markers become conditions
  (kernel/Windows|Linux|Darwin, language/python); unsupported markers are
  skipped with a notice. The wheel's METADATA records them as
  Requires-Dist.
- Read the --meta file before building, so a meta file's name, version
  and description feed the wheel (flags > meta file > pyproject.toml).
- Add `state export ingredient-meta <namespace>/<name>[@Version]`, which
  writes an existing ingredient's meta file for use with --edit --meta.

Fix --edit along the way: keep the meta file's dependencies and authors
instead of overwriting them with the previous revision's, keep inherited
dependency types and conditions, add the private-builder dependency only
once, and reject author/description changes on an existing version up
front instead of failing in the API. InventoryRequirementsToString no
longer panics on requirements without a version.

Add a readme walking through the publish and edit workflow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Address review: keep inherited deps, edit the meta file's version

- When every pyproject.toml dependency is skipped, keep the dependencies
  already in place (on --edit, those inherited from the previous
  revision) instead of replacing them with an empty list.
- Let a meta file's version select the version to publish or edit, like
  --version (which still takes precedence). An exported meta file passed
  back with `state publish --edit --meta` without --build was editing
  the latest version instead of the exported one.

Add tests for both, for the up-front author/description rejection on
--edit, for InventoryRequirementsToString with requirements missing a
version or comparator, and for inherited conditions that carry
requirements.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 44e750f)
@icanhasmath @claude
Add 0.48.2 changelog.
6e97d19 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@icanhasmath
icanhasmath enabled auto-merge (rebase) October 23, 2026 02:06
@icanhasmath
icanhasmath requested a review from MDrakos October 23, 2026 02:07
@icanhasmath
icanhasmath disabled auto-merge October 23, 2026 16:13
@icanhasmath
icanhasmath merged commit e15cf90 into version/0-48-2-RC1 Oct 23, 2026
14 of 15 checks passed
@icanhasmath
icanhasmath deleted the marcg/0-48-2-RC1-from-rc4 branch October 23, 2026 16:13
This was referenced Oct 23, 2026
Merged
Open
MDrakos pushed a commit that referenced this pull request Oct 23, 2026
@icanhasmath @mitchell-as @claude
0.48.2-RC1 (#3864)
89954c6 
* Update version.txt

* 0.48.2-RC1: #3832, #3858, #3857 on top of 0.48.1-RC4 (#3863)

* Speed up file installation on Windows (#3832)

* Resolve link paths once per tree instead of once per file

smartlink.Link resolved both src and dest via ResolveUniquePath for every
entry it recursed into. On Windows ResolveUniquePath calls GetLongPathName,
which is a syscall, so installing a runtime with many files performed two
extra syscalls per file - a major contributor to slow Windows installs.

The tree root is already resolved by LinkContents (and now by the public
Link), and descendant paths are built by joining an already-resolved parent
with real (long) entry names, so they need no further resolution. Split the
recursion into an unexported link() worker that assumes resolved inputs and
does not re-resolve, keeping resolution only for the rare symlink-to-file
case where the pre-existing behavior links to the dereferenced target.

Also treat an "already exists" error from linkFile the same as the existing
TargetExists skip, so concurrent deploys of a shared file degrade to the
documented "first one wins" behavior rather than failing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Deploy linked artifacts concurrently instead of serializing them

DeployViaLink held the depot-wide fsMutex for the entire link operation, so
even though the install worker pool runs several artifacts at once, the
actual file linking was serialized to one artifact at a time. On Windows,
where each link is an individual (relatively slow) syscall, this made the
install phase far slower than it needed to be.

Linking touches no shared in-memory depot state (Exists guards its own map
via mapMutex and depotPath is immutable after construction), directory
creation is idempotent, and smartlink now treats an already-existing target
as a skip, so it is safe to run without the lock. Drop it from DeployViaLink
so linking parallelizes across the worker pool.

DeployViaCopy keeps the lock: two artifacts copying the same file would race
on os.Create and could corrupt the destination. That path is only used for
artifacts needing file transforms, when hard links are unsupported, or in
portable mode, so it remains the less common case.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Hold a read lock while linking so copies can't write through links

Linking without the depot lock let a concurrent copy deployment write
through a link created moments earlier: copyFiles checks the destination
is free, a link to another artifact's depot file appears there, and
os.Create then truncates and overwrites that depot file, corrupting the
shared cache. TestConcurrentCopyAndLinkKeepDepotIntact reproduces it in
every run without the lock.

Make fsMutex an RWMutex: DeployViaLink holds it for reading, so links
still run concurrently, while copies, Put and Undeploy hold it for
writing. Also fix the non-constant format string in DeployViaCopy, which
made go test's vet step fail the whole package.

Add smartlink tests for an existing target, concurrent links to the same
destination, and the Link refactor's handling of directories and
symlinks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Skip per-file metadata calls when linking

link() checked each file with IsDir, IsSymlink, MkdirUnlessExists on its
parent and TargetExists before linking it, plus another IsDir inside the
Windows linkFile. Each is a file open on Windows. Instead:

- take the entry type from the directory listing (os.ReadDir already
  has it), so only symlinks need a Stat;
- rely on the parent directory existing, since the recursion creates it
  (Link creates it for its single entry);
- drop the TargetExists pre-check and treat the link call's "already
  exists" error as the skip, which already covered concurrent links;
- check for a directory only on the symlink-to-directory path, which is
  the only one that can pass one to linkFile.

A regular file now costs one syscall: the link itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 8f473b3)

* Remediate CVE-2026-63209 in klauspost/compress (#3858)

Bump github.com/klauspost/compress (indirect, via mholt/archives) from
v1.18.0 to v1.18.7, the first patched release for GHSA-259r-337f-4rfw
(integer overflow and out-of-bounds write in s2). Re-vendored.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit b09f7ee)

* Support dependencies for encrypted private ingredients (#3857)

* Support dependencies for encrypted private ingredients

`state publish --build` can now declare and edit the dependencies of an
encrypted private ingredient:

- Read [project].dependencies from pyproject.toml and publish them as
  runtime dependencies on language/python packages, with names normalized
  by the inventory API. Version ranges carry over (~= and ==X.* are
  expanded). OS and Python version markers become conditions
  (kernel/Windows|Linux|Darwin, language/python); unsupported markers are
  skipped with a notice. The wheel's METADATA records them as
  Requires-Dist.
- Read the --meta file before building, so a meta file's name, version
  and description feed the wheel (flags > meta file > pyproject.toml).
- Add `state export ingredient-meta <namespace>/<name>[@Version]`, which
  writes an existing ingredient's meta file for use with --edit --meta.

Fix --edit along the way: keep the meta file's dependencies and authors
instead of overwriting them with the previous revision's, keep inherited
dependency types and conditions, add the private-builder dependency only
once, and reject author/description changes on an existing version up
front instead of failing in the API. InventoryRequirementsToString no
longer panics on requirements without a version.

Add a readme walking through the publish and edit workflow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Address review: keep inherited deps, edit the meta file's version

- When every pyproject.toml dependency is skipped, keep the dependencies
  already in place (on --edit, those inherited from the previous
  revision) instead of replacing them with an empty list.
- Let a meta file's version select the version to publish or edit, like
  --version (which still takes precedence). An exported meta file passed
  back with `state publish --edit --meta` without --build was editing
  the latest version instead of the exported one.

Add tests for both, for the up-front author/description rejection on
--edit, for InventoryRequirementsToString with requirements missing a
version or comparator, and for inherited conditions that carry
requirements.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 44e750f)

* Add 0.48.2 changelog.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* check-format: don't fail when a PR changes no files.

With an empty diff, grep -v exits 1 and set -e aborts the script with no
output, so a PR that changes nothing (e.g. merging beta into a version
branch) fails Check Format on every OS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Update to Go 1.26.10 and golang.org/x/net v0.60.0 to remediate CVEs.

The CI Trivy scan fails on 13 Go stdlib CVEs (fixed in 1.26.9/1.26.10)
and 11 golang.org/x/net findings (fixed in v0.60.0) published since the
0.48.1 scan. Re-vendored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Compare RSA keys by value in keypair tests.

The tests compared a generated keypair with the same key parsed back from
PEM using deep struct equality, which also compares rsa.PrivateKey's
precomputed internals. Under Go 1.26.10 those can differ (e.g. a leading
zero byte in dP) even though the keys are identical, so the tests failed
intermittently. Compare with rsa.PrivateKey.Equal instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Allow for slow sw_vers in the macOS OSVersion performance test. (#3866)

TestPerformance/OSVersion averages ten calls against a 10ms budget, so
the single uncached call bounds it at ~100ms. On macOS that call shells
out to sw_vers, which took 115-716ms on macos-latest runners and failed
3 of 4 runs. Cached calls take nanoseconds. Keep 10ms elsewhere; allow
100ms on darwin.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: mitchell <mitchellb@activestate.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@MDrakos MDrakos MDrakos approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@icanhasmath @MDrakos