Skip to content

fix(auth): align mTLS transport and bound token certificate startup polling and well-known discovery - #14748

Merged
macastelaz merged 8 commits into
googleapis:mainfrom
macastelaz:fix/mtls-transport-startup-polling
Oct 30, 2026
Merged

macastelaz merged 8 commits into
googleapis:mainfrom
macastelaz:fix/mtls-transport-startup-polling

Conversation

@macastelaz

@macastelaz macastelaz commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes two certificate discovery asymmetries between mTLS transport initialization (MtlsUtils / X509Provider) and Agent Identity bound-token acquisition (AgentIdentityUtils):

  1. Add 30-second initial startup polling to MtlsUtils for the well-known SPIFFE directory:

    • When GOOGLE_API_CERTIFICATE_CONFIG (or certConfigPathOverride) and its referenced cert_path / key_path reside in /var/run/secrets/workload-spiffe-credentials/ (or /run/secrets/workload-spiffe-credentials/), MtlsUtils polls for up to 30 seconds (50 × 100ms + 50 × 500ms) until the configuration file, cert_path, and key_path are non-empty readable regular files and the JSON configuration parses cleanly.
    • MtlsUtils.waitForWorkloadConfigStartupIfNeeded(...) consolidates startup config polling across both MtlsUtils (X509Provider) and AgentIdentityUtils.
    • Non-retryable states (missing files outside the well-known SPIFFE directory, directories, unreadable files, valid ECP-only configs, or incomplete workload configs missing cert_path/key_path) fail immediately with 0 sleeps.
  2. Remove unconfigured legacy credentialbundle.pem and certificates.pem + private_key.pem fallbacks in AgentIdentityUtils:

    • When GOOGLE_API_CERTIFICATE_CONFIG is unset, AgentIdentityUtils.getWellKnownCertificatePathWithRetry() now checks only the GKE Pod Certificate bundle (/var/run/secrets/workload-spiffe-credentials/x509.credential-bundle.private-key.pem), matching MtlsUtils.getWorkloadCertPath() and X509Provider 1:1 so AgentIdentityUtils never mints a certificate-bound token for a certificate that the transport ignores.

Testing

  • Unit Tests: Added unit tests in MtlsUtilsTest, X509ProviderTest, and AgentIdentityUtilsTest covering:
    • Delayed config and cert/key creation in /var/run/secrets/workload-spiffe-credentials/
    • Partial/truncated JSON writes during startup polling
    • 0-byte certificate/key files during startup polling
    • Startup poll timeout (99 sleeps) and fail-fast (0 sleeps) for paths outside the well-known SPIFFE directory
    • Ignoring unconfigured credentialbundle.pem and certificates.pem + private_key.pem when GOOGLE_API_CERTIFICATE_CONFIG is unset
  • Manual & Live Cloud Run Verification:
    • Full test setup, 6-scenario local end-to-end simulation output, and live Cloud Run Agent Identity (--functional-type=agent --identity-type=agent-identity) execution logs (agent-bound-token-java-test-4mrbz): https://paste.googleplex.com/6755399436524544

@macastelaz
macastelaz requested review from a team as code owners October 23, 2026 13:40

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces container startup polling in MtlsUtils to wait for credential file readiness when configuration or certificate files reside in the well-known SPIFFE directory, and unifies startup state tracking across mTLS transport initialization and Agent Identity token binding. The feedback suggests minor code simplifications by removing redundant checks in MtlsUtils (specifically in isExistingNonFileOrUnreadable and the exception handling block) and proposes a concurrency optimization to check the startup completion flag inside the polling loop to prevent unnecessary sleep cycles.

@macastelaz macastelaz added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 26, 2026
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 26, 2026
@macastelaz
macastelaz requested a review from nbayati October 27, 2026 03:12
@macastelaz
macastelaz requested a review from nbayati October 30, 2026 02:51
@macastelaz
macastelaz merged commit 0bae268 into googleapis:main Oct 30, 2026
307 of 311 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants