Skip to content

Allow Foundry smoke tests to finish model retries - #59

Merged
Changyong Gong (chagong) merged 2 commits into
mainfrom
chagong-foundry-deployment-failure
Oct 19, 2026
Merged

Changyong Gong (chagong) merged 2 commits into
mainfrom
chagong-foundry-deployment-failure

Conversation

@chagong

Copy link
Copy Markdown
Collaborator

Problem

Deployment run 39211267548 published an active agent version but failed the Responses smoke check. Foundry reported a transient no_capacity failure, and the SDK's retry completed after the caller's 120-second deadline had canceled the request. The Invocations smoke check was consequently skipped.

Changes

  • Increase both Responses and Invocations CLI deadlines from 120 to 300 seconds, with six-minute step limits for setup and validation headroom.
  • Preserve the 40-minute job bound, exact-reply/terminal-state validation, pinned version and fresh sessions, approval/CI gates, secret handling, and failure behavior. Do not add workflow-level retries, rollback, or model/configuration changes.
  • Add executable Responses coverage for success, malformed/incomplete/canceled streams, wrong replies, duplicate completion, error events, and nonzero CLI exits; update both protocol timeout assertions.
  • Document the new bounds and their limits: additional retry headroom does not guarantee model capacity.

Validation

  • py -m unittest discover -s tests -p test_deploy_foundry_workflow.py — 34 tests passed using Python 3.12, Git Bash, and jq 1.8.1 locally.
  • actionlint -shellcheck= -pyflakes= .github/workflows/deploy-foundry.yml — passed with actionlint 1.7.12.
  • git diff --check — passed.

All invocations in the regression tests are local stubs. No live invocation, deployment, rollback, quota, or Azure configuration change was performed. Python 3.13/Linux execution remains covered by PR CI; live verification requires a separately authorized operation.

Extend both protocol invocation deadlines to 300 seconds within six-minute steps. Preserve deployment gates and strict reply validation, cover Responses execution failures, and document the bounded retry headroom.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The narrow timeout changes are consistent across workflow, tests, and documentation, with no blocking issues identified.

No open issues 🎉

What changed in this PR

Extends Foundry deployment smoke-test deadlines so existing SDK model retries have more time to finish.

Changes:

  • Raises both protocol deadlines to 300 seconds within six-minute steps.
  • Adds executable Responses regression coverage and updates timeout assertions.
  • Documents retry headroom and capacity limitations while preserving existing safeguards.
File Description
tests/​test_deploy_foundry_workflow.py Adds Responses coverage and checks updated timeout bounds.
docs/​guide.md Explains longer deadlines and their limitations.
.github/​workflows/​deploy-foundry.yml Extends both smoke-test invocation and step limits.

Review effort: Balanced

@chagong
Changyong Gong (chagong) merged commit c431d42 into main Oct 19, 2026
7 checks passed
@chagong
Changyong Gong (chagong) deleted the chagong-foundry-deployment-failure branch October 19, 2026 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants