Repository navigation
test(workflow): cover async cleanup before cancellation and retry - #7621
Closed
cyhandsome-123 wants to merge 1 commit into
Closed
cyhandsome-123 wants to merge 1 commit into
cyhandsome-123 wants to merge 1 commit into
Conversation
Exercise sibling failures and caller cancellation across static, nested, and dynamic children. Verify that asynchronous cleanup finishes before the invocation returns and cancelled workers are not retried. Verify that a timed-out attempt releases its resources before the next attempt starts and only the recovered output reaches downstream nodes. Add Runner-based contract tests without modifying runtime behavior.
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
cyhandsome-123
marked this pull request as ready for review
October 20, 2026 03:46
copybara-service Bot
pushed a commit
that referenced
this pull request
Oct 20, 2026
Merge #7621 PiperOrigin-RevId: 1003408772
Collaborator
|
Thank you @cyhandsome-123 for your contribution! 🎉 Your changes have been successfully imported and merged via Copybara in commit 3e59429. Closing this PR as the changes are now in the main branch. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link to Issue or Description of Change
Problem: Existing workflow tests check that siblings receive cancellation
and that a timed-out node retries. They do not hold an asynchronous finalizer
open to verify that the invocation waits for resource release, or that a retry
cannot overlap the previous attempt's cleanup.
Solution: Add Runner-based contract tests to the existing failure and timeout
test modules:
and dynamically scheduled children. Keep cleanup blocked, verify the
invocation remains pending, then verify resource release, the original error
or cancellation, and a single worker attempt despite a configured retry policy.
attempt 2 and only the recovered output reaches the downstream node.
The tests use real Workflow/Runner components and the existing invariant-checking
test helper. Events coordinate execution; a 100 ms non-cancelling observation
window checks that blocked cleanup prevents early completion. Five-second
watchdogs bound the positive waits. This change adds seven cases in two existing
files and leaves runtime behavior and public APIs unchanged.
Testing Plan
Unit Tests:
Windows, Python 3.12.14, base
557743c4484c7745f6c52b3bca07727011852e78:python -m pytest tests/unittests/workflow/test_workflow_failures.py tests/unittests/workflow/test_workflow_node_timeout.py -q # 48 passedThe same 48 cases also pass against a wheel installed into a fresh virtual
environment. Both pinned format checks (isort 8.0.1 and pyink 25.12.0) and the
repository compliance/link checks pass.
Two process-local mutation checks validate the assertions, without changing
the submitted source: skipping the workflow cleanup gather fails all six
cancellation cases; retrying immediately after requesting timeout cancellation
fails the timeout case. The unchanged implementation passes all seven cases.
Broader validation limitations:
pytest tests/unittests/workflow -n 4 -q: 1170 passed, 1 xfailed, 2 failed,with no collection errors. Both failures are the unchanged
test_execute_bash_tool_node_runs_command_after_approvalvariants, reproducedindependently; ExecuteBashTool returns a POSIX-only error on Windows.
with uv's "Missing expected target directory for Python minor version link";
the 3.14 setup attempt was stopped after the repeated setup failures.
pre-commit run --files <the two changed tests>passed, and the normalcommit hook passed without bypasses. The all-files run passed formatting,
lint, compliance, documentation and spelling checks; two POSIX script hooks
(
check-new-py-prefixandupdate-constraints) could not start because/bin/bashis unavailable to Windows Python. This is not a claim that thefull suite or every all-files hook is green.
Manual End-to-End (E2E) Tests:
Built the wheel with the project's flit backend, installed it and its runtime
dependencies in a fresh venv, and ran a standalone public
InMemoryRunnerscenario with a dynamic worker, async resource finalizer and failing sibling.
No model endpoint, cloud credentials or external service is needed.
Reproduce the offline Runner check
After installing the locally built wheel, save this as
lifecycle_demo.pyandrun
python lifecycle_demo.py:The logged ValueError is the deliberately injected sibling failure. The script
asserts that the original error is propagated and exits successfully.
Checklist
Additional context
Prepared with AI assistance and verified locally. This is a focused test-only
contribution, with the validation limitations above left explicit for review.