Repository navigation
fix(tasks): make the silent-model stuck test independent of setup time - #482
Merged
itsskofficial merged 1 commit intoOct 21, 2026
Conversation
ShryukGrandhi
requested review from
Kabeer2004 and
itsskofficial
as code owners
October 20, 2026 07:30
5 tasks done
itsskofficial
approved these changes
Oct 21, 2026
itsskofficial
left a comment
Contributor
There was a problem hiding this comment.
Thanks @ShryukGrandhi, nice diagnosis! Reproducing it with a slowed-down setup was exactly the right move. Locally, main failed 2 of 45 runs next to the full suite. Your version passed all 45 there, plus 50 quiet runs and 64 runs at 16 at once. Moving the time jump to after the model is called makes the test independent of setup time instead of just stretching a deadline, and it's about 4x faster too. Approving and merging.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
test_a_silent_model_gets_stuck_and_cancel_ends_the_runfailed on the Windows engine tests of #406 withassert 0 == 1on its last line,llm.stream_calls == 1.The test set
tasks.stuck_after_minutes = 0.01, which is about 0.6 s of real time. The executor starts the no-activity timer (stall = asyncio.timeout(...)intasks/executor.py) before it builds the system prompt, recalls memories and saves the first transcript. On a busy runner that setup can take longer than 0.6 s. When it does, the run is marked stuck before the model is ever called. The reason text still matches ("the AI model hasn't answered"), so every assertion passes except the final call count, which is 0.The fix follows the other no-activity tests in the file, which already skip time with
skip_clockinstead of relying on real time. The silent model now moves the clock past a 30 s limit only once it has actually been called, then stays silent. To time out early now, setup would need 30 s of real time, and the outcome no longer depends on how fast the machine is. This is a test-only change, so the engine's behaviour stays the same, and noflakymarker was needed.Fixes #438
How I tested it
_executor_system_promptsleep 1 s before building the prompt (a pytest plugin outside the repo, not committed). Before the fix the test fails withassert 0 == 1, exactly as on CI. After the fix it passes with that 1 s delay, and also with a 5 s delay.pytest -q tests/tasks/test_stuck_runs.pypasses 8/8, run 5 times in a row.pytest -q tests/taskspasses (374 tests).ruff check sentient tests packagingis clean.pytest -qrun: 2740 passed, 7 failed. The 7 failures are in voice, Docker sandbox, Claude Code and hardware-detection tests, none of which this change touches. When I re-ran them on the unmodified test file, six failed the same way and the seventh passed, so none of them come from this change.Real test
No model involved: this only changes a unit test's timing. I checked it against the executor code it exercises (the stall timer in
tasks/executor.py) and forced a slow setup to reproduce the CI failure.Checklist
pytest/npm run typecheckpass locally for what I touchedruff check sentient testsis cleandocs/API.mdupdated if the desktop ↔ engine contract changed (not changed)changes/if users will notice this change (test only, not needed)Summary by CodeRabbit