You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[DOCS] Chat example says role-only stream chunks validate, but the model rejects them #147
The Streaming Chat Completion snippet in examples/chat/README.md says "The first chunk carries only role (no content) and validates fine." The code contradicts this. The generated ChatCompletionStreamResponseDelta declares content: str with no default (inference_gateway/models.py:2408), and ChatCompletionStreamChoice.finish_reason is a required FinishReason whose literal does not allow null (inference_gateway/models.py:2479 and :800-803). The repo's own strict xfail test test_stream_response_delta_allows_missing_content (tests/test_client.py:487-511) asserts that ChatCompletionStreamResponseDelta.model_validate({"role": "assistant"}) raises ValidationError. The snippet only catches json.JSONDecodeError, so a role-only chunk, or any chunk with "finish_reason": null, raises an uncaught pydantic.ValidationError out of the loop. examples/chat/main.py, which the README tells users to run, uses the same except json.JSONDecodeError-only pattern. The root README.md "Streaming Content" section already handles this correctly: it catches (json.JSONDecodeError, ValidationError) and skips the chunk.
Acceptance Criteria
examples/chat/README.md: remove the comment that a role-only first chunk "validates fine". CreateChatCompletionStreamResponse.model_validate raises ValidationError for a delta without content and for a choice whose finish_reason is null.
examples/chat/README.md: catch pydantic.ValidationError alongside json.JSONDecodeError in the streaming snippet and skip the chunk, matching the root README.md "Streaming Content" example.
examples/chat/main.py: apply the same ValidationError handling so python main.py does not crash on a chunk the strict model rejects.
Gather context (read example + root README.md pattern)
Understand request (docs/example fix)
examples/chat/README.md: removed the "validates fine" claim, now notes the model rejects a delta without content and a null finish_reason; snippet catches (json.JSONDecodeError, ValidationError) and continues
examples/chat/main.py: added a ValidationError handler so the script skips rejected chunks instead of crashing
Commit + open PR
Lint/test - not run: pip install is blocked in this sandbox, so black/isort/mypy/pytest were unavailable. Add Bash(pip install:*) to --allowedTools if you want them run here. CI will cover it; no library code changed.
Import order in the README snippet was aligned with the root README.md ("Streaming Content"). No SDK behavior change, so no [DOCS] ticket and no downstream repo impact.
Activity
edenreich commentedon Oct 4, 2026
@claude can you work on this? use /ponytail:ponytail
inference-gateway-maintainer commentedon Oct 4, 2026
Claude finished @edenreich's task in 1m 14s —— View job •
docs/issue-147-20261004-1152• [Create PR ➔](https://github.com/inference-gateway/python-sdk/compare/main...docs/issue-147-20261004-1152?quick_pull=1&title=Issue%20%23147%3A%20Changes%20from%20Claude&body=This%20PR%20addresses%20issue%20%23147%0A%0AGenerated%20with%20%5BClaude%20Code%5D(https%3A%2F%2Fclaude.ai%2Fcode)Done - PR #150
README.mdpattern)examples/chat/README.md: removed the "validates fine" claim, now notes the model rejects a delta withoutcontentand a nullfinish_reason; snippet catches(json.JSONDecodeError, ValidationError)andcontinuesexamples/chat/main.py: added aValidationErrorhandler so the script skips rejected chunks instead of crashingpip installis blocked in this sandbox, soblack/isort/mypy/pytestwere unavailable. AddBash(pip install:*)to--allowedToolsif you want them run here. CI will cover it; no library code changed.Import order in the README snippet was aligned with the root
README.md("Streaming Content"). No SDK behavior change, so no[DOCS]ticket and no downstream repo impact.PR: #150
| Branch
inference-gateway-releaser commentedon Oct 21, 2026
🎉 This issue has been resolved in version 0.22.0 🎉
The release is available on GitHub release
Your semantic-release bot 📦🚀