Repository navigation
feat(openai): emit AG-UI text and tool events - #161
Conversation
Handlers can yield the AG-UI text and tool subset. config().stream() still yields chunk then done. stream() is the opt-in entry. Co-authored-by: alexis <alexis@launchdarkly.com>
A handler that raises mid-call can still report the lifecycle without the tool result. Co-authored-by: alexis <alexis@launchdarkly.com>
Messages and agents map Responses stream events and the tool loop onto the shared subset. Argument text and results follow capture_content. Co-authored-by: alexis <alexis@launchdarkly.com>
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Do not close a call from the tool hook once argument deltas have started. Read hosted tool output from response.output_item.done, and close every open call when the messages stream fails. Co-authored-by: alexis <alexis@launchdarkly.com>
|
bugbot run |
File-search results and code-interpreter outputs are SDK models, so json.dumps marked a successful call failed. A streamed prefix no longer blocks the rest of the arguments, and a messages failure or a call that never becomes a function result still gets TOOL_CALL_RESULT. Co-authored-by: alexis <alexis@launchdarkly.com>
|
bugbot run |
jeffdupont
left a comment
jeffdupont
left a comment
There was a problem hiding this comment.
Reviewed at 4d7504c against base b7a9f3a (#164). 263 tests pass. Findings 1 and 2 come from replaying the event order against this PR's TurnEmitter, not from a live OpenAI run.
Must fix
Both come from one timing gap in openai-agents: the Runner works in a background task, puts raw model events on a queue (put_nowait, agents/run_internal/run_loop.py:1484) and calls our hooks directly. Hooks mutate the shared emitter immediately; raw events reach it only when the consumer reads them. A slow consumer (proxy, backpressure) reorders the two.
1. Tool arguments are published truncated when the consumer is slow (openai-agents/.../handler.py:327-337, 354, 790-791)
Consumer has read output_item.added and {"q":; the tool finishes before more is read. on_tool_start skips closing because deltas exist (L330); on_tool_end → tool_result closes the call with no arguments (agui.py:202); later deltas are dropped as ended (agui.py:163). Replay output: START, ARGS '{"q":', END, RESULT. test_a_streaming_prefix_is_not_closed_by_the_tool_hook covers only the on_tool_start half.
Fix: in on_tool_end, before tool_result, self.pending.extend(self.emitter.tool_end(str(call_id), arguments=getattr(context, "tool_arguments", None))), plus a test replaying this order.
2. Two model turns merge into one message when the consumer is slow (openai-agents/.../handler.py:260)
on_llm_start closes the open text via pending. Turn 1 "Let me " read, "check." still queued, turn 2 starts: msg-1 closes early, the late "check." opens msg-2, and turn 2's text joins it. Replay: msg-1 = "Let me ", msg-2 = "check.Answer: 42".
Fix: close text from the stream itself, in read order: in _map_responses_event, if kind == "response.created": return emitter.end_text(), and delete L260. (#165 does the same on message_start.)
Should fix
3. The hosted-tool mapping can't run, and it's in both packages (openai-messages/.../handler.py:447-524, 551-556; openai-agents/.../handler.py:606-683, 710-715)
_build_tools builds only "type": "function" (openai-messages/.../handler.py:53-65) and _build_agent_tools only FunctionTool (openai-agents/.../handler.py:79-117, Agent(...) at 219-224), so mcp_call, custom_tool_call, file_search_call and code_interpreter_call events never arrive. ~75 lines per copy plus tests.
Fix: delete _HOSTED_OUTPUT_FIELD, _HOSTED_TOOL_NAME, _json_safe, _hosted_item_result, the mcp_*/custom_tool_call_* keys, the mcp_call.failed and output_item.done branches, and test_hosted_* / test_model_objects_are_json_not_failures. Re-add when hosted tools are wired.
4. The mapper is byte-identical in both packages (openai-messages/.../handler.py:437-560 vs openai-agents/.../handler.py:596-719)
Fixes 1–2 and any new event type must be made twice; openai-agents doesn't depend on openai-messages. After 3 it's ~30 lines: put one copy where both already depend, or at least cross-reference the copies.
5. Tool-result content differs between the two OpenAI handlers (openai-messages/.../handler.py:738 vs openai-agents/.../handler.py:107, 354)
A tool returning {"a": 1}: openai-messages emits '{"a":1}' while the model was sent str(result) = {'a': 1} (L744); openai-agents emits {'a': 1}. #163 and #165 use yet other rules.
Fix: report what the model received, emitter.tool_result(str(tc.call_id), str(result)), and pin the rule in the spec.
6. No test sends function_call_arguments.delta (openai-messages/tests/test_handler.py)
Deltas carry only item_id (fc_…), mapped to call_id via item_to_call. If that breaks, deltas go to an unknown id and are silently dropped (agui.py:162). One test: two parallel function_call items with interleaved deltas, asserting each toolCallId gets its full argument string.
7. Shared-reader edit duplicated with #163, and no spec / JS counterpart
This PR edits agui.py (_arg_text, new tool_end, agui.py:107, 167, 170-194); #163 has the same hunk (only has_argument_deltas, L154-156, is unique here). Move it into #164 and rebase. The monorepo has no AG-UI spec and js-ai-sdk has nothing matching.
Nice to have
- The
tool_enddocstring says "a partial delta cannot publish truncated JSON", but when the streamed prefix isn't a prefix of compactargument_text(arguments),extrais""and the call ends truncated (hits LangChain in #163, not this PR). Weaken the docstring or fall back to "streamed text wins"; belongs in #164.
Confirmed correct: parallel calls stay separate via item_to_call; post-stream start/end (openai-messages/.../handler.py:695-699) dedupes against streamed ones; model/tool/max-steps/outer errors all call fail_open; graph() and config().stream() still get chunk via StableProjector.
Verdict: fix 1 and 2 first. ~−150 lines possible (3), ~−30 more with 4. Not checked: a live OpenAI stream; direct external callers of handler.stream(), who now get TEXT_MESSAGE_* instead of chunk.
The agents hook can finish a tool before the consumer has read the rest of its argument deltas. The result now closes with the full argument string. A new response closes the open text message from the stream itself, not from the background hook. Co-authored-by: alexis <alexis@launchdarkly.com>
|
bugbot run |
Co-authored-by: alexis <alexis@launchdarkly.com>
|
bugbot run |
Message ids are unique across runs, tool results always carry content, and a run cannot finish while a tool call is open. Argument text is completed only when it extends what was streamed. Disconnects close the handler, and callers can echo threadId and runId. Co-authored-by: alexis <alexis@launchdarkly.com>
Co-authored-by: alexis <alexis@launchdarkly.com>
Hosted tool events cannot arrive while only function tools are offered, so that mapping is gone. Both handlers report the string the model was sent, and argument deltas stay on the call they belong to. Co-authored-by: alexis <alexis@launchdarkly.com>
|
Addressed the review. The slow-consumer fixes are in Must fix:
Should fix: |
|
bugbot run |
A tool result was emitted while the assistant message was still open, so an AG-UI client could attach the result to the wrong turn. The agents hook now ends that message before the tool result. A provider error in the messages handler ended the span and then raised without TEXT_MESSAGE_END; the failure path now closes the message after the span is failed. Co-authored-by: alexis <alexis@launchdarkly.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 3248d3c. Configure here.
The agents tool hook runs before the consumer has read every text delta from that model call. Closing the message in the hook split the rest of the turn into a second message and put the tool result ahead of it. The stream now keeps that text on the open message, then closes it before the tool result. Co-authored-by: alexis <alexis@launchdarkly.com>
|
bugbot run |
1 similar comment
|
bugbot run |
Handlers can yield the AG-UI text and tool subset. config().stream() still yields chunk then done. stream() is the opt-in entry. Co-authored-by: alexis <alexis@launchdarkly.com>
A handler that raises mid-call can still report the lifecycle without the tool result. Co-authored-by: alexis <alexis@launchdarkly.com>
Message ids are unique across runs, tool results always carry content, and a run cannot finish while a tool call is open. Argument text is completed only when it extends what was streamed. Disconnects close the handler, and callers can echo threadId and runId. Co-authored-by: alexis <alexis@launchdarkly.com>
contextlib.aclosing always awaits aclose. A handler may return a plain async iterator, and that call raised AttributeError after the loop had already succeeded. The tracker now closes the stream only when aclose is present. Co-authored-by: alexis <alexis@launchdarkly.com>
ae82b2c to
37c7387
Compare
The reader now sits on main, including configure(). Keep the OpenAI AG-UI stream and that capture default so this branch merges into the reader. Co-authored-by: alexis <alexis@launchdarkly.com>
The reader and Claude streams are on main. Keep the OpenAI argument-prefix test that main's copy of the reader tests does not have. Co-authored-by: alexis <alexis@launchdarkly.com>
3 participants
OpenAI messages and agents map Responses stream events and the handler tool loop onto the shared AG-UI subset. Argument text and results follow
capture_content.config().stream()stayschunkthendone.Verified with the OpenAI handler tests and
mypy.