Expand cross-SDK scenario and RPC E2E coverage - #2724
Merged
Merged
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The Rust atomic-replacement scenario is not atomic, and one .NET ordering test relies on a fixed timing delay.
Get a fresh assessment by requesting another Copilot review.
Review effort: Balanced
Findings: 2
Open (2)
What changed in this PR
Expands deterministic cross-SDK scenario and generated RPC coverage while fixing transport, serialization, and extension-context defects.
Changes:
- Adds cross-language workflow scenarios and replay snapshots.
- Audits generated RPC surfaces across all SDKs.
- Fixes Python/Go/.NET transport and decoding behavior plus Rust attachment support.
| File | Description |
|---|---|
test/snapshots/scenario_testing_utility/should_send_wait_observe_idle_events_and_delete_suggestion_session.yaml |
Adds utility workflow replay. |
test/snapshots/scenario_testing_tools/should_preserve_scenario_tool_invocation_identity_arguments_and_text.yaml |
Adds tool identity replay. |
test/snapshots/scenario_testing_tools/should_isolate_scenario_tool_handler_error.yaml |
Adds tool-error replay. |
test/snapshots/scenario_testing_tools/should_deliver_expanded_scenario_tool_result_to_the_model.yaml |
Adds expanded-result replay. |
test/snapshots/scenario_testing_tools/should_cancel_scenario_tool_handler_when_session_disposes.yaml |
Adds cancellation replay. |
test/snapshots/scenario_testing_tools/should_advertise_scenario_tool_schema_override_and_availability.yaml |
Adds tool-schema replay. |
test/snapshots/scenario_testing_skills_and_agents/should_reload_atomically_replaced_skill_and_replay_it_on_resume.yaml |
Adds skill-reload fixture. |
test/snapshots/scenario_testing_skills_and_agents/should_classify_agent_method_not_found_as_remote_protocol_error.yaml |
Adds protocol-error fixture. |
test/snapshots/scenario_testing_sends/should_order_idle_queued_and_immediate_scenario_delivery.yaml |
Adds delivery-order replay. |
test/snapshots/scenario_testing_runtime/should_ping_then_reuse_client_across_two_sessions.yaml |
Adds runtime-reuse replay. |
test/snapshots/scenario_testing_providers/should_apply_reasoning_context_and_auto_atomically_without_implicit_reset.yaml |
Adds provider-state fixture. |
test/snapshots/scenario_testing_persistence/should_truncate_history_and_resend_from_boundary.yaml |
Adds history-truncation replay. |
test/snapshots/scenario_testing_persistence/should_retry_from_existing_history_with_empty_sendmessages.yaml |
Adds empty-batch replay. |
test/snapshots/scenario_testing_persistence/should_page_persisted_events_backward_without_resuming.yaml |
Adds event-pagination replay. |
test/snapshots/scenario_testing_permissions/should_forward_exact_scenario_permission_callback_payload.yaml |
Adds permission replay. |
test/snapshots/scenario_testing_mcp/should_preserve_disabled_scenario_mcp_servers_across_reload_and_resume.yaml |
Adds MCP-state replay. |
test/snapshots/scenario_testing_lifecycle_recovery/should_suspend_disconnect_and_resume_scenario_state_without_delete.yaml |
Adds lifecycle-resume replay. |
test/snapshots/scenario_testing_lifecycle_recovery/should_abort_active_scenario_turn_and_remain_usable.yaml |
Updates abort-recovery replay. |
test/snapshots/scenario_testing_js_extension_bridge/should_surface_structured_canvaserror_from_js_extension.yaml |
Adds canvas-error fixture. |
test/snapshots/scenario_testing_js_extension_bridge/should_bridge_js_extension_canvas_context_log_and_session_continuation.yaml |
Adds extension-bridge replay. |
test/snapshots/scenario_testing_event_subscriptions/should_stop_closed_and_replaced_scenario_event_sources.yaml |
Adds event-source replay. |
test/snapshots/scenario_testing_event_subscriptions/should_deliver_mixed_scenario_event_stream_in_order_after_handler_lag.yaml |
Adds ordered-event replay. |
test/snapshots/scenario_testing_empty_runtime/empty_mode_minimal_toolless_session_has_no_tools.yaml |
Adds empty-runtime replay. |
test/snapshots/scenario_testing_control_state/should_report_processing_while_scenario_tool_is_running.yaml |
Adds processing-state replay. |
test/snapshots/scenario_testing_composition/should_send_scenario_message_with_metadata_and_extension_context.yaml |
Adds extension-context replay. |
test/snapshots/scenario_testing_composition/should_retry_resume_on_replacement_client_after_recoverable_setup_failure.yaml |
Adds replacement-client replay. |
test/snapshots/scenario_testing_composition/should_resume_with_reattached_scenario_host_state.yaml |
Adds host-state replay. |
test/snapshots/scenario_testing_composition/should_read_persisted_scenario_events_without_resuming.yaml |
Adds persisted-event replay. |
test/snapshots/scenario_testing_composition/should_not_emit_redundant_model_change_when_resuming_same_model.yaml |
Adds model-resume replay. |
test/snapshots/scenario_testing_composition/should_classify_queued_and_immediate_scenario_messages_while_busy.yaml |
Adds busy-delivery replay. |
test/snapshots/scenario_testing_cloud/should_notify_steerability_then_send_first_message_without_remote_enable.yaml |
Adds cloud-steering replay. |
test/snapshots/scenario_testing_canvas/should_run_ordered_scenario_canvas_lifecycle_with_exact_context_and_snapshot.yaml |
Adds canvas-lifecycle fixture. |
test/snapshots/scenario_testing_canvas/should_reattach_scenario_canvas_and_route_all_callbacks_after_resume.yaml |
Adds canvas-resume replay. |
test/snapshots/scenario_testing_canvas/should_handle_structured_scenario_canvas_error.yaml |
Adds canvas-error fixture. |
test/snapshots/scenario_testing_callbacks/should_run_scenario_prompt_and_tool_hooks_with_full_context_and_suppression.yaml |
Adds hook replay. |
test/snapshots/scenario_testing_callbacks/should_cancel_scenario_host_callback_when_channel_disconnects.yaml |
Adds callback-cancellation replay. |
test/snapshots/scenario_testing_callbacks/should_auto_switch_scenario_mode_after_rate_limit.yaml |
Adds rate-limit replay. |
test/snapshots/scenario_testing_callbacks/should_approve_scenario_exit_plan_with_full_callback_and_event_state.yaml |
Adds plan-approval replay. |
test/snapshots/mode_handlers/should_invoke_exit_plan_mode_handler_when_model_uses_tool.yaml |
Updates plan-handler response. |
test/snapshots/mode_handlers/should_invoke_auto_mode_switch_handler_when_rate_limited.yaml |
Updates auto-mode response. |
test/harness/test-mcp-app-server.mjs |
Adds local MCP app server. |
test/harness/replayingCapiProxy.ts |
Adds replay-only proxy mode. |
test/harness/replayingCapiProxy.test.ts |
Tests replay-only behavior. |
scripts/codegen/python.ts |
Fixes external union loading. |
rust/tests/e2e/skills.rs |
Adds skill reload/resume coverage. |
rust/tests/e2e/event_fidelity.rs |
Adds delivery-order coverage. |
rust/tests/e2e.rs |
Registers RPC surface tests. |
rust/src/types.rs |
Adds extension-context attachments. |
python/e2e/test_scenario_session_setup_e2e.py |
Tests session setup ordering. |
python/e2e/test_scenario_sends_e2e.py |
Tests send serialization and cancellation. |
python/e2e/test_scenario_lifecycle_recovery_e2e.py |
Tests recoverable lifecycle failures. |
python/e2e/test_scenario_cloud_e2e.py |
Tests cloud workflows. |
python/e2e/test_scenario_canvas_e2e.py |
Tests canvas callback routing. |
python/copilot/session.py |
Exposes extension-context attachments. |
python/copilot/generated/rpc.py |
Uses generated union loaders. |
python/copilot/_jsonrpc.py |
Normalizes process-exit write errors. |
nodejs/test/session-event-codegen.test.ts |
Tests union-loader codegen. |
nodejs/test/e2e/scenario_testing_recovery.e2e.test.ts |
Adds lifecycle recovery scenarios. |
nodejs/test/e2e/scenario_testing_persistence.e2e.test.ts |
Adds persistence scenario. |
nodejs/test/e2e/scenario_testing_composition.e2e.test.ts |
Adds resume composition scenario. |
nodejs/test/e2e/rpc_workspace_checkpoints.e2e.test.ts |
Expands workspace diff modes. |
nodejs/test/e2e/permissions.e2e.test.ts |
Verifies full permission payload. |
nodejs/test/e2e/client_options.e2e.test.ts |
Covers resumed provider options. |
nodejs/test/e2e/canvas.e2e.test.ts |
Covers structured canvas errors. |
nodejs/test/e2e/abort.e2e.test.ts |
Expands abort-state assertions. |
java/sdk/src/test/java/com/github/copilot/ScenarioTestCli.java |
Adds in-process scenario runtime. |
java/sdk/src/test/java/com/github/copilot/ScenarioCoverageE2ETest.java |
Adds Java workflow coverage. |
java/sdk/src/test/java/com/github/copilot/RpcSurfaceTestCli.java |
Adds RPC recording runtime. |
go/rpc/sessions_client_metadata_json.go |
Decodes metadata unions. |
go/rpc/sessions_client_metadata_json_test.go |
Tests metadata decoding. |
go/internal/jsonrpc2/jsonrpc2.go |
Completes requests on disconnect. |
go/internal/jsonrpc2/jsonrpc2_test.go |
Tests disconnect completion. |
go/internal/e2e/scenario_testing_control_state_e2e_test.go |
Tests Go processing state. |
dotnet/test/Unit/JsonRpcTests.cs |
Tests disposal cleanup. |
dotnet/test/Unit/CloneTests.cs |
Covers launch-provider cloning. |
dotnet/test/Harness/ReplayProxy.cs |
Passes replay-only configuration. |
dotnet/test/Harness/E2ETestContext.cs |
Exposes replay-only setup. |
dotnet/test/Harness/E2ETestBase.cs |
Configures replay-only suites. |
dotnet/test/E2E/ScenarioTestingUtilityE2ETests.cs |
Adds utility scenario. |
dotnet/test/E2E/ScenarioTestingToolsE2ETests.cs |
Adds tool scenarios. |
dotnet/test/E2E/ScenarioTestingSkillsAndAgentsE2ETests.cs |
Adds skills and agent scenarios. |
dotnet/test/E2E/ScenarioTestingServerControlE2ETests.cs |
Adds server-control scenarios. |
dotnet/test/E2E/ScenarioTestingPersistenceE2ETests.cs |
Adds persistence scenarios. |
dotnet/test/E2E/ScenarioTestingPermissionsE2ETests.cs |
Adds permission scenarios. |
dotnet/test/E2E/ScenarioTestingLifecycleRecoveryE2ETests.cs |
Adds lifecycle recovery scenarios. |
dotnet/test/E2E/ScenarioTestingEventSubscriptionsE2ETests.cs |
Adds event subscription scenarios. |
dotnet/test/E2E/ScenarioTestingEmptyRuntimeE2ETests.cs |
Adds empty-runtime scenario. |
dotnet/test/E2E/ScenarioTestingE2ETestBase.cs |
Defines replay-only scenario base. |
dotnet/test/E2E/ScenarioTestingControlStateE2ETests.cs |
Adds control-state scenarios. |
dotnet/test/E2E/ScenarioTestingCloudE2ETests.cs |
Adds cloud scenarios. |
dotnet/test/E2E/ModeHandlersE2ETests.cs |
Removes redundant mode-handler tests. |
dotnet/test/E2E/ExternalToolCancellationE2ETests.cs |
Removes redundant cancellation test. |
dotnet/src/Types.cs |
Adds launch-provider option. |
dotnet/src/JsonRpc.cs |
Guarantees disposal cleanup. |
dotnet/src/Client.cs |
Registers providers and process-exit handling. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This comment has been minimized.
This comment has been minimized.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Generated by SDK Consistency Review Agent for #2724 · copilot · sonnet50 · 110.2 AIC · ⌖ 11.7 AIC · ⊞ 7.8K
This comment has been minimized.
This comment has been minimized.
Contributor
There was a problem hiding this comment.
Generated by SDK Consistency Review Agent for #2724 · copilot · sonnet50 · 116.7 AIC · ⌖ 11.8 AIC · ⊞ 7.8K
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Contributor
There was a problem hiding this comment.
Generated by SDK Consistency Review Agent for #2724 · copilot · sonnet50 · 172.6 AIC · ⌖ 13.2 AIC · ⊞ 7.8K
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Contributor
There was a problem hiding this comment.
Generated by SDK Consistency Review Agent for #2724 · copilot · sonnet50 · 61.9 AIC · ⌖ 11.8 AIC · ⊞ 7.8K
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Contributor
There was a problem hiding this comment.
Generated by SDK Consistency Review Agent for #2724 · copilot · sonnet50 · 77.1 AIC · ⌖ 11.6 AIC · ⊞ 7.8K
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
stephentoub
force-pushed
the
stephentoub-e2e-github-app-coverage
branch
from
September 20, 2026 02:16
7522e91 to
c2c4644
Compare
Contributor
SDK Consistency Review — PR #2724Reviewed the authoritative PR delta ( Findings: ✅ Consistent across all SDKsThis PR introduces two new experimental features and rolls them out uniformly:
No cross-SDK gaps or naming inconsistencies were found. No inline review comments needed.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The SDKs had broad feature tests, but they did not systematically cover representative production workflows or guarantee that every public generated RPC method was referenced by a test. This made it easy for important composition, lifecycle, callback, persistence, and generated-contract regressions to go unnoticed.
What changed
ScenarioTestingbaseline in C# covering realistic startup, session, send, event, recovery, permission, callback, provider, MCP, canvas, persistence, queue, factory, and remote-control workflows.The expanded coverage found and fixes several SDK defects, including C# extension launch-provider registration and transport-failure completion, Python generated union dispatch and transport error normalization, Go metadata-union decoding and pending TCP request completion, and Rust extension-context optionality.
Validation
net8.0andnet472; RPC surface tests passed on both frameworks; final integrated regression passed 98/98 onnet8.0.gofmt,go vet, build, unit tests, focused E2Es, and repeated reliability runs passed. A broad run encountered existing replay misses and timeout behavior outside the changed coverage.mvn verifypassed; native Node validation passed 93/93.Review notes
This is intentionally a large test-focused change because the scenario catalog and generated RPC audits are cross-language consistency guarantees. Most tests share the existing replay infrastructure; fake-runtime tests are reserved for host-owned or contract-only surfaces that cannot be exercised deterministically through the replayed runtime.
Generated by Copilot