Skip to content

feat(evaluations)!: keep the management API client internal - #202

Merged
aknight-ld merged 1 commit into
mainfrom
AIC-3642-match-spec-exports
Oct 23, 2026
Merged

feat(evaluations)!: keep the management API client internal#202
aknight-ld merged 1 commit into
mainfrom
AIC-3642-match-spec-exports

Conversation

@aknight-ld

@aknight-ld aknight-ld commented Oct 22, 2026 •
edited by cursor Bot
Loading

Copy link
Copy Markdown
Contributor

Another follow up to match the spec. This PR cleans up exports for the experimental evaluations module to more narrowly match what is expected + currently shown in JS.

Note

Overview
Breaking change: narrows the experimental evaluations public surface so the LaunchDarkly management API client stays internal, aligned with the JS SDK.

The package no longer re-exports LDApiClient, HttpResponse, Transport, urllib_transport, ToolsClient, or SuccessDirection from launchdarkly_ai_server.evaluations. EvaluationsModule drops the public api property; tools and other documented entry points are unchanged. Tests that still need the client import from evaluations.api and assert via evals._api.

Reviewed by Cursor Bugbot for commit 19223db. Bugbot is set up for automated code reviews on this repo. Configure here.

@aknight-ld
aknight-ld merged commit 17270ed into main Oct 23, 2026
8 checks passed
@aknight-ld
aknight-ld deleted the AIC-3642-match-spec-exports branch October 23, 2026 13:11
@github-actions github-actions Bot mentioned this pull request Oct 23, 2026
Merged
@aknight-ld aknight-ld mentioned this pull request Oct 23, 2026
Closed
@jeffdupont jeffdupont mentioned this pull request Oct 23, 2026
Merged
jeffdupont added a commit that referenced this pull request Oct 23, 2026
@jeffdupont @claude
chore(AIC-3500): regenerate the evaluations API report after #202 (#205)
99b57dc 
Main's **Tests** job has failed since #202 merged (run 39959179499):
`tests/test_api_report.py::test_committed_reports_are_current` reports
that `launchdarkly_ai_server.experimental.evaluations.txt` is out of
date.

#202 removed `EvaluationsModule.api`. Its last commit was pushed before
the API report check (#195, #196) merged, so its own CI never ran the
check. This PR is the output of `make api-report`: one line removed
(`api: LDApiClient [property]`).

Checked locally: `make api-report-check` reports "15 API reports are
current", and `tests/test_api_report.py` passes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Overview**
> Regenerates the committed **evaluations** API report so it matches the
current experimental public surface after `EvaluationsModule.api` was
removed in #202.
> 
> The only report change is dropping the `api: LDApiClient` property
line from `EvaluationsModule`; this unblocks
`test_committed_reports_are_current` / `make api-report-check`.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
728bc50. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
jeffdupont pushed a commit that referenced this pull request Oct 23, 2026
@github-actions
chore: release main (#166)
662d3f2 
🤖 I have created a release *beep* *boop*
---


<details><summary>launchdarkly-ai-server: 0.4.0</summary>

##
[0.4.0](launchdarkly-ai-server-0.3.0...launchdarkly-ai-server-0.4.0)
(2026-10-23)


### ⚠ BREAKING CHANGES

* **evaluations:** keep the management API client internal
([#202](#202))
* **evaluations:** rename base uri to match spec
([#199](#199))
* **evaluations:** add test proving harness edge case matches spec
([#200](#200))
* **evaluations:** route handlers by provider and mode
* **evaluations:** mark evaluations from code as experimental
([#157](#157))

### Experimental

* **evaluations:** add test proving harness edge case matches spec
([#200](#200))
([fa7162e](fa7162e))
* **evaluations:** keep the management API client internal
([#202](#202))
([17270ed](17270ed))
* **evaluations:** mark evaluations from code as experimental
([#157](#157))
([1389765](1389765))
* **evaluations:** rename base uri to match spec
([#199](#199))
([7d93f9c](7d93f9c))
* **evaluations:** route handlers by provider and mode
([f62591f](f62591f))
* **evaluations:** Set JSON as standard for output to pass to judge
([#203](#203))
([ffc728b](ffc728b))
* **skills:** Correct the HTTP 403 advice now that server-side FDv2 is
available to every account
([#178](#178))
([b17e322](b17e322))


### Features

* **agui:** add a shared reader for handler streams
([#164](#164))
([fc08dd5](fc08dd5))
* **AIC-3534:** add a TypeSafe Jev judge handler
([#149](#149))
([1b41ea4](1b41ea4))
* **client:** add configure for SDK defaults
([#158](#158))
([3736767](3736767))
* **langchain:** emit AG-UI text and tool events
([#163](#163))
([ab5e9e1](ab5e9e1))
* **openai:** emit AG-UI text and tool events
([#161](#161))
([2b1b2d4](2b1b2d4))


### Bug Fixes

* **AIC-3570:** re-resolve graph flags on every invoke and stream
([#187](#187))
([8fdc4d4](8fdc4d4))
* **AIC-3571:** visit every child before its parent in reverse_traverse
([#189](#189))
([87c73d9](87c73d9))
* **AIC-3572:** treat a variation without _ldMeta.enabled as disabled
([#184](#184))
([7a0d260](7a0d260))
* **AIC-3574:** build the LaunchDarkly client off the event loop
([#185](#185))
([bf03b6c](bf03b6c))
* **AIC-3576:** record deferred judge scores under the evaluated config
([#190](#190))
([354d3f1](354d3f1))
* **AIC-3577:** send object tool results to the model as JSON
([#188](#188))
([dead4bd](dead4bd))
* **AIC-3596:** name the [otel] extra in the missing-OTel warning
([#186](#186))
([90ce7bf](90ce7bf))
* **client:** match a handler's provides_for after normalizing its mode
([f62591f](f62591f))
* give native graph spans their LaunchDarkly identity
([#121](#121))
([e863b6d](e863b6d))
</details>

<details><summary>launchdarkly-ai-python: 0.3.0</summary>

##
[0.3.0](launchdarkly-ai-python-0.2.0...launchdarkly-ai-python-0.3.0)
(2026-10-23)


### ⚠ BREAKING CHANGES

* **evaluations:** mark evaluations from code as experimental
([#157](#157))

### Experimental

* **evaluations:** mark evaluations from code as experimental
([#157](#157))
([1389765](1389765))
</details>

<details><summary>launchdarkly-ai-claude-agents: 0.2.6</summary>

##
[0.2.6](launchdarkly-ai-claude-agents-0.2.5...launchdarkly-ai-claude-agents-0.2.6)
(2026-10-23)


### Features

* **claude:** emit AG-UI text and tool events
([#165](#165))
([3f3a0ce](3f3a0ce))
* **client:** add configure for SDK defaults
([#158](#158))
([3736767](3736767))


### Bug Fixes

* **AIC-3571:** visit every child before its parent in reverse_traverse
([#189](#189))
([87c73d9](87c73d9))
* **AIC-3577:** send object tool results to the model as JSON
([#188](#188))
([dead4bd](dead4bd))
* **claude-agents:** fail the run when the CLI reports an API error as
success
([#174](#174))
([3d1a768](3d1a768))
* give native graph spans their LaunchDarkly identity
([#121](#121))
([e863b6d](e863b6d))
</details>

<details><summary>launchdarkly-ai-claude-messages: 0.2.5</summary>

##
[0.2.5](launchdarkly-ai-claude-messages-0.2.4...launchdarkly-ai-claude-messages-0.2.5)
(2026-10-23)


### Features

* **claude:** emit AG-UI text and tool events
([#165](#165))
([3f3a0ce](3f3a0ce))
* **client:** add configure for SDK defaults
([#158](#158))
([3736767](3736767))


### Bug Fixes

* **AIC-3577:** send object tool results to the model as JSON
([#188](#188))
([dead4bd](dead4bd))
</details>

<details><summary>launchdarkly-ai-openai-agents: 0.2.6</summary>

##
[0.2.6](launchdarkly-ai-openai-agents-0.2.5...launchdarkly-ai-openai-agents-0.2.6)
(2026-10-23)


### Features

* **client:** add configure for SDK defaults
([#158](#158))
([3736767](3736767))
* **openai:** emit AG-UI text and tool events
([#161](#161))
([2b1b2d4](2b1b2d4))


### Bug Fixes

* **AIC-3571:** visit every child before its parent in reverse_traverse
([#189](#189))
([87c73d9](87c73d9))
* **AIC-3577:** send object tool results to the model as JSON
([#188](#188))
([dead4bd](dead4bd))
* give native graph spans their LaunchDarkly identity
([#121](#121))
([e863b6d](e863b6d))
</details>

<details><summary>launchdarkly-ai-vercel-agents: 0.1.1</summary>

##
[0.1.1](launchdarkly-ai-vercel-agents-0.1.0...launchdarkly-ai-vercel-agents-0.1.1)
(2026-10-23)


### Features

* **AIC-3408:** add Vercel AI SDK messages and agents adapters
([#103](#103))
([64d679e](64d679e))
</details>

<details><summary>launchdarkly-ai-litellm-agents: 0.1.1</summary>

##
[0.1.1](launchdarkly-ai-litellm-agents-0.1.0...launchdarkly-ai-litellm-agents-0.1.1)
(2026-10-23)


### Features

* add LiteLLM messages and agents integrations
([#102](#102))
([f2e9886](f2e9886))
</details>

<details><summary>launchdarkly-ai-openai-messages: 0.2.5</summary>

##
[0.2.5](launchdarkly-ai-openai-messages-0.2.4...launchdarkly-ai-openai-messages-0.2.5)
(2026-10-23)


### Features

* **client:** add configure for SDK defaults
([#158](#158))
([3736767](3736767))
* **openai:** emit AG-UI text and tool events
([#161](#161))
([2b1b2d4](2b1b2d4))


### Bug Fixes

* **AIC-3577:** send object tool results to the model as JSON
([#188](#188))
([dead4bd](dead4bd))
</details>

<details><summary>launchdarkly-ai-vercel-messages: 0.1.1</summary>

##
[0.1.1](launchdarkly-ai-vercel-messages-0.1.0...launchdarkly-ai-vercel-messages-0.1.1)
(2026-10-23)


### Features

* **AIC-3408:** add Vercel AI SDK messages and agents adapters
([#103](#103))
([64d679e](64d679e))
</details>

<details><summary>launchdarkly-ai-litellm-messages: 0.1.1</summary>

##
[0.1.1](launchdarkly-ai-litellm-messages-0.1.0...launchdarkly-ai-litellm-messages-0.1.1)
(2026-10-23)


### Features

* add LiteLLM messages and agents integrations
([#102](#102))
([f2e9886](f2e9886))
</details>

<details><summary>launchdarkly-ai-langchain-agents: 0.2.6</summary>

##
[0.2.6](launchdarkly-ai-langchain-agents-0.2.5...launchdarkly-ai-langchain-agents-0.2.6)
(2026-10-23)


### Features

* **client:** add configure for SDK defaults
([#158](#158))
([3736767](3736767))
* **langchain:** emit AG-UI text and tool events
([#163](#163))
([ab5e9e1](ab5e9e1))


### Bug Fixes

* **AIC-3577:** send object tool results to the model as JSON
([#188](#188))
([dead4bd](dead4bd))
* give native graph spans their LaunchDarkly identity
([#121](#121))
([e863b6d](e863b6d))
</details>

<details><summary>launchdarkly-ai-langchain-messages: 0.2.5</summary>

##
[0.2.5](launchdarkly-ai-langchain-messages-0.2.4...launchdarkly-ai-langchain-messages-0.2.5)
(2026-10-23)


### Features

* **client:** add configure for SDK defaults
([#158](#158))
([3736767](3736767))
* **langchain:** emit AG-UI text and tool events
([#163](#163))
([ab5e9e1](ab5e9e1))


### Bug Fixes

* **AIC-3577:** send object tool results to the model as JSON
([#188](#188))
([dead4bd](dead4bd))
</details>

<details><summary>launchdarkly-ai-opencode-agents: 0.2.0</summary>

##
[0.2.0](launchdarkly-ai-opencode-agents-0.1.0...launchdarkly-ai-opencode-agents-0.2.0)
(2026-10-23)


### ⚠ BREAKING CHANGES

* **opencode:** start and stop opencode serve in the handler
([#198](#198))

### Features

* **opencode-agents:** add an OpenCode handler
([#159](#159))
([fc08e8c](fc08e8c))
* **opencode-agents:** emit the AG-UI text and tool subset
([#162](#162))
([7677621](7677621))
* **opencode-agents:** open one OpenCode session per call
([#179](#179))
([b225113](b225113))
* **opencode-agents:** render agent instruction templates
([#182](#182))
([83175aa](83175aa))
* **opencode:** start and stop opencode serve in the handler
([#198](#198))
([0748dcf](0748dcf))


### Bug Fixes

* **opencode-agents:** own the HTTP client
([#194](#194))
([ed2d419](ed2d419))
* **opencode-agents:** skip judges and record structured history
([#177](#177))
([6fadf1d](6fadf1d))
</details>

<details><summary>launchdarkly-ai-typesafe: 0.1.1</summary>

##
[0.1.1](launchdarkly-ai-typesafe-0.1.0...launchdarkly-ai-typesafe-0.1.1)
(2026-10-23)


### Features

* **AIC-3534:** add a TypeSafe Jev judge handler
([#149](#149))
([1b41ea4](1b41ea4))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jeffdupont jeffdupont jeffdupont approved these changes
@donei003 donei003 donei003 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@aknight-ld @jeffdupont @donei003