Skip to content

fix(client): concurrent first init_client calls share one attempt - #255

Open
XieX wants to merge 8 commits into
mainfrom
xie/aic-3600-shared-init-attempt
Open

XieX wants to merge 8 commits into
mainfrom
xie/aic-3600-shared-init-attempt

Conversation

@XieX

@XieX XieX commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #254. Retarget to main once #254 merges.

Problem

Since #254, init_client raises when the client it built has not initialized within start_wait=10, and caches nothing so a later call retries. While LaunchDarkly is unreachable, every lazy call (config().invoke(), graph(), inspect_config) builds its own client in asyncio.to_thread and blocks for 10 s. N concurrent callers each hold a default-executor thread for that long, which can starve other to_thread users.

Change

Concurrent first calls now share one in-flight attempt, as the TypeScript SDK's initClient does.

  • The attempt builds the client on a daemon thread it starts itself, not on a default-executor thread. Its result is a concurrent.futures.Future, so callers on any event loop or thread can join it. The skills watcher, for example, runs asyncio.run on its own thread.
  • The attempt thread adopts the client itself, under _state_lock. Each caller waits on a future of its own loop, which is woken through call_soon_threadsafe, so cancelling one caller leaves the attempt running for the rest. (asyncio.wrap_future would cancel the shared future.)
  • Every caller of a failed attempt gets the same error. The attempt is cleared before any of them sees it, so a later call retries. A failure is still not cached.
  • The attempt closes its client instead of adopting it when shutdown() ran, when a client was passed in, or when every caller waiting on it was cancelled. A lone cancelled call still leaves nothing behind. Before this change, an attempt still in flight when shutdown() ran adopted its client afterwards.
  • Callers waiting on an attempt that a passed-in client replaced get the passed client, as before.
  • The README no longer says concurrent calls each build their own client.

Tests

New tests in packages/client/tests/test_lifecycle.py use a client constructor that blocks until the test releases it:

  • 5 concurrent calls construct one client.
  • 5 concurrent calls during a failing init construct one client, all raise, and the client is closed once; a later call constructs a new one.
  • Cancelling one of two waiters doesn't cancel the attempt for the other.
  • A call from asyncio.run on another thread joins the attempt.
  • shutdown() abandons an attempt in flight.
  • A client passed in mid-attempt replaces the attempt.

The old two-calls-build-two-clients test was replaced, since it tested the old behavior.

make lint format-check typecheck api-report-check test passes: 3805 passed, 18 skipped.

🤖 Generated with Claude Code


Note

Overview
Concurrent lazy init_client calls now share a single in-flight connection attempt instead of each starting its own 10-second LD client build on the default executor (which could starve other asyncio.to_thread work during outages).

The lifecycle module introduces _InitAttempt guarded by _state_lock: one daemon thread builds the client and settles a shared concurrent.futures.Future, while each async caller waits on a loop-local future wired via call_soon_threadsafe so cancelling one waiter does not tear down the attempt for others. Failed attempts are still not cached; shutdown(), a BYOC client passed mid-flight, or no remaining waiters abandons the attempt (closes the built client, raises abandoned where appropriate, and uses a successor when BYOC replaced the attempt). README text is updated to match.

Lifecycle tests replace the old “two concurrent calls → two clients” expectation with coverage for shared success/failure, cross-loop join, shutdown/BYOC races, and cancellation edge cases.

Reviewed by Cursor Bugbot for commit cecb9b0. Bugbot is set up for automated code reviews on this repo. Configure here.

XieX and others added 3 commits October 30, 2026 13:00
…t initialize

The LaunchDarkly SDK's constructor returns normally when start_wait runs
out and when the service rejects the SDK key; it only logs. init_client
adopted that client anyway, so a bad key or an unreachable LaunchDarkly
served every AI Config disabled with no error at startup.

init_client now checks is_initialized() on the client it built. When it
is false the client is closed and RuntimeError is raised; nothing is
cached, so a later call retries. This matches the JavaScript and Go
SDKs, which already fail here. A client passed as client= is not
checked: the caller owns its readiness.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With no shared in-flight attempt, every concurrent lazy call during an
outage built its own LaunchDarkly client in asyncio.to_thread and held a
default-executor thread for the whole 10 s start wait, which could starve
other to_thread users.

Calls made while an attempt is in flight now wait for it. The attempt runs
on a thread of its own, and its result is a concurrent.futures.Future, so
callers on any event loop or thread can join it. The attempt thread adopts
the client itself under a lock. Each caller waits on a future of its own
loop, so cancelling one leaves the attempt running for the rest. Every
caller of a failed attempt gets the failure, and the attempt is dropped
before they see it, so a later call retries. The client is closed instead
of adopted when shutdown() or a passed client replaced the attempt, or
when every caller waiting on it was cancelled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@XieX
XieX marked this pull request as ready for review October 30, 2026 18:43
@XieX
XieX requested a review from jeffdupont October 30, 2026 18:43
@XieX XieX changed the title fix(AIC-3600): concurrent first init_client calls share one attempt fix(client): concurrent first init_client calls share one attempt Oct 30, 2026

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@jeffdupont jeffdupont left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The shared-attempt design is sound. I checked cancellation (per-loop futures, waiters only under the lock), retry after failure (attempt cleared before the future settles), and the all-cancelled path; they behave as described. Three timed-out wait_for calls during an outage build one client.

Requesting changes for the two spec mismatches in the inline comments (passed client when the built client is uninitialized, and shutdown followed by re-init). Both are small. Please add tests for those two orders and for "all waiters cancelled, a new caller joins before the attempt finishes".

Nit: start_wait=10 and the "10 seconds" in the message and docstring could be one constant.

Base automatically changed from xie/aic-3600-init-client-check to main October 30, 2026 19:51
XieX and others added 3 commits October 30, 2026 15:52
…nt's state

Review of the shared init attempt found two orders where a waiting call
got the wrong answer:

- A client passed while an attempt was in flight, during an outage: the
  attempt's own client was uninitialized, and that check ran first, so
  its callers raised "failed to initialize" although the passed client
  was now the singleton. The replaced check now comes first.
- shutdown(), then a new call's attempt completing before the old one:
  the old attempt returned _client, which was the new attempt's client.
  A passed client now records itself on the attempt it replaces, and
  only that is returned; an attempt shutdown() replaced always raises.

Adds tests for both orders and for a call made after every waiter was
cancelled joining the attempt, and puts the 10 s start wait in one
constant.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nit-attempt

# Conflicts:
#	packages/client/README.md
#	packages/client/src/launchdarkly_ai_server/lifecycle.py
…nit-attempt

# Conflicts:
#	packages/client/src/launchdarkly_ai_server/lifecycle.py

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 6452178. Configure here.

…tempt

A passed client clears _attempt when it replaces one in flight, so a
shutdown() that followed could no longer reach that attempt. When the
attempt finished it returned its successor, the passed client that
shutdown() had just closed. shutdown() now bumps a generation that each
attempt records when it starts, and an attempt from an earlier
generation fails as abandoned whatever replaced it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants