Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .agents/skills/instrumentation/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,13 @@ Map the change before editing:
- Automatic capture of inline multimodal bytes as Braintrust attachments must be opt-in.
Guard attachment creation and binary observation with `BRAINTRUST_CAPTURE_ATTACHMENTS`, and enable them only when its value is `1` or `true`.
Preserve useful text, metadata, metrics, and remote references when attachment capture is disabled, and omit inline bytes from logged payloads.
- Segmentable streaming media must use the shared `configureInstrumentation({ streamingAttachmentBufferBytes })` setting instead of provider-specific chunk-size or time-based controls.
It defaults to 4 MiB per logical media stream and counts binary attachment payload bytes after decoding and format conversion, including any required padding or silence.
Flush near the threshold at valid media boundaries and flush remaining valid content when the recording ends.
Preserve ordering, format metadata, and timestamps needed to interpret the attachments.
Keep whole-file responses intact, even if they exceed the threshold; never arbitrarily split encoded images, audio, or video.
This is a flush threshold, not a total memory limit or attachment-capture opt-in.
Preserve application stream behavior and existing capture consent; keep any capture safety ceiling internal and distinct from the threshold.
- Do not modify package READMEs during instrumentation work unless the user specifically requests it or the change corrects outdated information.
- We want to limit our instrumentation to operations that are relevant for AI generations and operations (LLMs, embeddings, media generation, ...). Things like creating entities on platforms (CRUD for Workflows of Agent entities) is irrelevant to us.
- When building instrumentation, we should always have a vendored type/interface for what we are wrapping. The type or interface should not be larger than what is relevant to the instrumentation. The type or interface should be used for typing invocation hooks and also should be used to assert the type on whatever is passed into wrappers as soon as the wrapper has verified that the passed in value is plausibly what should be wrapped.
Expand Down
5 changes: 5 additions & 0 deletions .changeset/solid-donuts-drop.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"braintrust": minor
---

feat: Add instrumentation for LiveKit cascade mode
16 changes: 13 additions & 3 deletions dev-packages/seinfeld/src/recorder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -807,15 +807,25 @@ async function writeFetchResponse(

const reader = response.body.getReader();
try {
for (;;) {
while (!res.destroyed) {
const { done, value } = await reader.read();
if (done) break;
if (!res.write(Buffer.from(value))) {
await new Promise<void>((resolve) => res.once("drain", resolve));
// A client that disconnects mid-stream never drains.
await new Promise<void>((resolve) => {
const resume = () => {
res.off("drain", resume);
res.off("close", resume);
resolve();
};
res.on("drain", resume);
res.on("close", resume);
});
}
}
} finally {
res.end();
reader.releaseLock();
if (res.destroyed) await reader.cancel();
else reader.releaseLock();
}
}
15 changes: 15 additions & 0 deletions e2e/config/pr-comment-scenarios.json
Original file line number Diff line number Diff line change
Expand Up @@ -983,5 +983,20 @@
"label": "v1 latest"
}
]
},
{
"scenarioDirName": "livekit-instrumentation",
"label": "LiveKit cascade instrumentation and PCM recording (streaming STT, tools, interruption recovery; automatic and manual)",
"metadataScenario": "livekit-instrumentation",
"variants": [
{
"variantKey": "livekit-v1",
"label": "v1.9 pinned"
},
{
"variantKey": "livekit-v1-latest",
"label": "v1.9 latest"
}
]
}
]
Binary file not shown.
Loading
Loading