Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ opt-out.
- **Behavior change:** content captured for traces is scrubbed for secrets by default. `redactContent` is `boolean | ((text) => string)`: unset or `true` runs a built-in scrubber over message text, system instructions, tool-call arguments and tool results that replaces API-key shapes (`sk-…`, `lsv2_…`, GitHub tokens, AWS access key ids, the token after `Bearer`) with `[redacted]`; a function replaces the scrubber (text it throws on is recorded as `[redacted]`); `false` records content as is. It catches common key formats, not every secret, so set `captureContent: false` if content must never reach the db. See [Secrets in captured content](docs/observability.md#secrets-in-captured-content).
- **Behavior change:** sessions end. The daemon marks a session `ended` once none of its runs is open and nothing in it has started or finished for `lousho observe start --session-idle-minutes N` (default 30, `0` turns it off); `ended_at` is the last activity, and a new run makes it `active` again. Sessions in an existing db close on the first sweep that finds them idle, and ended sessions then age out with retention. See [Idle sessions](docs/observability.md#idle-sessions).
- **Behavior change:** the daemon keeps 30 days of runs by default. On its first sweep after start, and daily after that, it deletes traces whose observations all finished more than `--retention-days` ago, with their scores; a trace with a run still open, and a session still `active`, is kept. Start it with `--retention-days 0` to keep an existing db's history. See [Retention](docs/observability.md#retention).
- Studio redesign: a new look in light and dark (Settings > Appearance picks Light, Dark or System, plus Compact or Comfortable table density), a left nav that collapses to an icon rail, and a Settings screen in sections (among them Appearance, Provider keys with Set / From environment / Not set, Collector, Keyboard shortcuts). Traces gains **Refresh** and a **Live** toggle that refreshes every 5 seconds, the trace viewer switches between **Tree** and **Waterfall**, Evaluations moves between Scores, Datasets, Evaluators and Annotation queues with tabs, Ctrl/Cmd+Enter runs the Playground, and the Agents canvas opens with the whole graph fitted and the Inspector folded until you select a node. See [Agent Forge](docs/agent-forge.md).

### Fixed

Expand Down
50 changes: 37 additions & 13 deletions apps/agent-forge/e2e/docShots.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,9 @@ import { SEED_ERROR, SEED_NEWEST_FAILURE, seedObserve } from './fixtures/seedObs
* or relative to the process cwd)
* LOUSHO_DOC_SHOTS_SCHEME=light|dark the colour scheme to shoot (default light)
*
* Into docs/images (no OUT) the `x-` review shots are skipped, and a dark run
* writes only `studio-{agents,traces,trace,monitoring}-dark.png`.
*
* The small fixtures below fill the screens the seed leaves thin: a fleet with
* mixed health, a day of chart buckets for it, and a few threads.
*/
Expand Down Expand Up @@ -279,11 +282,32 @@ async function open(page: Page, hash: string) {
await expect(page.locator('.app')).toBeVisible();
}

/** The docs set shot in dark too, written as `studio-<screen>-dark.png`. */
const DARK_DOCS = new Set(['agents', 'traces', 'trace', 'monitoring']);

/**
* Where `file` goes, or null to skip it. With LOUSHO_DOC_SHOTS_OUT set every
* shot is written as named (reviewers rely on the `x-` ones). Into docs/images
* the `x-` review shots never go, and a dark run writes only the DARK_DOCS set.
*/
function shotPath(file: string): string | null {
if (process.env.LOUSHO_DOC_SHOTS_OUT) return path.join(OUT, file);
if (/^(studio-)?x-/.test(file)) return null;
if (shotScheme() === 'light') return path.join(OUT, file);
const screen = /^studio-(.+)\.png$/.exec(file)?.[1];
return screen && DARK_DOCS.has(screen) ? path.join(OUT, `studio-${screen}-dark.png`) : null;
}

async function snap(page: Page, file: string, options: { fullPage?: boolean } = {}) {
const target = shotPath(file);
if (target) await page.screenshot({ ...options, path: target });
}

async function shoot(page: Page, screen: string) {
// Let toasts, focus rings and chart transitions settle.
await page.mouse.move(0, 0);
await page.waitForTimeout(300);
await page.screenshot({ path: path.join(OUT, `studio-${screen}.png`) });
await snap(page, `studio-${screen}.png`);
}

/** Data rows only: a first load shows skeleton rows (`.skeleton-row`) until the rows arrive. */
Expand Down Expand Up @@ -373,7 +397,7 @@ for (const width of [860, 375]) {
await monitoringReady(page);
await page.mouse.move(0, 0);
await page.waitForTimeout(300);
await page.screenshot({ path: path.join(OUT, `studio-x-monitoring-${width}.png`) });
await snap(page, `studio-x-monitoring-${width}.png`);
});
}

Expand Down Expand Up @@ -546,7 +570,7 @@ test('command palette: open over Traces', async ({ page }) => {
await expect(input).toBeFocused();
await input.fill('go');
await page.waitForTimeout(150);
await page.screenshot({ path: path.join(OUT, 'studio-command-palette.png') });
await snap(page, 'studio-command-palette.png');
});

// Review-only shots of the shell (`x-` names: not in the docs set).
Expand All @@ -556,18 +580,18 @@ test('x: collapsed rail on Traces', async ({ page }) => {
await page.getByRole('button', { name: 'Collapse navigation' }).click();
await page.waitForTimeout(250);
await page.mouse.move(0, 0);
await page.screenshot({ path: path.join(OUT, 'x-rail-collapsed.png') });
await snap(page, 'x-rail-collapsed.png');
});

test('x: 375 wide, Traces with the nav drawer closed and open', async ({ page }) => {
await page.setViewportSize({ width: 375, height: 800 });
await open(page, '/traces');
await expect(rows(page).first()).toBeVisible();
await page.waitForTimeout(300);
await page.screenshot({ path: path.join(OUT, 'x-narrow-traces.png') });
await snap(page, 'x-narrow-traces.png');
await page.getByRole('button', { name: 'Navigation', exact: true }).click();
await page.waitForTimeout(300);
await page.screenshot({ path: path.join(OUT, 'x-narrow-drawer-open.png') });
await snap(page, 'x-narrow-drawer-open.png');
});

/*
Expand Down Expand Up @@ -714,7 +738,7 @@ test('x-kit: filter bar with a facet and the custom range open', async ({ page }
await main.locator('.facet-btn').first().click();
await expect(main.locator('.facet-pop')).toBeVisible();
await page.waitForTimeout(300);
await page.screenshot({ path: path.join(OUT, 'studio-x-kit-filters.png') });
await snap(page, 'studio-x-kit-filters.png');
});

// --- Overlay/message/chart kit states (x- shots, D3b) -------------------------
Expand Down Expand Up @@ -800,7 +824,7 @@ test('x-kit: chart tooltip, crosshair and legend with a series toggled off', asy
await page.mouse.move(box.x + box.width * 0.62, box.y + box.height / 2);
await expect(runs.locator('.tsc-crosshair')).toHaveCount(1); // a zero-width line never counts as visible
await page.waitForTimeout(200);
await page.screenshot({ path: path.join(OUT, 'studio-x-kit-chart-hover.png') });
await snap(page, 'studio-x-kit-chart-hover.png');
});

test('x-kit: toasts (info and error)', async ({ page }) => {
Expand All @@ -822,7 +846,7 @@ test('x-kit: toasts (info and error)', async ({ page }) => {
await expect(page.getByRole('alert').filter({ hasText: 'Could not disable' })).toBeVisible();
await page.mouse.move(0, 0);
await page.waitForTimeout(200);
await page.screenshot({ path: path.join(OUT, 'studio-x-kit-toasts.png') });
await snap(page, 'studio-x-kit-toasts.png');
});

test('x-kit: command palette sections, scrolled to the last group', async ({ page }) => {
Expand All @@ -834,7 +858,7 @@ test('x-kit: command palette sections, scrolled to the last group', async ({ pag
// ArrowUp from the first option wraps to the last one (Toggle theme), scrolling the list to the end.
await page.keyboard.press('ArrowUp');
await page.waitForTimeout(250);
await page.screenshot({ path: path.join(OUT, 'studio-x-kit-palette-sections.png') });
await snap(page, 'studio-x-kit-palette-sections.png');
});

/*
Expand Down Expand Up @@ -1119,7 +1143,7 @@ test('x-traces: detail at 375 wide', async ({ page }) => {
await open(page, `/traces/${SEED_NEWEST_FAILURE.trace_id}`);
await expect(page.getByRole('main').getByRole('alert')).toContainText(SEED_ERROR);
await shoot(page, 'x-trace-narrow');
await page.screenshot({ path: path.join(OUT, 'studio-x-trace-narrow-full.png'), fullPage: true });
await snap(page, 'studio-x-trace-narrow-full.png', { fullPage: true });
});

test("x-traces: the Agents drawer's Trace tab (narrow host)", async ({ page }) => {
Expand Down Expand Up @@ -1253,7 +1277,7 @@ test('x-threads: detail with long turns (collapsed, expanded, errored)', async (
await shoot(page, 'x-thread-expanded');
await turnsOf(page).nth(2).scrollIntoViewIfNeeded();
await shoot(page, 'x-thread-error');
await page.screenshot({ path: path.join(OUT, 'studio-x-thread-long-full.png'), fullPage: true });
await snap(page, 'studio-x-thread-long-full.png', { fullPage: true });
});

test('x-threads: detail at 375 wide', async ({ page }) => {
Expand All @@ -1262,7 +1286,7 @@ test('x-threads: detail at 375 wide', async ({ page }) => {
await open(page, `/threads/${LONG_THREAD}`);
await expect(turnsOf(page)).toHaveCount(THREAD_TURNS.length);
await shoot(page, 'x-thread-narrow');
await page.screenshot({ path: path.join(OUT, 'studio-x-thread-narrow-full.png'), fullPage: true });
await snap(page, 'studio-x-thread-narrow-full.png', { fullPage: true });
});

// --- Evaluators and annotation queues (x- shots, P7) -------------------------
Expand Down
35 changes: 27 additions & 8 deletions docs/agent-forge.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ This doc covers: installation, the `lousho studio` quickstart, the
authoring, time travel (replaying a run from a past step), the playground
endpoint, and how settings/secrets/deploy wiring works.

![The Agents screen: nav, canvas, inspector and the bottom drawer](./images/studio-agents.png)
![The Agents screen: nav, agent rail, the fitted canvas with the Inspector folded, and the bottom drawer](./images/studio-agents.png)

## Installation

Expand Down Expand Up @@ -49,8 +49,8 @@ agent rail (your saved agents plus a node/hook palette), the canvas, the
Inspector (right), and a bottom drawer with Chat, Logs, Trace, Output,
History and Settings tabs. An agent opens with nothing selected and the
whole graph fitted in view; the Inspector stays folded away until you
select a node, or press **Show inspector** (the panel icon at the canvas's
top-right), which is remembered. The other screens are one click away in the
select a node, or press the **Inspector** button at the canvas's
top-right (its accessible name is "Show inspector"), which is remembered. The other screens are one click away in the
navigation; see [Navigation](#navigation).

Below 860px wide the navigation becomes a drawer opened from the
Expand Down Expand Up @@ -299,26 +299,34 @@ says so and keeps an empty editor, and runs still work. Runs go through the

![The Traces list](./images/studio-traces.png)

![The Traces list in the dark theme](./images/studio-traces-dark.png)

The same list in the dark theme (Settings > Appearance, or **Toggle theme** in the command palette).

`#/traces` lists the daemon's runs, newest first: status, name, agent,
input and output previews, error message, model, start, duration, tokens
and cost (narrow windows hide the less important columns). Filter with the
search box (name, input or output), the **Agent**, **Status**, **Model** and
**Project** facets, and a time range (15m, 1h, 24h, 7d, 30d or Custom;
default the last 24 h). Click **Duration**, **Tokens** or **Cost** to sort
by it. 50 rows a page. A `prompt` filter (set by the Prompts screen's
by it. 50 rows a page. The page header's refresh icon (**Refresh**) re-fetches the list
now, and **Live** (a toggle) refreshes it every 5 seconds until you switch
it off or the daemon goes down. A `prompt` filter (set by the Prompts screen's
"Runs using this version" link) shows as a removable chip.

### Trace

`#/traces/<trace id>` is one run. The header has its name, status, agent
(a link to the agent's traces), model, start, duration, tokens, cost and
session (a link to its thread), plus **Copy link**, **Add to dataset** and
**Add to queue**.
session (a link to its thread), plus **Copy link**, **Add to dataset**,
**Add to queue** and, when the run has a model call, **Open in Playground**.
**All traces** in the page header goes back to the list.

![A failed run: the error banner on top of the inspector](./images/studio-trace.png)

Below it is the trace viewer the builder's Trace tab and the queue review
also use: the steps as a **Tree** or a **Waterfall**, and an inspector for
also use: the steps as a **Tree** or a **Waterfall** (a two-way switch above
the steps), and an inspector for
the selected step (a failed step is selected by default, else the root).
The selected step is `?obs=` in the URL. The inspector shows, in order:

Expand Down Expand Up @@ -852,12 +860,23 @@ LOUSHO_DOC_SHOTS=1 npx playwright test -c apps/agent-forge/playwright.config.ts
```

It writes `docs/images/studio-<screen>.png` (1280x800, light theme) from the
same fake daemon. The images are not part of the npm package.
same fake daemon. The images are not part of the npm package. The dark
examples (`studio-{agents,traces,trace,monitoring}-dark.png`) come from the
same command with the dark scheme:

```bash
LOUSHO_DOC_SHOTS=1 LOUSHO_DOC_SHOTS_SCHEME=dark npx playwright test -c apps/agent-forge/playwright.config.ts docShots
```

Into `docs/images` the spec skips its `x-` review shots, and a dark run
writes only those four.

Two optional variables help review a restyle before and after:
`LOUSHO_DOC_SHOTS_OUT=<dir>` writes the PNGs to another directory (absolute,
or relative to where you run the command), and `LOUSHO_DOC_SHOTS_SCHEME=dark`
shoots the dark theme (`light` is the default; any other value fails the run).
With an output directory every shot is written under its own name, `x-`
review shots included.

```bash
LOUSHO_DOC_SHOTS=1 LOUSHO_DOC_SHOTS_OUT=/tmp/shots-dark LOUSHO_DOC_SHOTS_SCHEME=dark npx playwright test -c apps/agent-forge/playwright.config.ts docShots
Expand Down
Binary file added docs/images/studio-agents-dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/studio-agents.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/studio-command-palette.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/studio-datasets.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/studio-evaluators.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/studio-monitoring-dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/studio-monitoring.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/studio-playground.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/studio-prompts.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/studio-queue-review.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/studio-scores.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/studio-threads.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/studio-trace-dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/studio-trace.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/studio-traces-dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/studio-traces.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
31 changes: 24 additions & 7 deletions llms-full.txt
Original file line number Diff line number Diff line change
Expand Up @@ -1952,8 +1952,8 @@ agent rail (your saved agents plus a node/hook palette), the canvas, the
Inspector (right), and a bottom drawer with Chat, Logs, Trace, Output,
History and Settings tabs. An agent opens with nothing selected and the
whole graph fitted in view; the Inspector stays folded away until you
select a node, or press **Show inspector** (the panel icon at the canvas's
top-right), which is remembered. The other screens are one click away in the
select a node, or press the **Inspector** button at the canvas's
top-right (its accessible name is "Show inspector"), which is remembered. The other screens are one click away in the
navigation; see [Navigation](#navigation).

Below 860px wide the navigation becomes a drawer opened from the
Expand Down Expand Up @@ -2196,24 +2196,30 @@ says so and keeps an empty editor, and runs still work. Runs go through the

### Traces

The same list in the dark theme (Settings > Appearance, or **Toggle theme** in the command palette).

`#/traces` lists the daemon's runs, newest first: status, name, agent,
input and output previews, error message, model, start, duration, tokens
and cost (narrow windows hide the less important columns). Filter with the
search box (name, input or output), the **Agent**, **Status**, **Model** and
**Project** facets, and a time range (15m, 1h, 24h, 7d, 30d or Custom;
default the last 24 h). Click **Duration**, **Tokens** or **Cost** to sort
by it. 50 rows a page. A `prompt` filter (set by the Prompts screen's
by it. 50 rows a page. The page header's refresh icon (**Refresh**) re-fetches the list
now, and **Live** (a toggle) refreshes it every 5 seconds until you switch
it off or the daemon goes down. A `prompt` filter (set by the Prompts screen's
"Runs using this version" link) shows as a removable chip.

### Trace

`#/traces/<trace id>` is one run. The header has its name, status, agent
(a link to the agent's traces), model, start, duration, tokens, cost and
session (a link to its thread), plus **Copy link**, **Add to dataset** and
**Add to queue**.
session (a link to its thread), plus **Copy link**, **Add to dataset**,
**Add to queue** and, when the run has a model call, **Open in Playground**.
**All traces** in the page header goes back to the list.

Below it is the trace viewer the builder's Trace tab and the queue review
also use: the steps as a **Tree** or a **Waterfall**, and an inspector for
also use: the steps as a **Tree** or a **Waterfall** (a two-way switch above
the steps), and an inspector for
the selected step (a failed step is selected by default, else the root).
The selected step is `?obs=` in the URL. The inspector shows, in order:

Expand Down Expand Up @@ -2733,12 +2739,23 @@ LOUSHO_DOC_SHOTS=1 npx playwright test -c apps/agent-forge/playwright.config.ts
```

It writes `docs/images/studio-<screen>.png` (1280x800, light theme) from the
same fake daemon. The images are not part of the npm package.
same fake daemon. The images are not part of the npm package. The dark
examples (`studio-{agents,traces,trace,monitoring}-dark.png`) come from the
same command with the dark scheme:

```bash
LOUSHO_DOC_SHOTS=1 LOUSHO_DOC_SHOTS_SCHEME=dark npx playwright test -c apps/agent-forge/playwright.config.ts docShots
```

Into `docs/images` the spec skips its `x-` review shots, and a dark run
writes only those four.

Two optional variables help review a restyle before and after:
`LOUSHO_DOC_SHOTS_OUT=<dir>` writes the PNGs to another directory (absolute,
or relative to where you run the command), and `LOUSHO_DOC_SHOTS_SCHEME=dark`
shoots the dark theme (`light` is the default; any other value fails the run).
With an output directory every shot is written under its own name, `x-`
review shots included.

```bash
LOUSHO_DOC_SHOTS=1 LOUSHO_DOC_SHOTS_OUT=/tmp/shots-dark LOUSHO_DOC_SHOTS_SCHEME=dark npx playwright test -c apps/agent-forge/playwright.config.ts docShots
Expand Down
Loading