Repository navigation
feat(ui): voice dictation on AI prompt inputs - #97
Merged
Merged
Conversation
Adds a mic button to every surface where a natural-language prompt is typed, backed by the browser's Web Speech API — no new dependencies, no API route, no AWS cost. New modules: - hooks/use-dictation.ts — binds speech recognition to an already-controlled text field. Snapshots the value at session start, then rewrites `snapshot + finalSoFar + interim` on each result event, so words appear live while the interim tail is swapped for the final transcript instead of duplicating after it. Continuous + interim recognition, mic released on unmount, session stopped when the field disables, and permission/capture errors surfaced as toasts while no-speech/aborted stay silent. - components/voice/mic-button.tsx — ghost icon button that pulses and reports aria-pressed while listening; renders nothing on browsers without the API rather than showing a dead control. - components/voice/dictation-textarea.tsx — drop-in Textarea wrapper that tucks the mic into the bottom-right corner. Wired into nine surfaces: AI Ops composer, Inventory Ask AI, KB chat, Deep Agent chat, Agent Ops new-run dialog, scheduled-task objective, both HIL respond textareas, and the chat clarification card. Dictation appends to existing text everywhere rather than replacing it. Also fixes Deep Agent chat auto-resizing its textarea inside the onChange handler off e.target, which meant programmatic writes never resized it. Moved to an effect keyed on the value, which also fixes the box not shrinking when cleared on send. The engine is isolated inside useDictation, so a server-side model (Parakeet, Transcribe) can be swapped in later without touching any call site. Tests: 21 new (append-not-replace, interim to final swap, cumulative multi-segment accumulation, maxLength clamping, unsupported browser, disabled-mid-session, silence timeout, permission errors, unmount cleanup). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a mic button to every surface where a natural-language prompt is typed, backed by the browser's Web Speech API. No new dependencies, no API route, no AWS cost.
What's new
hooks/use-dictation.tscomponents/voice/mic-button.tsxaria-pressedwhile listeningcomponents/voice/dictation-textarea.tsxTextareawrapper with the mic in the bottom-right corneruseDictationsnapshots the field value when a session starts, then rewritessnapshot + finalSoFar + interimon each result event. Words appear live as you speak, and the interim tail is swapped for the final transcript rather than duplicating after it. Continuous + interim recognition; the mic is released on unmount, the session stops when the field disables, andnot-allowed/audio-capture/networkerrors surface as toasts whileno-speech/abortedstay silent.isSupportedresolves in an effect so SSR and hydration agree.Surfaces wired (9)
AI Ops composer (next to the existing Enhance-prompt wand, sharing its 2000-char cap) · Inventory Ask AI · KB chat · Deep Agent chat · Agent Ops new-run dialog · scheduled-task Objective · both HIL respond textareas · chat clarification card.
Dictation appends to existing text everywhere rather than clobbering it. On browsers without the API (Firefox) the mic renders nothing at all instead of showing a dead control.
Incidental fix
Deep Agent chat auto-resized its textarea inside the
onChangehandler offe.target, so programmatic writes never resized it. Moved to an effect keyed on the value — this also fixes the box not shrinking when cleared on send.Verification
maxLengthclamping, unsupported browser, disabled-mid-session, silence timeout, permission errors, unmount cleanupcomponents/agent,components/voice,hookstsc --noEmitclean on every touched file; ESLint clean on all three new filesnext buildsucceedsThe full suite reports 59 failures across 19 files — all pre-existing, none in files touched here. They are DB-backed tests failing on
Can't reach database server at localhost:5432plus the known mock-harness set.Note on the engine
Chrome routes captured audio to Google's servers for recognition, worth weighing for prompts that name real account IDs and ARNs. The engine is isolated inside
useDictation, so a server-side model (self-hosted Parakeet viasherpa-onnx-node, or AWS Transcribe streaming) can be swapped in later by changing one file — the nine call sites,MicButton, andDictationTextareaall stay as they are.🤖 Generated with Claude Code