From eb011c5cad0b5a5eb3cf62bedd48e65a4c74f23d Mon Sep 17 00:00:00 2001 From: Stephen Smith Date: Thu, 29 Oct 2026 10:04:24 -0700 Subject: [PATCH] Updated Live call control and listen to a call pages --- fern/calls/call-dynamic-transfers.mdx | 10 +- fern/calls/call-features.mdx | 535 +++++++++++++----- fern/calls/call-listen.mdx | 190 +++++++ fern/calls/call-queue-management.mdx | 2 +- fern/calls/customer-join-timeout.mdx | 2 +- fern/calls/transfer-calls-with-api-lookup.mdx | 6 +- fern/calls/websocket-transport.mdx | 4 +- fern/docs.yml | 4 +- fern/server-url/events.mdx | 3 +- fern/tools/api-request-vs-function.mdx | 4 +- fern/tools/integrations/make.mdx | 2 +- 11 files changed, 592 insertions(+), 170 deletions(-) create mode 100644 fern/calls/call-listen.mdx diff --git a/fern/calls/call-dynamic-transfers.mdx b/fern/calls/call-dynamic-transfers.mdx index 7c2539389..fe08a50b7 100644 --- a/fern/calls/call-dynamic-transfers.mdx +++ b/fern/calls/call-dynamic-transfers.mdx @@ -29,7 +29,7 @@ For a lookup that chooses among predefined branch or department phone numbers, f ## How It Works -Dynamic transfers with live call control use a server-controlled pattern that gives you maximum flexibility: +Dynamic transfers with [live call control](/calls/call-features) use a server-controlled pattern that gives you maximum flexibility: 1. **User initiates transfer**: The user requests a transfer in natural language during the conversation 2. **Vapi triggers a Function tool**: Vapi sends the Function tool request to your HTTP server @@ -309,8 +309,8 @@ sequenceDiagram - - Your server will receive the tool call with `message.call.monitor.controlUrl` and use it to execute the transfer via Live Call Control. + + Your server will receive the tool call with `message.call.monitor.controlUrl` and use it to execute the transfer through live call control. @@ -361,7 +361,7 @@ sequenceDiagram }; } - // Execute transfer via Live Call Control + // Execute transfer through live call control await axios.post(controlUrl, { type: "transfer", destination: destination, @@ -433,7 +433,7 @@ sequenceDiagram "number": "+1555555555" } - # Execute transfer via Live Call Control + # Execute transfer through live call control async with httpx.AsyncClient() as client: await client.post( control_url, diff --git a/fern/calls/call-features.mdx b/fern/calls/call-features.mdx index 9300bfdb7..15cc6741e 100644 --- a/fern/calls/call-features.mdx +++ b/fern/calls/call-features.mdx @@ -1,105 +1,193 @@ --- -title: Live Call Control +title: Live call control +subtitle: Send commands to an active call slug: calls/call-features -description: Control live Vapi calls in real time with Call Control to inject conversation elements and Call Listen to stream call audio over a WebSocket connection. +description: Use a call's controlUrl to speak, add messages, mute, transfer, hand off, or end a live Vapi call from your server. --- -Vapi offers two main features that provide enhanced control over live calls: +Live call control lets your server change a call while it's in progress. You send each command as an HTTPS POST to the call's `controlUrl`. Commands can make the assistant speak, add context to the conversation, mute either side, transfer the call, hand it off to another assistant or squad, or end it. -1. **Call Control**: This feature allows you to inject conversation elements dynamically during an ongoing call. -2. **Call Listen**: This feature enables real-time audio data streaming using WebSocket connections. +To stream a call's audio and events instead, see [Listen to live calls](/calls/call-listen). -To use these features, you first need to obtain the URLs specific to the live call. These URLs can be retrieved by triggering a `/call` endpoint, which returns the `listenUrl` and `controlUrl` within the `monitor` object. + +The control URL doesn't require authentication by default. Turn on [authentication](#require-authentication) so only requests with your public API key can control the call. + -## Obtaining URLs for Call Control and Listen +## Get the control URL -To initiate a call and retrieve the `listenUrl` and `controlUrl`, send a POST request to the `/call` endpoint. +Every call has its control URL in `call.monitor.controlUrl`. Read the URL from the call object instead of building it yourself. The host can change between calls. -### Sample Request +The control URL accepts commands while the call is in progress. You can get it from the create call response, from server messages, or from the call record. -```bash -curl 'https://api.vapi.ai/call' \ --H 'authorization: Bearer YOUR_API_KEY' \ --H 'content-type: application/json' \ ---data-raw '{ - "assistantId": "5b0a4a08-133c-4146-9315-0984f8c6be80", - "customer": { - "number": "+12345678913" - }, - "phoneNumberId": "42b4b25d-031e-4786-857f-63b346c9580f" -}' +### From the create call response -``` +For outbound calls, the [Create Call](/api-reference/calls/create) response includes `monitor.controlUrl`: -### Sample Response +```bash +curl https://api.vapi.ai/call \ + -H "Authorization: Bearer YOUR_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{ + "assistantId": "00000000-0000-4000-8000-000000000002", + "phoneNumberId": "00000000-0000-4000-8000-000000000003", + "customer": { + "number": "+14155550100" + } + }' +``` -```json +```json title="Response" { - "id": "7420f27a-30fd-4f49-a995-5549ae7cc00d", - "assistantId": "5b0a4a08-133c-4146-9315-0984f8c6be80", - "phoneNumberId": "42b4b25d-031e-4786-857f-63b346c9580f", + "id": "00000000-0000-4000-8000-000000000001", + "assistantId": "00000000-0000-4000-8000-000000000002", + "phoneNumberId": "00000000-0000-4000-8000-000000000003", "type": "outboundPhoneCall", - "createdAt": "2024-09-10T11:14:12.339Z", - "updatedAt": "2024-09-10T11:14:12.339Z", - "orgId": "eb166faa-7145-46ef-8044-589b47ae3b56", + "createdAt": "2026-10-28T17:00:00.000Z", + "updatedAt": "2026-10-28T17:00:00.000Z", + "orgId": "00000000-0000-4000-8000-000000000009", "cost": 0, "customer": { - "number": "+12345678913" + "number": "+14155550100" }, "status": "queued", "phoneCallProvider": "twilio", - "phoneCallProviderId": "CA4c6793d069ef42f4ccad69a0957451ec", + "phoneCallProviderId": "CA00000000000000000000000000000001", "phoneCallTransport": "pstn", "monitor": { - "listenUrl": "wss://aws-us-west-2-production1-phone-call-websocket.vapi.ai/7420f27a-30fd-4f49-a995-5549ae7cc00d/transport", - "controlUrl": "" + "listenUrl": "wss://phone-call-websocket.aws-us-west-2-backend-production1.vapi.ai/00000000-0000-4000-8000-000000000001/listen", + "controlUrl": "https://phone-call-websocket.aws-us-west-2-backend-production1.vapi.ai/00000000-0000-4000-8000-000000000001/control" } } +``` + +### From server messages + +For inbound calls, read the control URL from a [server message](/server-url/events). Server messages that include the call object carry it at `message.call.monitor.controlUrl`. The first `status-update` message with `status` set to `in-progress` arrives as soon as the call connects: +```json title="status-update message (trimmed)" +{ + "message": { + "type": "status-update", + "status": "in-progress", + "call": { + "id": "00000000-0000-4000-8000-000000000001", + "monitor": { + "listenUrl": "wss://phone-call-websocket.aws-us-west-2-backend-production1.vapi.ai/00000000-0000-4000-8000-000000000001/listen", + "controlUrl": "https://phone-call-websocket.aws-us-west-2-backend-production1.vapi.ai/00000000-0000-4000-8000-000000000001/control" + } + } + } +} ``` +The `tool-calls` server message also includes the control URL, so a [Function tool](/tools/custom-tools) handler can control the call it's running in. The `assistant-request` message doesn't include the control URL. + -GPT-Live supports a subset of live call control: HTTP `end-call`, `append-context` for speaker commentary, thinking, or instructions, and cold `transfer` on supported phone connections. The `say`, `add-message`, and mute/unmute controls below aren't supported by GPT-Live. Use the [GPT-Live live call control guide](/gpt-live/configuration#live-call-control) for its request examples and limits. +The `status-update` message is in the default `serverMessages` list. If you set `serverMessages` on your assistant, include `status-update`, because your list replaces the default. -## Call Control Features +### From the call record + +[Get Call](/api-reference/calls/get) returns `monitor.controlUrl` for any call. The URL stays on the call record after the call ends, so check that `status` is `in-progress` before you send a command. + +## Control settings + +Live call control is configured in the assistant's `monitorPlan`. It's on by default and doesn't require authentication by default. + +| Field | Default | Effect | +| --- | --- | --- | +| `controlEnabled` | `true` | Issues a `controlUrl` for the assistant's calls | +| `controlAuthenticationEnabled` | `false` | Requires a public API key on every control command | + +For the Call Listen fields, see [Listen settings](/calls/call-listen#listen-settings). + +### Turn live call control on or off + +Set `controlEnabled` to `false` to turn off live call control for an assistant. If any assistant on a call has `controlEnabled` set to `false` when the call starts, the call has no `controlUrl`. -Once you have the `controlUrl`, you can use various control features during a live call. Here are all the available control options: +A call that starts with control on can later reach an assistant with control off, such as after a handoff. From then on, every command returns `400` with this error: -### 1. Say Message -Makes the assistant say a specific message during the call. +```text +Couldn't Control Call. One of the assistants has `assistant.monitorPlan.controlEnabled` set to false. +``` + +### Require authentication + +Set `controlAuthenticationEnabled` to `true` to require your public API key on every command: + +```json +{ + "monitorPlan": { + "controlAuthenticationEnabled": true + } +} +``` + +Then send the public API key in the `Authorization` header: + +```bash +curl -X POST "$CONTROL_URL" \ + -H "Authorization: Bearer YOUR_PUBLIC_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{"type": "end-call"}' +``` + +A request with no key, or with your private API key, returns `400` with `{"error":"Unauthorized"}`. + +Authentication applies to the whole call. If any assistant on the call has `controlAuthenticationEnabled` set to `true`, every command on that call needs the public API key, including after a handoff to an assistant that doesn't require it. + +## Control actions + +Send each command as a JSON body to the control URL. The examples on this page set `CONTROL_URL` to the call's control URL. If you turned on [authentication](#require-authentication), add the `Authorization` header to each request. + +A `200` response with `{"status":"ok"}` means the command was accepted. It doesn't mean the command finished. For example, a transfer can still fail after the `200` response. + +### Action reference + +| `type` | What it does | Fields | +| --- | --- | --- | +| `say` | Speaks text or plays an audio file | `content`, `endCallAfterSpoken`, `interruptionsEnabled`, `interruptAssistantEnabled` | +| `add-message` | Adds a message to the conversation history | `message`, `triggerResponseEnabled` | +| `control` | Mutes or unmutes the assistant or the customer, or says the first message | `control` | +| `end-call` | Ends the call | None | +| `transfer` | Transfers the call to a phone number or Session Initiation Protocol (SIP) URI | `destination`, `content` | +| `handoff` | Hands the call to another assistant or a squad | `destination`, `content` | + +### Speak a message + +Send a `say` command to make the assistant speak exact text: ```bash -curl -X POST 'https://aws-us-west-2-production1-phone-call-websocket.vapi.ai/7420f27a-30fd-4f49-a995-5549ae7cc00d/control' \ --H 'content-type: application/json' \ ---data-raw '{ - "type": "say", - "content": "Welcome to Vapi, this message was injected during the call.", - "endCallAfterSpoken": false -}' +curl -X POST "$CONTROL_URL" \ + -H "Content-Type: application/json" \ + -d '{ + "type": "say", + "content": "Thanks for waiting. Your order is ready.", + "endCallAfterSpoken": false + }' ``` Optional fields: -- `endCallAfterSpoken`: end the call after the message is spoken. -- `interruptionsEnabled`: let the caller interrupt the message. Defaults to `false`, which ignores the caller's audio while the message plays. -- `interruptAssistantEnabled`: stop any speech already in progress or queued and say this message immediately. The assistant doesn't resume the interrupted response. Defaults to `false`, which waits until the assistant finishes speaking. +- `endCallAfterSpoken`: End the call after the message is spoken. +- `interruptionsEnabled`: Let the caller interrupt the message. Defaults to `false`, which ignores the caller's audio while the message plays. +- `interruptAssistantEnabled`: Stop any speech already in progress or queued and say this message immediately. The assistant doesn't resume the interrupted response. Defaults to `false`, which waits until the assistant finishes speaking. -#### Play an audio file +### Play an audio file -To play a pre-recorded audio file instead of synthesized speech, set `content` to the file's URL. +To play a pre-recorded audio file instead of synthesized speech, set `content` to the file's URL: ```bash -curl -X POST 'https://aws-us-west-2-production1-phone-call-websocket.vapi.ai/7420f27a-30fd-4f49-a995-5549ae7cc00d/control' \ --H 'content-type: application/json' \ ---data-raw '{ - "type": "say", - "content": "https://example.com/audio/hold-message.wav" -}' +curl -X POST "$CONTROL_URL" \ + -H "Content-Type: application/json" \ + -d '{ + "type": "say", + "content": "https://example.com/audio/hold-message.wav" + }' ``` -Set `content` to a single `https://` URL with no other text. The text might be read out loud to the caller. +Set `content` to a single `https://` URL with no other text. Other values are spoken as text or don't play, as the following table shows. | `content` value | Result | @@ -124,135 +212,278 @@ Each `say` request plays one audio file when you send it. Audio doesn't play aut If Vapi can't download or decode the file, nothing plays and the control request still returns a success response. Test each audio URL before you use it on live calls. -### 2. Add Message to Conversation -Adds a message to the conversation history and optionally triggers a response. +### Add a message to the conversation + +Send an `add-message` command to add a message to the conversation history. The `message` object takes a `role` of `system`, `user`, or `assistant`, and a `content` string. + +The `triggerResponseEnabled` field controls what happens next: + +- `true` (default): The assistant responds right away. +- `false`: The message is added silently. The assistant uses it on its next turn. + +This example adds a system message and has the assistant respond: + +```bash +curl -X POST "$CONTROL_URL" \ + -H "Content-Type: application/json" \ + -d '{ + "type": "add-message", + "message": { + "role": "system", + "content": "New message added to conversation" + }, + "triggerResponseEnabled": true + }' +``` + +This example adds context without interrupting the conversation: + +```bash +curl -X POST "$CONTROL_URL" \ + -H "Content-Type: application/json" \ + -d '{ + "type": "add-message", + "message": { + "role": "system", + "content": "The caller is a premium customer. Offer free shipping." + }, + "triggerResponseEnabled": false + }' +``` + +If the caller is speaking when a triggered response starts, the caller can interrupt it. + +### Mute or unmute the assistant + +Send a `control` command with `mute-assistant` to stop the assistant from speaking. Send `unmute-assistant` to let it speak again. + +```bash +curl -X POST "$CONTROL_URL" \ + -H "Content-Type: application/json" \ + -d '{ + "type": "control", + "control": "mute-assistant" + }' +``` + +While the assistant is muted, it still hears and transcribes the caller. + +### Mute or unmute the customer + +Send a `control` command with `mute-customer` to stop the assistant from hearing the caller. Send `unmute-customer` to let it hear the caller again. + +```bash +curl -X POST "$CONTROL_URL" \ + -H "Content-Type: application/json" \ + -d '{ + "type": "control", + "control": "mute-customer" + }' +``` + +Muting the customer doesn't change what anyone on the call hears. The assistant stops listening, and the caller's speech isn't transcribed. The caller gets no signal that they're muted, so send a `say` command first if the caller should know. + + +The customer stays muted until you send `unmute-customer`. If you never unmute, the assistant can't hear the caller for the rest of the call. + + +### Say the first message + +Send a `control` command with `say-first-message` to make the assistant say its first message. Use it when video recording is on and the conversation starts only after your client starts recording. ```bash -curl -X POST 'https://aws-us-west-2-production1-phone-call-websocket.vapi.ai/7420f27a-30fd-4f49-a995-5549ae7cc00d/control' \ --H 'content-type: application/json' \ ---data-raw '{ - "type": "add-message", - "message": { - "role": "system", - "content": "New message added to conversation" - }, - "triggerResponseEnabled": true -}' +curl -X POST "$CONTROL_URL" \ + -H "Content-Type: application/json" \ + -d '{ + "type": "control", + "control": "say-first-message" + }' ``` -### 3. Assistant Control -Control the assistant's behavior during the call. +### End the call + +Send an `end-call` command to end the call: ```bash -curl -X POST 'https://aws-us-west-2-production1-phone-call-websocket.vapi.ai/7420f27a-30fd-4f49-a995-5549ae7cc00d/control' \ --H 'content-type: application/json' \ ---data-raw '{ - "type": "control", - "control": "mute-assistant" // Options: "mute-assistant", "unmute-assistant", "say-first-message" -}' +curl -X POST "$CONTROL_URL" \ + -H "Content-Type: application/json" \ + -d '{"type": "end-call"}' ``` -### 4. End Call -Programmatically end the ongoing call. +The call ends with the `assistant-ended-call-after-message-spoken` ended reason. To say goodbye first, send a `say` command with `endCallAfterSpoken` set to `true` instead. + +### Transfer the call + +Send a `transfer` command to transfer the call to a phone number or a SIP URI. The optional `content` field is spoken to the caller before the transfer. ```bash -curl -X POST 'https://aws-us-west-2-production1-phone-call-websocket.vapi.ai/7420f27a-30fd-4f49-a995-5549ae7cc00d/control' \ --H 'content-type: application/json' \ ---data-raw '{ - "type": "end-call" -}' +curl -X POST "$CONTROL_URL" \ + -H "Content-Type: application/json" \ + -d '{ + "type": "transfer", + "destination": { + "type": "number", + "number": "+14155550101" + }, + "content": "Transferring your call now." + }' ``` -### 5. Transfer Call -Transfer the call to a different destination. +To transfer to a SIP URI, set the destination `type` to `sip`: ```bash -curl -X POST 'https://aws-us-west-2-production1-phone-call-websocket.vapi.ai/7420f27a-30fd-4f49-a995-5549ae7cc00d/control' \ --H 'content-type: application/json' \ ---data-raw '{ - "type": "transfer", - "destination": { - "type": "number", - "number": "+1234567890" - }, - "content": "Transferring your call now" -}' +curl -X POST "$CONTROL_URL" \ + -H "Content-Type: application/json" \ + -d '{ + "type": "transfer", + "destination": { + "type": "sip", + "sipUri": "sip:+14155550101@sip.example.com" + }, + "content": "Transferring your call now." + }' ``` -You can also transfer to a SIP URI: +With `content`, the transfer starts after the message is spoken. Without `content`, the transfer starts right away. + +The `destination` object takes the same fields as a [Transfer Call tool](/tools/transfer-call) destination. + +### Hand off to an assistant or squad + +Send a `handoff` command to move the call to another assistant or to a squad. The optional `content` field is spoken to the caller before the handoff. + +To hand off to a saved assistant, set `assistantId`: ```bash -curl -X POST 'https://aws-us-west-2-production1-phone-call-websocket.vapi.ai/7420f27a-30fd-4f49-a995-5549ae7cc00d/control' \ --H 'content-type: application/json' \ ---data-raw '{ - "type": "transfer", - "destination": { - "type": "sip", - "sipUri": "sip:+transferPhoneNumber@sip.telnyx.com" - }, - "content": "Testing transfer call." -}' +curl -X POST "$CONTROL_URL" \ + -H "Content-Type: application/json" \ + -d '{ + "type": "handoff", + "destination": { + "type": "assistant", + "assistantId": "00000000-0000-4000-8000-000000000004" + }, + "content": "Connecting you to our billing assistant." + }' ``` -### 6. Handoff Call -Handoff the call to a different assistant. +To hand off to an assistant defined inline, set `assistant`. This example also sets `contextEngineeringPlan` so the new assistant starts without the earlier conversation: ```bash -curl -X POST 'https://aws-us-west-2-production1-phone-call-websocket.vapi.ai/7420f27a-30fd-4f49-a995-5549ae7cc00d/control' \ --H 'content-type: application/json' \ ---data-raw '{ - "type": "handoff", - "destination": { - "type": "assistant", - "contextEngineeringPlan": "none", - "assistant": { - "name": "new_assistant", - "voice": { - "provider": "vapi", - "version": 2, - "voiceId": "Elliot" +curl -X POST "$CONTROL_URL" \ + -H "Content-Type: application/json" \ + -d '{ + "type": "handoff", + "destination": { + "type": "assistant", + "contextEngineeringPlan": { + "type": "none" + }, + "assistant": { + "name": "Billing assistant", + "firstMessage": "Hi, this is the billing assistant.", + "model": { + "provider": "openai", + "model": "gpt-4.1", + "messages": [ + { + "role": "system", + "content": "You help callers with billing questions. Keep replies short." + } + ] }, - } - }, - "content": "Handing off your call now" -}' + "voice": { + "provider": "vapi", + "voiceId": "Elliot" + } + } + }, + "content": "Handing off your call now." + }' +``` + +To hand off to a squad, set the destination `type` to `squad` and pass a `squadId` or an inline `squad`. This example defines the squad inline: + +```bash +curl -X POST "$CONTROL_URL" \ + -H "Content-Type: application/json" \ + -d '{ + "type": "handoff", + "destination": { + "type": "squad", + "squad": { + "members": [ + { + "assistant": { + "name": "Support assistant", + "firstMessage": "Hi, this is the support team.", + "model": { + "provider": "openai", + "model": "gpt-4.1", + "messages": [ + { + "role": "system", + "content": "You help callers with support questions. Keep replies short." + } + ] + }, + "voice": { + "provider": "vapi", + "voiceId": "Lily" + } + } + } + ] + } + }, + "content": "Connecting you to our support team." + }' ``` +Use `contextEngineeringPlan` to choose the conversation history the destination receives. For every destination field, see [Configure handoff destinations](/squads/handoff/destinations) and [Pass context and variables](/squads/handoff/context-and-variables). -## Call Listen Feature +## GPT-Live support -The `listenUrl` allows you to connect to a WebSocket and stream the audio data in real-time. You can either process the audio directly or save the binary data to analyze or replay later. +GPT-Live calls support a subset of live call control: -### Example: Saving Audio Data from a Live Call +- HTTP `end-call` +- `append-context` to add speaker commentary, thinking, or instructions +- Cold `transfer` on supported phone connections -Here is a simple implementation for saving the audio buffer from a live call using Node.js: +The `say`, `add-message`, `control`, and `handoff` commands aren't supported on GPT-Live calls. For request examples and limits, see the [GPT-Live live call control guide](/gpt-live/configuration#live-call-control). -```jsx -const WebSocket = require('ws'); -const fs = require('fs'); +## What live call control can't do -let pcmBuffer = Buffer.alloc(0); +Live call control has a fixed set of actions. It can't do these things: -const ws = new WebSocket("wss://aws-us-west-2-production1-phone-call-websocket.vapi.ai/7420f27a-30fd-4f49-a995-5549ae7cc00d/transport"); +- **Pause and resume a call:** There's no pause action. Muting both the assistant and the customer is the closest option. +- **Change the assistant's settings:** There's no action to change the model, voice, or prompt mid-call. Add context with [`add-message`](#add-a-message-to-the-conversation), or [hand off](#hand-off-to-an-assistant-or-squad) to an assistant that has the settings you need. +- **Control a call that has ended:** Commands only work while the call is in progress. -ws.on('open', () => console.log('WebSocket connection established')); +## Troubleshoot control commands -ws.on('message', (data, isBinary) => { - if (isBinary) { - pcmBuffer = Buffer.concat([pcmBuffer, data]); - console.log(`Received PCM data, buffer size: ${pcmBuffer.length}`); - } else { - console.log('Received message:', JSON.parse(data.toString())); - } -}); +Each entry starts with the response or symptom you see, then gives the cause and fix. -ws.on('close', () => { - if (pcmBuffer.length > 0) { - fs.writeFileSync('audio.pcm', pcmBuffer); - console.log('Audio data saved to audio.pcm'); - } -}); + + + A `400` response with `{"error":"Unauthorized"}` means the call requires authentication. Send your public API key in the `Authorization` header. A private API key is refused. See [Require authentication](#require-authentication). + + + This error means an assistant on the call has `controlEnabled` set to `false`: -ws.on('error', (error) => console.error('WebSocket error:', error)); + ```text + Couldn't Control Call. One of the assistants has `assistant.monitorPlan.controlEnabled` set to false. + ``` -``` + Set `controlEnabled` to `true` on every assistant the call can reach. See [Turn live call control on or off](#turn-live-call-control-on-or-off). + + + A `400` response with ``Call `` Not Active.`` means the call is no longer in progress. Don't retry; check the call's `status` and `endedReason` with [Get Call](/api-reference/calls/get). + + + A `500` response with `failed to route call` usually means the call has ended. This response can take about 30 seconds to arrive, so set your client timeout above 30 seconds. Retry once after 1 to 2 seconds. If it fails again, treat the call as ended. + + + An `add-message` command with `triggerResponseEnabled` set to `true` returns `200`, but the caller doesn't hear a reply. The caller was likely speaking when the response started and interrupted it. Send the command when the caller is quiet, or use a `say` command with exact text instead. + + diff --git a/fern/calls/call-listen.mdx b/fern/calls/call-listen.mdx new file mode 100644 index 000000000..d9a3a55da --- /dev/null +++ b/fern/calls/call-listen.mdx @@ -0,0 +1,190 @@ +--- +title: Listen to live calls +subtitle: Stream a call's audio and events over a WebSocket +slug: calls/call-listen +description: Use a call's listenUrl to stream live Vapi call audio and real-time events, such as transcripts, over a WebSocket to record, monitor, or analyze calls from your server. +--- + +Call Listen streams a call's audio and events to your server in real time over a WebSocket. Use it to record, monitor, or analyze a call while it's in progress. + +Call Listen is listen-only: you can't send audio into the call through it. To change the call, such as making the assistant speak or ending the call, use [live call control](/calls/call-features). + + +The listen URL doesn't require authentication by default. Turn on [authentication](#require-authentication) so only connections with your public API key can listen to the call. + + +## Get the listen URL + +Every call has its listen URL in `call.monitor.listenUrl`. Read the URL from the call object instead of building it yourself. The host can change between calls. The URL ends in `/listen`. + +You can get the listen URL from the create call response, from server messages, or from the call record. + +### From the create call response + +For outbound calls, the [Create Call](/api-reference/calls/create) response includes `monitor.listenUrl`: + +```json title="Response (trimmed)" +{ + "id": "00000000-0000-4000-8000-000000000001", + "status": "queued", + "monitor": { + "listenUrl": "wss://phone-call-websocket.aws-us-west-2-backend-production1.vapi.ai/00000000-0000-4000-8000-000000000001/listen", + "controlUrl": "https://phone-call-websocket.aws-us-west-2-backend-production1.vapi.ai/00000000-0000-4000-8000-000000000001/control" + } +} +``` + +### From server messages + +For inbound calls, read the listen URL from a [server message](/server-url/events). Server messages that include the call object carry it at `message.call.monitor.listenUrl`. The first `status-update` message with `status` set to `in-progress` arrives as soon as the call connects: + +```json title="status-update message (trimmed)" +{ + "message": { + "type": "status-update", + "status": "in-progress", + "call": { + "id": "00000000-0000-4000-8000-000000000001", + "monitor": { + "listenUrl": "wss://phone-call-websocket.aws-us-west-2-backend-production1.vapi.ai/00000000-0000-4000-8000-000000000001/listen", + "controlUrl": "https://phone-call-websocket.aws-us-west-2-backend-production1.vapi.ai/00000000-0000-4000-8000-000000000001/control" + } + } + } +} +``` + + +The `status-update` message is in the default `serverMessages` list. If you set `serverMessages` on your assistant, include `status-update`, because your list replaces the default. + + +### From the call record + +[Get Call](/api-reference/calls/get) returns `monitor.listenUrl` for any call. The URL stays on the call record after the call ends, so check that `status` is `in-progress` before you connect. + +## Listen settings + +Call Listen is configured in the assistant's `monitorPlan`. It's on by default and doesn't require authentication by default. + +| Field | Default | Effect | +| --- | --- | --- | +| `listenEnabled` | `true` | Issues a `listenUrl` for the assistant's calls | +| `listenAuthenticationEnabled` | `false` | Requires a public API key to connect to the `listenUrl` | + +### Turn Call Listen on or off + +Set `listenEnabled` to `false` to turn off Call Listen for an assistant: + +```json +{ + "monitorPlan": { + "listenEnabled": false + } +} +``` + +If any assistant on a call has `listenEnabled` set to `false` when the call starts, the call has no `listenUrl`. + +### Require authentication + +Set `listenAuthenticationEnabled` to `true` to require your public API key: + +```json +{ + "monitorPlan": { + "listenAuthenticationEnabled": true + } +} +``` + +Then send the public API key in the `Authorization` header when you open the WebSocket. Browser WebSocket clients can't set headers, so connect from your server. + +A connection with no key, or with your private API key, opens and then closes right away with code `1008` and the reason `Unauthorized`. + +Authentication applies to the whole call. If any assistant on the call has `listenAuthenticationEnabled` set to `true`, every connection to that call's listen URL needs the public API key. + +## Read the stream format + +The first message on the WebSocket is a JSON `start` message that describes the audio format: + +```json +{ + "type": "start", + "encoding": "linear16", + "container": "raw", + "sampleRate": 16000, + "channels": 2 +} +``` + +After that, binary messages carry raw audio in that format. With `linear16`, each sample is 16-bit signed pulse-code modulation (PCM), and stereo samples are interleaved. + +Text messages carry JSON call events alongside the audio: + +| `type` | Contents | +| --- | --- | +| `transcript` | Partial and final transcripts for the caller and the assistant | +| `speech-update` | When the caller or the assistant starts or stops speaking | +| `conversation-update` | The updated conversation history | +| `user-interrupted` | The caller interrupted the assistant | +| `model-output` | Text output from the model | +| `voice-input` | Text sent to the voice provider | +| `assistant.started` | A new assistant became active, such as after a handoff, with its configuration in `newAssistant` | + +## Example: save call audio + +This Node.js example saves the audio buffer from a live call to a raw PCM file and logs the JSON messages: + +```javascript +const WebSocket = require('ws'); +const fs = require('fs'); + +let pcmBuffer = Buffer.alloc(0); + +const ws = new WebSocket("wss://phone-call-websocket.aws-us-west-2-backend-production1.vapi.ai/00000000-0000-4000-8000-000000000001/listen"); + +ws.on('open', () => console.log('WebSocket connection established')); + +ws.on('message', (data, isBinary) => { + if (isBinary) { + pcmBuffer = Buffer.concat([pcmBuffer, data]); + console.log(`Received PCM data, buffer size: ${pcmBuffer.length}`); + } else { + console.log('Received message:', JSON.parse(data.toString())); + } +}); + +ws.on('close', () => { + if (pcmBuffer.length > 0) { + fs.writeFileSync('audio.pcm', pcmBuffer); + console.log('Audio data saved to audio.pcm'); + } +}); + +ws.on('error', (error) => console.error('WebSocket error:', error)); +``` + +If `listenAuthenticationEnabled` is `true`, pass your public API key in the `Authorization` header when you open the WebSocket: + +```javascript +const ws = new WebSocket(listenUrl, { + headers: { Authorization: 'Bearer YOUR_PUBLIC_API_KEY' }, +}); +``` + +To play the file, convert it with the values from the `start` message. For example, with `ffmpeg`: + +```bash +ffmpeg -f s16le -ar 16000 -ac 2 -i audio.pcm audio.wav +``` + +## Troubleshoot Call Listen + + + + A WebSocket close with code `1008` and the reason `Unauthorized` means the call requires authentication. Send your public API key in the `Authorization` header. A private API key is refused. See [Require authentication](#require-authentication). + + + An assistant on the call has `listenEnabled` set to `false`. Set `listenEnabled` to `true` on every assistant the call starts with. See [Turn Call Listen on or off](#turn-call-listen-on-or-off). + + diff --git a/fern/calls/call-queue-management.mdx b/fern/calls/call-queue-management.mdx index 908704129..fc6bc5821 100644 --- a/fern/calls/call-queue-management.mdx +++ b/fern/calls/call-queue-management.mdx @@ -701,7 +701,7 @@ Redis is required for this implementation. Ensure your Redis instance is properl Now that you have a production-ready call queue system with Redis persistence and callback-driven processing: -- **[Advanced Call Features](/calls/call-features):** Explore call recording, analysis, and advanced routing options +- **[Live call control](/calls/call-features):** Speak, transfer, hand off, or end an active call from your server - **[Monitoring & Analytics](/assistants/call-analysis):** Set up comprehensive call analytics and performance monitoring - **[Scaling Considerations](/calls/call-concurrency):** Plan, monitor, and scale simultaneous calls for high-volume deployments - **[Assistant Optimization](/assistants/personalization):** Enhance your assistants with personalization and dynamic variables diff --git a/fern/calls/customer-join-timeout.mdx b/fern/calls/customer-join-timeout.mdx index 4866366fa..cae521411 100644 --- a/fern/calls/customer-join-timeout.mdx +++ b/fern/calls/customer-join-timeout.mdx @@ -309,5 +309,5 @@ This message appears when a call ends naturally and is **informational only**— Now that you understand customer join timeouts: - **Monitor your metrics:** Check your [call analytics](/dashboard) for timeout-related failures -- **Explore call features:** Learn about [real-time call control](/calls/call-features) +- **Control active calls:** Learn about [live call control](/calls/call-features) - **Understand call failures:** Review [call ended reasons](/calls/call-ended-reason) for comprehensive troubleshooting diff --git a/fern/calls/transfer-calls-with-api-lookup.mdx b/fern/calls/transfer-calls-with-api-lookup.mdx index 511e8062f..d6c0abecf 100644 --- a/fern/calls/transfer-calls-with-api-lookup.mdx +++ b/fern/calls/transfer-calls-with-api-lookup.mdx @@ -17,13 +17,13 @@ The system prompt guides tool ordering and destination selection; it does not en Use this two-tool example when a lookup selects from predefined branch or department phone numbers. It suits a conversation where the assistant can wait for the lookup result, then offer a transfer. -Use [Dynamic call transfers](/calls/call-dynamic-transfers) when your server must control transfer execution, finish asynchronous work, or determine a destination phone number that is not preconfigured. That guide uses an asynchronous Function tool and [Live Call Control](/calls/call-features). +Use [Dynamic call transfers](/calls/call-dynamic-transfers) when your server must control transfer execution, finish asynchronous work, or determine a destination phone number that is not preconfigured. That guide uses an asynchronous Function tool and [live call control](/calls/call-features). The table compares the examples in these two guides, not every configuration the tools support. -| Consideration | API Request and Transfer Call tools | Asynchronous Function tool and Live Call Control | +| Consideration | API Request and Transfer Call tools | Asynchronous Function tool and live call control | | --- | --- | --- | -| Main benefit | Less server code. Your endpoint only returns a `2xx` JSON lookup result, with no Function tool webhook handler or Live Call Control request. | Your server validates routing rules and completes business operations before it sends the transfer request. | +| Main benefit | Less server code. Your endpoint only returns a `2xx` JSON lookup result, with no Function tool webhook handler or live call control request. | Your server validates routing rules and completes business operations before it sends the transfer request. | | Main tradeoff | Tool ordering and destination selection depend on the model following the prompt, not enforced backend rules. | You implement the webhook handler, transfer request, and backend failure handling. | | Destinations | Maintain the predefined phone numbers in the Transfer Call tool. | Your server determines the destination for each call. | | Waiting for work | The assistant waits for the lookup result before it offers a transfer. | The assistant does not wait for your webhook response. Your server sends the transfer while the call is active. | diff --git a/fern/calls/websocket-transport.mdx b/fern/calls/websocket-transport.mdx index 7ac27279a..631e13027 100644 --- a/fern/calls/websocket-transport.mdx +++ b/fern/calls/websocket-transport.mdx @@ -196,7 +196,7 @@ function hangupCall() { ## Ending the Call -The recommended way to end a call is using [Live Call Control](/calls/call-features#end-call) which provides more control and proper cleanup. +The recommended way to end a call is using [live call control](/calls/call-features#end-the-call), which provides more control and proper cleanup. Alternatively, you can end the WebSocket call directly: @@ -216,7 +216,7 @@ Vapi provides two WebSocket options: | Replaces phone/web as transport | Supplements existing calls | | Uses `provider: "vapi.websocket"` | Accessed via `monitor.listenUrl` | -Refer to [Live Call Control](/calls/call-features) for more on the Call Listen feature. +Refer to [Listen to live calls](/calls/call-listen) for more on the Call Listen feature. When using WebSocket transport, phone-based parameters (`phoneNumber` or `phoneNumberId`) are not permitted. These methods are mutually exclusive. diff --git a/fern/docs.yml b/fern/docs.yml index 21396897c..e12dd2feb 100644 --- a/fern/docs.yml +++ b/fern/docs.yml @@ -556,8 +556,10 @@ navigation: - section: In-call control icon: fa-light fa-sliders contents: - - page: Real-time call control + - page: Live call control path: calls/call-features.mdx + - page: Listen to live calls + path: calls/call-listen.mdx - page: Customer join timeout path: calls/customer-join-timeout.mdx - page: Voicemail detection diff --git a/fern/server-url/events.mdx b/fern/server-url/events.mdx index 1c6804d13..90131d12d 100644 --- a/fern/server-url/events.mdx +++ b/fern/server-url/events.mdx @@ -142,7 +142,7 @@ For inbound phone calls, you can specify the assistant dynamically. If a PhoneNu You must respond to the `assistant-request` webhook within 7.5 seconds end-to-end. This limit is fixed and not configurable: the telephony provider enforces a 15-second cap, and Vapi reserves ~7.5 seconds for call setup. The timeout value shown elsewhere in the dashboard does not apply to this webhook. To avoid timeouts: - - Return quickly with an existing assistantId or a minimal assistant, then enrich context asynchronously after the call starts using Live Call Control. + - Return quickly with an existing assistantId or a minimal assistant, then enrich context asynchronously after the call starts using live call control. - Host your webhook close to us-west-2 to reduce latency, and target < ~6s to allow for network jitter. @@ -202,7 +202,6 @@ If you want to immediately transfer the call without using an assistant, return When `destination` is present in the `assistant-request` response, the call forwards immediately and assistantId, assistant, squadId, and squad are ignored. You must still respond within 7.5 seconds. To transfer silently, set destination.message to an empty string. - For caller ID behavior, see Call features. Or return an error message to be spoken to the caller: diff --git a/fern/tools/api-request-vs-function.mdx b/fern/tools/api-request-vs-function.mdx index 3d199e865..655e8c4be 100644 --- a/fern/tools/api-request-vs-function.mdx +++ b/fern/tools/api-request-vs-function.mdx @@ -21,7 +21,7 @@ If you are unsure, start with an API Request tool. | Runs without waiting for a result | **Function tool** | Function tools support top-level `async: true`. | | Triggers a browser-side action through the Web SDK | **Function tool** | A Function tool without a server URL can emit a client-side `tool-calls` event. | | Must translate Vapi tool calls into XML, multipart data, or another custom protocol | **Function tool with an adapter** | Your webhook can transform the Vapi request before calling the downstream service. | -| Uses live call control | **Function tool** | The webhook includes call context and `message.call.monitor.controlUrl`, allowing your server to transfer, hand off, end, speak into, or add messages to the active call.| +| Uses [live call control](/calls/call-features) | **Function tool** | The webhook includes call context and `message.call.monitor.controlUrl`, allowing your server to transfer, hand off, end, speak into, or add messages to the active call.| ## Compare how they work @@ -38,7 +38,7 @@ If you are unsure, start with an API Request tool. | Failure response | A non-2xx response or invalid JSON fails the request | For synchronous tool-specific failures, return HTTP 200 with a string in the matching `results[].error` | | Execution | Synchronous; the assistant waits for the destination API | Synchronous by default; with top-level `async: true`, the assistant continues without waiting for the webhook response | | Web SDK calls | Supported; Vapi executes the API request | Supported; Vapi can call a server webhook or emit a client-side event | -| Performs custom, server-driven Live Call Control | **Function tool** | The webhook includes call context and `message.call.monitor.controlUrl`, allowing your server to control the active call. | +| Performs custom, server-driven live call control | **Function tool** | The webhook includes call context and `message.call.monitor.controlUrl`, allowing your server to control the active call. | Both tool types let the model decide when to call the tool and which model-generated arguments to supply. Validate caller-controlled values at the destination before using them for sensitive actions. diff --git a/fern/tools/integrations/make.mdx b/fern/tools/integrations/make.mdx index 3c5b88cd3..4b7135381 100644 --- a/fern/tools/integrations/make.mdx +++ b/fern/tools/integrations/make.mdx @@ -14,7 +14,7 @@ Connect a Vapi assistant to Make to invoke scenarios that coordinate actions and | An ordinary webhook that accepts and returns JSON | **[API Request tool](/tools/api-request)** | Vapi sends the configured HTTP request directly and gives the JSON response to the assistant. This is the recommended default. | | A curated tool surface that Make owns and Vapi discovers at runtime | **[MCP tool](/tools/mcp)** | Make exposes eligible scenarios as MCP tools and Vapi imports the tools exposed by that server. | | Vapi's `tool-calls` envelope or call, assistant, and artifact context | **[Function tool](/tools/custom-tools)** | Vapi sends its webhook payload to the scenario, which must return the expected Function tool response. | -| Asynchronous execution or server-driven [Live Call Control](/calls/call-features) | **[Function tool](/tools/custom-tools)** | Function tools support asynchronous execution, and their webhook payload includes the call context needed for live control. | +| Asynchronous execution or server-driven [live call control](/calls/call-features) | **[Function tool](/tools/custom-tools)** | Function tools support asynchronous execution, and their webhook payload includes the call context needed for live control. | ## API Request