> ## Documentation Index
> Fetch the complete documentation index at: https://docs.corti.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Add text to speech to your conversational agent

> Learn how to convert Corti agent responses into speech with a third-party text to speech provider

<Warning>
  **Beta.** This is a starting pattern for pairing a Corti agent with a third-party text to speech provider, not a Corti-hosted feature. Corti does not host, proxy, or bill for text to speech today: your integration connects to the provider you choose directly. Latency, voice quality, and streaming behavior are entirely provider-dependent. [Reach out](mailto:help@corti.ai) to discuss your use case or share feedback.
</Warning>

<Note>
  This guide assumes a working Corti agent integration is already in place, for example the [On-Demand Agent](/stt/guides/agent-on-demand), [Wake-Command Agent](/stt/guides/agent-wake-command), or [Conversational Agent](/stt/guides/agent-conversational) pattern. It covers only the last step: turning that agent's text response into audio.
</Note>

An agent built on the [Agentic Framework](/agentic/overview) returns a text response from `messageSend`. To close the loop into a voice interface, that text needs to become speech the user can hear. This guide walks through pairing your agent with an external text to speech provider: choosing one, sending it the agent's response, and playing back the result.

***

## Why this is a pairing, not a Corti endpoint

Text to speech models and hosting are widely available and improve quickly, including strong open-source options. Rather than adding a Corti-hosted proxy in front of a fast-moving space, we recommend connecting your agent integration directly to a text to speech provider. That gives you direct control over latency, voice quality, language coverage, and cost, and lets you change providers without waiting on a Corti release.

***

## Choose a text to speech provider

The right provider depends on your priorities. For example, providers with a streaming API (WebSocket or server-sent events) return the first audio chunk well before the full response finishes synthesizing. If low time-to-first-audio matters for your use case, prefer a provider's streaming endpoint over its plain REST endpoint.

A few vendors to consider include the following:

| Provider                                                                                 | Good fit when                                                         |
| ---------------------------------------------------------------------------------------- | --------------------------------------------------------------------- |
| [Cartesia](https://cartesia.ai)                                                          | Low-latency, conversational voice agents; native WebSocket streaming  |
| [ElevenLabs](https://elevenlabs.io)                                                      | Voice realism and broad multilingual coverage are the priority        |
| [OpenAI text to speech](https://platform.openai.com/docs/guides/text-to-speech)          | Simple, lower-volume use cases already on the OpenAI platform         |
| Self-hosted / open source (e.g. [Chatterbox](https://github.com/resemble-ai/chatterbox)) | Data residency or on-premises requirements rule out a hosted provider |

<Warning>
  **Verify compliance directly with the provider.** The text you send for speech synthesis, including any clinical content in an agent's response, is processed by that provider under their terms, not Corti's. Confirm their HIPAA, GDPR, or other data-handling and retention commitments (including any BAA) before sending real patient data through them.
</Warning>

***

## Prompt the agent for speakable text

An agent tuned for a text-based chat interface will often return markdown formatting, bullet lists, or headers, none of which sound right read aloud. Add explicit instructions to the agent's `systemPrompt` so its responses are written to be heard, not read:

```ts title="Speakable-response system prompt" theme={null}
const VOICE_AGENT_PROMPT = `You are a voice assistant. Your responses are converted to speech, so write for listening, not reading.
Use short, complete sentences and plain punctuation.
Never use markdown, bullet points, headers, or emoji.
Spell out symbols instead of using them (say "and" instead of "&", "at" instead of "@").
Keep responses concise: a user hears audio at a fixed pace and cannot skim.`;
```

<Tip>
  Treat this as a starting point, not a final prompt. Test it against real responses from your agent and refine the wording until playback sounds natural for your use case.
</Tip>

***

## Send the agent's response to your provider

Once `messageSend` resolves, extract the text from the response and hand it to your chosen provider's text to speech endpoint. The exact request shape is provider-specific: consult that provider's docs for authentication, voice selection, and output format. The pattern is the same regardless of provider:

```ts title="JavaScript" theme={null}
// Note this example shows the v2 A2A message shape directly 
const response = await client.agents.messageSend(agentId, {
  message: {
    role: "ROLE_USER",
    parts: [{ text: userUtterance }],
    messageId: crypto.randomUUID(),
  },
});

// A quick conversational turn returns `message` directly. A long-running task returns `task`, with the reply as the last ROLE_AGENT message in `task.history`.
const replyMessage =
  response.message ??
  [...(response.task?.history ?? [])].reverse().find((m) => m.role === "ROLE_AGENT");
const replyText = replyMessage?.parts.find((part) => "text" in part)?.text;

if (replyText) {
  await speak(replyText);
}
```

`speak` is your own function, not part of the Corti API, that calls the chosen provider's text to speech endpoint with `replyText` and plays back the resulting audio, for example through the [Web Audio API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API) or an `<audio>` element fed from a `MediaSource`. Match the playback method to the provider's output format: compressed formats like `mp3` or `opus` play fine through an `<audio>` element, but raw PCM output needs the Web Audio API instead.

<Tip>
  Guard against empty or partial responses the same way you would for any agent call: if `replyText` is empty, skip the text to speech request rather than sending blank input to the provider.
</Tip>

***

## Account for end-to-end latency

Time to first audio in this pattern is the sum of three stages: speech to text on the user's utterance, the agent's `messageSend` response time, and the provider's text to speech synthesis time. Each stage adds up, so:

* Prefer providers with a streaming synthesis API to shorten the third stage.
* Keep agent responses concise; shorter text reaches the provider's synthesis step sooner and produces less audio to wait for.
* Measure the full pipeline, not just the text to speech call in isolation. A fast provider can still feel slow if it sits behind a slow agent response.

***

## Handle interruptions

Some text to speech providers offer their own barge-in control (for example, a streaming `Interrupt` or flush/cancel message that stops synthesis server-side). That's worth using when it's available, but not every provider offers it, and it only stops the provider's audio, not your local playback. Detecting the interruption itself doesn't have to depend on the provider at all: your existing Corti speech to text session can do it, so the same approach works regardless of which provider you paired with above.

The core idea: don't stop listening while the agent's response plays back. Keep the [dictation](/quickstart/dictation) session running through playback, and treat a new incoming transcript as the user talking over the agent.

<Steps>
  <Step title="Keep the microphone hot during playback">
    Don't pause or close the dictation session while `speak` is running. Track whether playback is active with a simple flag so you can distinguish "the agent is talking" from "the user is talking."
  </Step>

  <Step title="Turn on echo cancellation">
    With the speaker and microphone active at the same time, the agent's own audio can loop back into the mic and be misread as user speech. Set `echoCancellation: true` on the capture track, the same setting Corti recommends for [ambient conversation capture](/stt/audio#ambient-conversation) whenever a device's own speaker output could reach its microphone.
  </Step>

  <Step title="Treat an interim transcript during playback as a barge-in">
    Listen for the dictation session's `transcript` event as usual. If an interim result arrives while `isSpeaking` is `true`, the user has started talking over the agent: stop audio playback immediately and cancel the in-flight text to speech request if your provider call is still streaming or generating.

    ```ts title="JavaScript" theme={null}
    let isSpeaking = false;
    let ttsAbortController: AbortController | null = null;

    dictation.addEventListener("transcript", (e: CustomEvent) => {
      const { text, isFinal } = e.detail.data;
      if (isSpeaking && text.trim()) {
        interruptAgent();
      }
      if (!isFinal) {
        handleInterim(text);
      } else {
        handleFinal(text);
      }
    });

    function interruptAgent() {
      isSpeaking = false;
      stopPlayback(); // pause the <audio> element or stop the AudioBufferSourceNode
      ttsAbortController?.abort(); // cancel the provider request if still in flight
      ttsAbortController = null;
    }
    ```
  </Step>

  <Step title="Feed the new turn through your normal flow">
    Once the barge-in is handled, the transcript that triggered it continues through the same debounce and turn-finalization logic from the [Conversational Agent](/stt/guides/agent-conversational#debounce-finals-and-flush-the-turn) guide, so it's sent to the agent as the next turn like any other utterance.
  </Step>
</Steps>

<Tip>
  This method stops playback the instant it detects speech, but it doesn't know how much of the interrupted response the user actually heard. If you need that (for example, to tell the agent what context to assume the user has), check whether your provider reports it: some streaming APIs return spoken-versus-remaining text on interrupt.
</Tip>

***

## Fall back gracefully on provider failures

The text to speech provider is now a dependency your integration doesn't control. Treat it as one that can fail independently of the agent call: wrap it with a timeout and a fallback so a slow or failed provider request doesn't stall the conversation or leave the user with nothing.

```ts title="JavaScript" theme={null}
async function speakOrFallback(replyText: string) {
  try {
    await speak(replyText);
  } catch (err) {
    console.error("Text to speech request failed, falling back to text", err);
    showTextResponse(replyText); // display the reply in the UI instead
  }
}
```

<Tip>
  Set a timeout on the provider request, an `AbortController` with a budget of a few seconds works for most REST calls, so an unresponsive provider doesn't stall the turn indefinitely.
</Tip>

***

## What this pattern doesn't cover

This is a starting point, not a complete voice interface. It does not handle:

* **Long-running tasks.** If the agent needs to do substantial work before responding, this pattern has nothing to say back to the user in the meantime.
* **SSML or fine-grained prosody control.** Handled entirely by your chosen provider, if it supports it.

***

## Next steps

<CardGroup cols={2}>
  <Card title="On-Demand Agent" icon="bot" href="/stt/guides/agent-on-demand">
    The agent lifecycle and message API, for a single isolated pass.
  </Card>

  <Card title="Wake-Command Agent" icon="mic" href="/stt/guides/agent-wake-command">
    Gate the agent behind a spoken wake phrase and hold a multi-turn thread.
  </Card>

  <Card title="Conversational Agent" icon="audio-lines" href="/stt/guides/agent-conversational">
    Always-on voice agent, no wake phrase, every utterance forwarded with speculative prefetch.
  </Card>

  <Card title="Context & Memory" icon="brain" href="/agentic/context-memory">
    How conversation threads and memory work in the Agentic Framework.
  </Card>
</CardGroup>

<Note>Please [contact us](mailto:help@corti.ai) for help or questions.</Note>