Text to speech with multiple voices: one voice per line

To make text to speech with multiple voices, split the script by speaker, synthesize each speaker's lines in that speaker's voice, then join the clips in script order into one audio file. A Sume TTS request speaks with exactly one voice, so a dialogue is one POST /v1/tts-1.0/generate per turn plus a Timeline audio concat that joins up to 20 clips with no gaps.

The rules come from the TTS schema in the Sume API reference, the OpenAPI document behind the API reference docs, and the Timeline audio docs, read on 2026-09-27. The price is read from the code behind API pricing. For the same dialogue on camera, see AI avatar conversation video API.

Why does each turn need its own request?

A TTS request takes one voice selector: the avatar_id or avatar_handle of an avatar whose voice is ready, or a voice.id. If you send an avatar and a voice.id together, they must match, or the request fails with 400. The schema has no field for a second voice, so a voice can't change partway through a transcript. Merge consecutive lines by the same speaker into one request to keep the clip count down.

How do I pick a voice for each speaker?

Give each speaker one selector and keep it for the whole script: two avatars with ready voices, two voice.id values you already hold, or one of each. Text to speech API covers listing avatars with a ready voice and the voice.id shapes Sume accepts, and How to create a reusable AI avatar covers making an avatar.

How do I generate the lines?

Send one request per turn, in script order:

• Use the speaker's selector, and the same language and output_format on every line.
• Ask for WAV (pcm_s16le). The Timeline audio docs advise keeping WAV for audio that will be joined again.
• Give each line its own Idempotency-Key, such as dialogue-07-line-02. A retry with the same key and body returns the original job instead of a second paid one.

curl -X POST https://api.sume.com/v1/tts-1.0/generate \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: dialogue-07-line-02" \
  -d '{
    "transcript": "Good question. Let me show you how the refund works.",
    "avatar_handle": "acme_guest",
    "language": "en",
    "output_format": { "container": "wav", "encoding": "pcm_s16le", "sample_rate": 44100 }
  }'

How do I join the lines into one audio file?

When every line's job has completed, send their audio URLs in script order to POST /v1/timeline-1.0/audio with operation: "concat" and an Idempotency-Key, which this route requires. Poll GET /v1/jobs/:id/status, then read GET /v1/jobs/:id/result: it has one audio_url and segments[] with the start offset of each part, which is where each line begins, handy for captions or a speaker-labeled transcript.

Past 20 lines, join in groups of up to 20, then join the group files: each concat returns a durable media.sume.com file, within the 1,800-second output cap.

curl -X POST https://api.sume.com/v1/timeline-1.0/audio \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: dialogue-07-join" \
  -d '{
    "operation": "concat",
    "parts": [
      { "url": "https://media.sume.com/artifacts/artf_demo/line-01-host.wav" },
      { "url": "https://media.sume.com/artifacts/artf_demo/line-02-guest.wav" },
      { "url": "https://media.sume.com/artifacts/artf_demo/line-03-host.wav" }
    ]
  }'

Can speakers overlap, or pause between lines?

Not in the join. Timeline audio has two operations, concat and split, and concat plays the parts one after another with no silence at the seams, so voices never overlap or crossfade. A pause between lines is whatever silence the clips already carry. On Sume, music under the voices takes a Timeline 1.0 render, which produces an MP4, as in Add background music to a video.

What does a multi-voice track cost?

Every line is billed on its own characters at $0.0475 per 1,000 characters, spaces and punctuation included, plus a 5.5% agent fee by default. Each concat is billed per job, at the rate on the Timeline audio page, which says to confirm it in GET /v1/catalog.