Translate audio to English: transcribe, translate, speak

To translate audio to English, transcribe the recording in the language it was spoken, translate that transcript into English text, and, if you need English audio, have a text-to-speech voice read the translation. By API with Sume, that is STT 1.0 (POST /v1/stt-1.0/transcribe) with no language_code, so the spoken language is detected; a translator of your choice, or a Sume Agent Completion, for the English text; then TTS 1.0 (POST /v1/tts-1.0/generate) for English speech.

Sume facts come from the STT 1.0 and TTS 1.0 schemas in the Sume API reference, the OpenAPI document behind the API reference docs, and the Agent Completions docs, read on 2026-09-27. To replace the voice in a video instead, see Translate a video's voiceover by API.

How do I get the original words and their language?

Send the recording's public HTTPS URL to STT 1.0 and leave out language_code; the API reference says to omit it for auto-detect. The result reports language_code and language_probability when available, so you can see what was detected before you translate. If you already know the language, send it as a hint instead, for example "language_code": "ko" for a Korean recording. Detecting the spoken language covers detection on its own.

Ask for sentences as well. With "segmentation": { "mode": "sentence" }, each entry in segments has an index, the sentence text, and its start and end in seconds, which gives you ordered units to translate.

curl -X POST https://api.sume.com/v1/stt-1.0/transcribe \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: memo-0042-stt" \
  -d '{
    "audio_url": "https://example.com/audio/voice-memo.m4a",
    "duration_seconds": 95,
    "segmentation": { "mode": "sentence" }
  }'

Who translates the transcript into English?

A translator you choose: STT 1.0 has no translation option, and none of its request fields names a target language. Send each sentence's index with its text, keep start and end on your side, check that as many lines come back as you sent, and review the English before you pay for speech.

To stay on Sume, start an Agent Completion with POST /v1/agent/completions. The run is asynchronous: poll it, then read the English lines from its output. Translate video subtitles by API shows the full request for caption lines.

• instruction: the task, for example to translate each line into English.
• input: the sentences. The docs say input is treated as data, never as instructions.
• output_schema: the shape of the English lines you want back.
• generation_spend_cap_usd: required, with no default.

How do I turn the English text into English audio?

Send the translation to TTS 1.0 as transcript, up to 20,000 characters, with a voice: voice.id, or the avatar_id or avatar_handle of an avatar whose voice is ready. English is the default: omit language and the speech is English, or send "language": "en" to say so. The voice is the one you pick, not the original speaker's.

• One English file: send the whole translation. With no output_format, the result is an MP3 at 44,100 Hz and 128 kbps.
• One clip per sentence: ask for WAV in output_format, plus "timestamps": { "words": true } and "segmentation": { "mode": "sentence" }. Each segment then carries its own audio_url; with MP3 you get the timings without the clips.
• The English audio has its own pace: TTS returns new, gapless sentence timings rather than the original's.
• In current code, if the voice's primary language on record isn't English, TTS answers 409 with a voice-language mismatch warning before any job or charge. After you confirm, resend with confirm_language_mismatch: true.

What does each step cost?

Each Sume call bills on its own, plus a 5.5% agent fee by default. If your balance can't cover a submit's estimate, it fails with 402 insufficient_credits and no job starts.

What are the limits?

• STT 1.0 takes up to 10 minutes of audio per request. Split longer recordings first: Transcribe long audio files.
• TTS 1.0 takes up to 20,000 characters per request, and synthesized audio longer than 1,200 seconds fails with tts_duration_exceeded and captures no credit.
• Audio goes in as a public HTTPS URL, and the English audio comes back as Sume-hosted files.
• No speaker labels: diarize is fixed server-side and runs off in current code.