How to add a voiceover to a video and keep its sound

To add a voiceover to a video, record or generate the narration, lay it over the picture, turn the video's own sound down underneath it, and export a new file. With Sume, speak the script with TTS 1.0, detach the clip's original sound, and render the clip in Timeline 1.0 with that sound turned down as the audio spine and the narration mixed over it as the soundtrack.

The facts come from the Timeline 1.0, Audio detach, and Models docs and the TTS and Timeline schemas in the Sume API reference, read on 2026-09-27. Anything described as current behavior is read from Sume's code.

How do I make the narration?

Generate it with TTS 1.0: send the script to POST /v1/tts-1.0/generate as transcript, up to 20,000 characters, with a voice: the avatar_id or avatar_handle of an avatar whose voice is ready, or a voice.id. Set language for a non-English script; omitted, it defaults to English. The output is an mp3 by default, and speech longer than 1,200 seconds fails with tts_duration_exceeded. Text to speech API covers voices, word timings, and the other fields.

How do I put the voiceover over the video's own sound?

The video must already be your workspace's media.sume.com file, such as an earlier Sume job's output; which URLs each endpoint accepts explains the rule. The TTS narration is already there: completed TTS results carry mirrored audio artifacts.

In the current compiler, a render's sound comes only from its audio spine and an optional soundtrack, never from the clip itself. So detach the clip's sound first with POST /v1/audio-detach: its default wav is the file a spine wants, and the result reports duration_seconds. Then render:

• Leave soundtrack.loop off. It repeats the bed until the spine ends, so it would replay the narration. Without it, a shorter narration simply stops, and the render warns soundtrack_shorter_than_spine.
• A clip with no audio track can't be detached (detach_source_has_no_audio). Then make the narration the spine, set audio.duration_seconds to its length, and leave soundtrack out.
• In current code the mix ends with the spine, so a narration longer than the video is cut off at the end. The soundtrack has no start-time field, so the voice begins at 0.
• Set output to the clip's size: the default frame is 1080×1920. The default mode is async, so poll GET /v1/jobs/:id/status, then read the new MP4's video_url from GET /v1/jobs/:id/result.

curl -X POST https://api.sume.com/v1/timeline-1.0/render \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: voiceover-clip-001" \
  -d '{
    "audio": {
      "url": "https://media.sume.com/artifacts/artf_demo/original.wav",
      "duration_seconds": 45,
      "gain_db": -12
    },
    "soundtrack": {
      "url": "https://media.sume.com/artifacts/artf_demo/narration.mp3",
      "gain_db": 0
    },
    "output": { "width": 1920, "height": 1080 },
    "video": [
      { "source_url": "https://media.sume.com/artifacts/artf_demo/clip.mp4", "start": 0, "duration": 45 }
    ]
  }'

Can the original sound dip only while the voice speaks?

Yes, by swapping the roles: the narration becomes the spine (audio.url) and the detached original sound the soundtrack, at gain_db 0 so it isn't turned down to the −16 dB default, with duck_db from 0 to 20 as the target dip while the spine is speaking. The output then lasts as long as the narration: set audio.duration_seconds to its length, for example from the last word's end when you ask TTS for timestamps.words, and end the clip's slot there. Add background music to a video explains ducking in detail.

Will the voiceover match the lips on screen?

No. A voiceover changes only the sound, and Sume's model docs say video models do not lip-sync to generated TTS or to a later voice-over. Narrate over footage without on-camera speech; to make a new talking clip from a still image and audio, see Lip sync API.

What does adding a voiceover cost?

Three jobs bill: TTS at $0.0475 per 1,000 characters, where spaces and punctuation count; the detach at a flat per-job rate listed in GET /v1/catalog; and the render at $0.10 per output minute, reserved as ceil(audio.duration_seconds / 60) minutes. The TTS and render rates are on API pricing, and all three are billed plus a 5.5% agent fee by default.