Lip sync AI for long videos: past the 5-minute clip limit

To lip sync a long video with AI, split the speech into parts that each fit one lip-sync job, animate the same face for every part, and lay the clips back over the full audio. On Sume, one VEED Fabric 1.0 job takes at most 300 seconds of audio in a file of at most 10 MB, and one Timeline 1.0 render joins the clips into a video of up to 30 minutes.

Sume facts come from the Timeline audio and Timeline 1.0 docs and the Fabric and TTS request schemas in the Sume API reference, read on 2026-09-27. For one clip of 300 seconds or less, see Lip sync API: turn an image and audio into a talking clip.

Why does one lip-sync job stop at 5 minutes?

POST /v1/veed/fabric-1.0 takes a duration_seconds of 1 to 300, and its audio_url must be on the Sume media host and at most 10 MB. For WAV, the byte cap arrives first:

• 16-bit WAV stores 2 bytes per sample per channel: 88,200 bytes a second at 44.1 kHz and 96,000 at 48 kHz, in mono. So 10 MB holds under two minutes of mono WAV, and stereo halves that.
• TTS's default MP3, 44.1 kHz at 128 kbps, is 16,000 bytes per second, so a 300-second MP3 fits in under 5 MB.
• Even so, Sume's Timeline audio docs say to keep WAV when the audio drives lip sync, since MP3 re-adds priming padding at every edge. Today Sume's TTS writes its WAV in mono, Timeline audio writes each part at 48 kHz, and Fabric refuses a part over the cap at submit with 400 audio_too_large. So plan parts of about 100 seconds.

How do I split the speech into parts?

Make the narration once, then cut it at pauses with Timeline audio. The split keeps WAV by default, and each segment in the result gets its own audio_url. Each range is { start, end }, and a range without end runs to the end of the file:

• Generate the whole script with POST /v1/tts-1.0/generate, choosing the voice with a ready avatar's avatar_handle. Ask for WAV with output_format: { "container": "wav", "encoding": "pcm_s16le", "sample_rate": 44100 } and for timestamps: { "words": true }, whose word timings show where the pauses are.
• One TTS request takes up to 20,000 characters, and audio longer than 1,200 seconds fails with tts_duration_exceeded. For more, generate several files and join them with a Timeline audio concat.
• Send the file to POST /v1/timeline-1.0/audio with operation: "split" and 1–20 ranges, each ending at a pause and under the byte cap.

curl -X POST https://api.sume.com/v1/timeline-1.0/audio \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: talk-split-001" \
  -d '{
    "operation": "split",
    "url": "https://media.sume.com/artifacts/artf_demo/talk.wav",
    "ranges": [
      { "start": 0, "end": 98.6 },
      { "start": 98.6, "end": 196.9 },
      { "start": 196.9, "end": 294.2 },
      { "start": 294.2, "end": 391.5 }
    ]
  }'

How do I lip sync each part and join the clips?

Lip sync every part to the same face, then render the clips over the unsplit file:

• Send each part's audio_url to POST /v1/veed/fabric-1.0 with the part's measured length as duration_seconds and the same visual source every time: one image_url still or one ready avatar's avatar_handle, never both.
• Render once with POST /v1/timeline-1.0/render: the whole WAV as audio.url, its length as audio.duration_seconds, and one video[] slot per clip whose start is the part's range start and whose duration is the part's length.
• video[0].start must be 0, later starts must increase, and the slots may trail the spine by at most 0.5 seconds.
• Timeline renders 1080×1920 by default. If your still isn't vertical, set output.width and output.height to its shape.
• Cut at pauses, so any change in the face at a seam falls where nobody is speaking.

How much does a long lip-sync video cost?

Fabric is billed per audio second, rounded up per job: $0.1875 per audio second (720p), with the 480p rate on API pricing. A 10-minute talk is 600 audio seconds, about $112.50 at 720p. The narration is $0.0475 per 1,000 characters, the split $0.01 per job, and the render is listed at $0.10 per output minute. Each is plus a 5.5% agent fee by default.

What are the limits?

Each step has its own cap, and two things are out of reach:

• The audio must already be on Sume's media host, like a TTS result. The asset upload routes are hidden from the public API, so a local recording can't go in directly.
• Fabric animates a still image or an avatar's still and takes no video input, so it can't re-sync the mouth in footage you already have; dubbing an avatar video rebuilds it from a still instead.