How to make a photo talk with AI: voice, then lip sync

To make a photo talk, turn the words into speech first, then run a lip sync model that moves the face in the photo to match that audio. On Sume that is two API jobs: TTS 1.0 speaks your text in a ready voice, and VEED Fabric 1.0 turns the photo plus that speech into a talking clip of up to 300 seconds.

Facts come from Sume's Models overview and the TTS and Fabric request schemas in the Sume API reference, read on 2026-09-27. Anything described as current behavior is read from Sume's code, and prices from the code behind API pricing. If your audio is already on Sume, the lip sync API guide covers that half alone.

Can AI make a photo talk from text alone?

Yes. On Sume it takes two jobs, not one: Sume's models guide builds every on-camera speaking shot from a still plus TTS on Fabric, and it says video models do not lip-sync to generated TTS or to a later voice-over. An image-to-video clip with narration laid underneath will not move the lips to the words; the lip sync job does that.

Where does the voice come from?

TTS 1.0 speaks in a voice your workspace already has, from one of two places:

• A ready avatar's voice. Send the avatar's avatar_id or avatar_handle; it works once that avatar's voice.status is ready in GET /v1/avatar-1.0/avatars. Creating an avatar costs $0.95 per avatar.
• A voice from the Sume app. Today, in Assets → Voices, you upload or record audio to clone a voice, or describe a person and Sume generates one. Copy ID gives its id, voi_ plus 32 hex characters, which you send as voice.id. The voice itself is made in the app; the API takes its id. AI voiceover in your own voice covers cloning.

How do I turn the text into speech?

Send the words as transcript, up to 20,000 characters, to POST /v1/tts-1.0/generate, and set language for any text that is not English; omitted, it defaults to English, with Korean or Japanese inferred from a Hangul- or kana-only transcript as a fallback.

Ask for word timestamps too: in the current code, the result then reports the audio's duration_seconds next to its audio_url, a file on the Sume media host, and the lip sync step needs both. The default output is MP3 at 44,100 Hz and 128 kbps.

curl -X POST https://api.sume.com/v1/tts-1.0/generate \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: talking-photo-tts-001" \
  -d '{
    "transcript": "Hello and welcome to the spring open house. Doors open at ten.",
    "avatar_handle": "product_host",
    "language": "en",
    "timestamps": { "words": true }
  }'

How do I lip sync the photo to the speech?

Send POST /v1/veed/fabric-1.0 with the photo as image_url (a public HTTPS still), the TTS file as audio_url, and the audio's measured length as duration_seconds, which also reserves the cost at submit. To use an avatar's face instead of a photo, send its avatar_handle in place of image_url.

Add an Idempotency-Key so a retry returns the original job instead of starting a second one, then poll the job and read the talking clip from the Sume-hosted artifacts in the result. The lip sync API guide covers the other fields, such as resolution.

curl -X POST https://api.sume.com/v1/veed/fabric-1.0 \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: talking-photo-001" \
  -d '{
    "image_url": "https://example.com/portrait.jpg",
    "audio_url": "https://media.sume.com/artifacts/example/open-house.mp3",
    "duration_seconds": 4.6,
    "resolution": "720p"
  }'

How long can the clip be, and what does it cost?

Each step bills on its own, plus a 5.5% agent fee by default. Fabric counts audio seconds rounded up, so a 30-second talking clip at 720p comes to $5.625 before the fee. TTS counts every character, spaces and punctuation included.

What stops a photo from talking?

These requests are refused:

• Audio hosted anywhere else. In the current code, an audio_url off the Sume media host is refused with 400 unsupported_audio_source. A TTS result qualifies; a recording on your own server does not, and signed upload URLs are not part of the public API.
• A file over 10 MB. The current code refuses it with 400 audio_too_large, and the suggested next action is to shorten the segment or use MP3.
• Speech longer than 300 seconds. Split it and lip sync each part as its own clip.
• Both a photo and an avatar, or neither. Fabric takes exactly one visual source.