AI avatar language tutor: slow, captioned lesson videos

An AI avatar language tutor is either a live conversation partner or a recorded presenter; the recorded kind is a talking avatar that says each lesson line in the target language, slowly, with the words on screen. With Sume you build the recorded kind: TTS 1.0 speaks each line in the target language at a slower speed, VEED Fabric 1.0 lip syncs a tutor avatar to it, and captions or word timings put the text on screen. Every clip is a job that renders first and plays back later.

Facts come from Sume's Models overview and Video captions docs and the TTS and Fabric schemas in the Sume API reference, read on 2026-09-27; anything called current behavior is read from Sume's code. For audio-only drills without a face, see Text to speech for language learning.

Can the tutor speak the language I teach?

For English, yes; for other languages, through TTS after a quick test. The one-call Avatar 1.0 talking video speaks English only in current code and has no speed setting, so it only suits English lessons at the pace generation picks; any other language, or slower English, takes TTS 1.0 plus Fabric. The TTS schema gives ko, ja, and en as example codes and publishes no full list, so synthesize one short line in your language and listen before you record a course.

Set language on every non-English transcript; left out, it defaults to English. In current code, language: "ko" needs at least one Hangul syllable in the transcript, or the request fails with 400 tts_language_script_mismatch.

How do I make the tutor speak slowly?

Set generation_config.speed, a multiplier from 0.6 to 1.5, on the TTS request. For the voice, send the tutor avatar's avatar_handle, whose own voice is cloned as English in current code, or a voice made for that language in Assets → Voices, sent as voice.id.

Add timestamps.words and sentence segmentation: with WAV output, each sentence comes back as its own audio_url, ready to become its own clip, and every word gets a start and end time. Text to speech for language learning covers that request in detail.

curl -X POST https://api.sume.com/v1/tts-1.0/generate \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: lesson-3-lines-001" \
  -d '{
    "transcript": "¿Dónde está la estación? Está a la derecha.",
    "voice": { "id": "voi_…" },
    "language": "es",
    "generation_config": { "speed": 0.75 },
    "output_format": { "container": "wav", "encoding": "pcm_s16le", "sample_rate": 44100 },
    "timestamps": { "words": true },
    "segmentation": { "mode": "sentence" }
  }'

How do I put the tutor on camera?

Send each audio file to POST /v1/veed/fabric-1.0 with the tutor avatar's avatar_handle and the audio's duration_seconds. Sume resolves the handle to the avatar's identity still, so every lesson starts from the same face. One job takes 1–300 seconds of Sume-hosted audio, at most 10 MB.

How do I show the words on screen?

• Burn captions onto each finished clip with POST /v1/video-captions. Pass the lesson text as script_text so the burned words follow your spelling; the timing still comes from speech-to-text.
• Caption clip by clip: in current code, the caption job refuses a source over 60 seconds or one without an audio stream.
• The documented caption styles draw in Latin or Hangul faces. For Korean, pick a Hangul style such as black-outline; the Latin styles slam, punch, and tiktok-green refuse Korean copy with a 400 (Korean captions).
• To highlight each word in your own player instead, use the TTS words timings, as in text to speech with highlighted words.

What does a lesson clip cost?

Each step bills on its own, plus a 5.5% agent fee by default. Fabric counts audio seconds rounded up, so a 20-second lesson clip at 720p comes to $3.75 before the fee.