How to assemble a long-form video with the Timeline 1.0 API

Sume Timeline 1.0 assembles a long-form video from a declarative document: one audio spine plus ordered video[] slots, rendered by POST /v1/timeline-1.0/render into one MP4. The server compiles ffmpeg itself, so callers never send filtergraphs, codecs, or shell fragments.

Every detail below is from the Timeline 1.0 docs, read on 2026-09-25.

What does Timeline 1.0 do, and what does it leave to other tools?

Timeline 1.0 is the only public assembly surface. Assembly means sequencing, transitions, and the audio spine. The docs send other needs to separate tools, covered in Trim, filter, or detach audio and Timeline compose and timeline audio:

• A [start, end) range of one clip: video trim.
• An audio track as a durable wav or mp3: audio detach.
• A dim, crop, or other pixel pass, which is not a render option: video filter.
• A still and a video on screen at once: timeline compose. Drop that MP4 into video[].
• A reusable merged audio file: timeline audio. Sliced voiceover needed only inside this render belongs on audio.parts[].

How do I render a timeline?

Send audio.duration_seconds (1–1800), either audio.url or audio.parts[] (unless audio.mode is "silence"), and 1–200 video[] slots. Every URL must already be your workspace's media.sume.com artifact or asset. Off-host URLs such as https://example.com/… are rejected at admit, so import first with POST /v1/media-imports. Idempotency-Key is required.

The default mode is async. Pass mode: "sync" to wait up to 30 seconds for a 200 finished job; otherwise you get 202 and poll. There is no GET /v1/timeline-1.0/:id: poll GET /v1/jobs/:id/status and GET /v1/jobs/:id/result. POST /v1/models/sume/timeline-1.0/runs takes the same body.

On the hosted MCP server the flow is timeline_create, then jobs_wait, then timeline_get. Writes need idempotency_key, and mcp:write under OAuth; see Connect Claude Code, Cursor, or Codex to Sume.

curl -X POST https://api.sume.com/v1/timeline-1.0/render \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: timeline-001" \
  -d '{
    "audio": {
      "url": "https://media.sume.com/artifacts/artf_demo/voice.wav",
      "duration_seconds": 24
    },
    "video": [
      { "source_url": "https://media.sume.com/artifacts/artf_demo/intro.mp4", "start": 0, "duration": 8 },
      {
        "source_url": "https://media.sume.com/artifacts/artf_demo/talk.mp4",
        "start": 8,
        "duration": 16,
        "transition": { "type": "fade", "duration": 0.25 }
      }
    ]
  }'

Which fields shape the program?

Declared starts are authoritative: the compiler compensates for the xfade and never pre-shifts a slot.

How do I check a timeline before it bills?

Call POST /v1/timeline-1.0/plan, the unbilled compile preflight. It runs the schema, the Sume-host URL checks, and the compiler, and returns object: timeline_plan with duration_seconds, segment_count, billable_minutes, estimated_cost_usd_micros, and a filtergraph_summary.

A plan does not create a job, reserve credits, or download media, and it does not need an Idempotency-Key. It cannot predict short-source pad or loop warnings.

What comes back, and how is it billed?

A successful submit returns a job with type: timeline_render and model: sume/timeline-1.0. When it is result_ready, GET /v1/jobs/:id/result returns kind: timeline_render with video_url, duration_seconds, segment_count, billable_minutes, and optional warnings[]. Soft warnings (padded or looped short sources, snapped transitions, ignored still motion) are not failures.

Default output is a 1080×1920 MP4. An omitted output.fps renders at the rate the sources already run at, and at 30 only when nothing has one. A rate that differs from a source's repeats or drops a frame every few frames, and is reported as output_fps_resamples_sources.

The public rate is charged per ceil(output minute), and the reserve is ceil(audio.duration_seconds / 60) minutes. There is no provider inference, only worker ffmpeg. The rate is on API pricing; the docs say to confirm it live in GET /v1/catalog.

Why was my timeline refused?

Refusals carry stable codes. Among them:

• timeline_must_start_at_zero: video[0].start is not 0.
• transition_on_first_segment: a transition on video[0].
• invalid_segment_timing / segment_overlap: starts do not increase, or slots overlap past the xfade.
• too_many_chained_transitions: more than 8 adjacent fades. Insert a hard cut.
• audio_parts_shorter_than_duration: declared part lengths sum to less than duration_seconds.
• render_strategy_unsafe: strategy: "single" with more than 12 slots.
• unsupported_media_source / source_not_found: an off-host or dead URL.
• 400 for provider or ffmpeg keys such as model, filtergraph, ffmpeg_args, codec, or crf.