Media tools

Timeline compose

Put one Sume-hosted still and one Sume-hosted video on screen at the same time. Returns one MP4 shot for Timeline 1.0.

Compose takes one still + one video and returns one MP4 with both on screen at the same time. The server compiles ffmpeg on the worker media runtime (apps/api/src/routes.ts createTimelineV1Compose / submitSumeTimelineComposeJob, and packages/timeline-compiler/src/compose.ts). Callers never send filtergraphs, codecs, or shell fragments.

text

POST /v1/timeline-1.0/compose

This surface has no GET /v1/timeline-1.0/compose/:id. Poll the job envelope:

text

GET /v1/jobs/:id/status
GET /v1/jobs/:id/result

The hosted MCP tool is timeline_compose (packages/mcp-server/src/mcp.ts). Writes need idempotency_key (and mcp:write under OAuth). The flow is timeline_compose → jobs_wait → jobs_result → put video_url into timeline_create video[].

The HTTP field that selects stack or overlay is operation, not mode. mode is the usual async / sync / webhook communication option.

Create a compose

The required fields are operation (stack | overlay), image.url, and video.url. Both URLs must already be this workspace’s media.sume.com artifact or asset. image.url must probe as a still (compose_image_not_still). video.url must probe as video (compose_video_not_video). Import the files first (POST /v1/media-imports). Idempotency-Key is required.

The optional fields are layout, output, video.source_in, and video.duration, plus the usual mode / webhook_url / wait_timeout_seconds communication fields.

The default mode is async. To wait up to 30 seconds for a 200 finished job, pass mode: "sync". Otherwise, you get 202. Then poll the job.

The output length always comes from the video layer (video.duration, or else the rest of the file from source_in). The still stays on screen for the whole clip and can never make the clip longer. The ceiling is 300 s (TIMELINE_COMPOSE_MAX_DURATION_SECONDS). If the duration goes past the end of the source, the job clamps it (compose_duration_clamped_to_source).

bash

curl -X POST https://api.sume.com/v1/timeline-1.0/compose \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: timeline-compose-001" \
  -d '{
    "operation": "stack",
    "image": { "url": "https://media.sume.com/artifacts/artf_demo/banner.png" },
    "video": { "url": "https://media.sume.com/artifacts/artf_demo/talk.mp4" },
    "layout": { "split": "horizontal", "image_region": "top", "ratio": 0.5 },
    "output": { "width": 720, "height": 1280, "fps": 25 }
  }'

A successful submit returns a job (model: sume/timeline-1.0/compose). When the job is result_ready, GET /v1/jobs/:id/result returns kind: timeline_compose with video_url (new artf_) and duration_seconds. Put that MP4 into Timeline 1.0 video[].

Each job uses a flat number of credits. The API rate card lists the current credits. The credits are flat because video.duration can be absent until the worker probes the video. The job uses no provider inference, only worker ffmpeg.

The default output is 1080×1920 MP4 at the frame rate of the video layer. This is the only rate that does not repeat or drop a frame. Set output.width / output.height to the timeline that you assemble into. This prevents a second rescale of the shot. If an explicit output.fps is different from the rate of the clip, the job warns output_fps_resamples_sources.

The output uses the audio of the video. A mute video causes a warning (compose_video_has_no_audio), not a failure. The clip still renders, and the Timeline 1.0 spine supplies the audio at assemble time.

Layout

stack tiles two regions of one frame. The defaults horizontal / top / 0.5 are 반배너 (half-banner: still on top, video below). ratio is the share of the still (0.1–0.9). The video takes the exact remainder.

`layout` key	`stack`	`overlay`
`split`	`horizontal` \| `vertical`	not permitted
`image_region`	`top` \| `bottom` on a horizontal split. `left` \| `right` on a vertical split	not permitted
`ratio`	still’s share of the frame	not permitted
`image_fit` / `video_fit`	`cover` \| `contain` \| `stretch` (`blur` is not a compose fit)	`video_fit` only
`position`	not permitted	`top` \| `center` \| `bottom`
`width_ratio`	not permitted	0.05–1 of width (default 0.9). The plate keeps its aspect
`margin_ratio`	not permitted	0–0.45 of height (default 0.05)

If you mix stack keys with overlay keys, the request returns 400 (compose_stack_takes_no_overlay_layout / compose_overlay_takes_no_stack_layout). If the region does not match the split, the request returns compose_image_region_wrong_axis.

Refusals (stable codes)

Code	When
`compose_image_not_still`	`image.url` is not a still.
`compose_video_not_video`	`video.url` is not a video.
`compose_image_region_wrong_axis`	`left`/`right` on a horizontal split, or `top`/`bottom` on a vertical split.
`compose_stack_takes_no_overlay_layout` / `compose_overlay_takes_no_stack_layout`	Mixed layout vocabulary.
`compose_duration_clamped_to_source`	Warning: `video.duration` went past the end of the file. The job still succeeds.
`compose_video_has_no_audio`	Warning: the video source is mute. The job still succeeds.
`unsupported_media_source` / `source_not_found`	Off-host or dead URL.
Provider / ffmpeg keys	400 — `filtergraph`, `ffmpeg_args`, `codec`, `crf`, and similar keys.

Sume rejects off-host URLs (https://example.com/…) at admit. Import the files first.

Not this surface

Need	Use
Sequence several clips	Timeline 1.0
Concat / split audio into reusable files	Timeline audio
`[start, end)` of one clip	Video trim
Pixel pass (dim / crop)	Video filter
Audio track as a durable wav / mp3	Audio detach

Media tools

Timeline compose

Put one Sume-hosted still and one Sume-hosted video on screen at the same time. Returns one MP4 shot for Timeline 1.0.

text

POST /v1/timeline-1.0/compose

This surface has no GET /v1/timeline-1.0/compose/:id. Poll the job envelope:

text

GET /v1/jobs/:id/status
GET /v1/jobs/:id/result

The HTTP field that selects stack or overlay is operation, not mode. mode is the usual async / sync / webhook communication option.

Create a compose

The optional fields are layout, output, video.source_in, and video.duration, plus the usual mode / webhook_url / wait_timeout_seconds communication fields.

The default mode is async. To wait up to 30 seconds for a 200 finished job, pass mode: "sync". Otherwise, you get 202. Then poll the job.

bash

curl -X POST https://api.sume.com/v1/timeline-1.0/compose \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: timeline-compose-001" \
  -d '{
    "operation": "stack",
    "image": { "url": "https://media.sume.com/artifacts/artf_demo/banner.png" },
    "video": { "url": "https://media.sume.com/artifacts/artf_demo/talk.mp4" },
    "layout": { "split": "horizontal", "image_region": "top", "ratio": 0.5 },
    "output": { "width": 720, "height": 1280, "fps": 25 }
  }'

Layout

`layout` key	`stack`	`overlay`
`split`	`horizontal` \| `vertical`	not permitted
`image_region`	`top` \| `bottom` on a horizontal split. `left` \| `right` on a vertical split	not permitted
`ratio`	still’s share of the frame	not permitted
`image_fit` / `video_fit`	`cover` \| `contain` \| `stretch` (`blur` is not a compose fit)	`video_fit` only
`position`	not permitted	`top` \| `center` \| `bottom`
`width_ratio`	not permitted	0.05–1 of width (default 0.9). The plate keeps its aspect
`margin_ratio`	not permitted	0–0.45 of height (default 0.05)

Refusals (stable codes)

Code	When
`compose_image_not_still`	`image.url` is not a still.
`compose_video_not_video`	`video.url` is not a video.
`compose_image_region_wrong_axis`	`left`/`right` on a horizontal split, or `top`/`bottom` on a vertical split.
`compose_stack_takes_no_overlay_layout` / `compose_overlay_takes_no_stack_layout`	Mixed layout vocabulary.
`compose_duration_clamped_to_source`	Warning: `video.duration` went past the end of the file. The job still succeeds.
`compose_video_has_no_audio`	Warning: the video source is mute. The job still succeeds.
`unsupported_media_source` / `source_not_found`	Off-host or dead URL.
Provider / ffmpeg keys	400 — `filtergraph`, `ffmpeg_args`, `codec`, `crf`, and similar keys.

Sume rejects off-host URLs (https://example.com/…) at admit. Import the files first.

Not this surface

Need	Use
Sequence several clips	Timeline 1.0
Concat / split audio into reusable files	Timeline audio
`[start, end)` of one clip	Video trim
Pixel pass (dim / crop)	Video filter
Audio track as a durable wav / mp3	Audio detach