Video inspect API: probe a clip, sample stills, get a transcript

Sume's video inspect API (POST /v1/video-inspect) reads one Sume-hosted clip and returns probe facts, sampled stills as durable image files, and, when you ask for it, a speech-to-text transcript. Probe and stills are unbilled; only the transcript is billed, and inspect never re-encodes the source or produces an MP4.

Every detail below is from the Video inspect docs, read on 2026-09-25.

How do I inspect a clip?

Send video_url, which must be your workspace's media.sume.com artifact or asset. There is no open-internet fetch: off-host URLs are rejected at admit, so import first with POST /v1/media-imports (see media inputs and outputs). Idempotency-Key is required.

The default mode is sync. The handler waits up to 30 seconds and answers 200 with the finished inspect, or 202 with the queued job to poll. The response carries request_id, which is the video_inspect_id and also the job id; read it later with GET /v1/video-inspect/:id.

On the hosted MCP server the tool is video_inspect. There is no GET wrapper, so when the submit is 202, poll with jobs_wait then jobs_result. Writes need idempotency_key, and mcp:write under OAuth.

curl -X POST https://api.sume.com/v1/video-inspect \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: video-inspect-001" \
  -d '{
    "video_url": "https://media.sume.com/artifacts/artf_demo/talk.mp4"
  }'

What does an inspect return?

When ready, the resource carries these fields. It does not return typed scenes: inspect is probe facts, stills, and optional speech-to-text.

• probe: facts about the file, including probe.has_audio.
• frames[{t,url,width,height}]: stills as durable media.sume.com image artifacts.
• transcript, when requested: text, words[], optional sentence segments[], and audio_url.
• warnings[].

How do I choose which stills come back?

The frames field picks the program. An object must pass exactly one of at[] or fps, and may also set format (jpeg by default, or png) and max_edge (64–2160, default 768). Caps: the source can be up to 1800 s, with 24 stills per call.

seek: "precise", the default, decodes to the exact instant. seek: "fast" snaps each still to the keyframe at or before its instant and skips the decode: earlier by up to one GOP (roughly 0–5 s on typical sources), never later. Use fast for skimming; keep precise when the timestamp must match.

How do I get a transcript, and what does it cost?

Set transcribe: true and Sume STT 1.0 runs on the clip's audio. Only this half reserves; probe and stills stay unbilled. The transcript is billed per audio minute at the rate on API pricing; the docs say to confirm it live in GET /v1/catalog.

• Omit duration_seconds and 1 minute is reserved. The maximum hint is 600 s.
• language_code (for example en or ko) is a hint. Omit it for auto-detect.
• segmentation.mode: "sentence" also returns gapless sentence segments[], shaped like caption lines, with an optional silence_split_seconds of 0.2–3.
• language_code, segmentation, or duration_seconds without transcribe: true returns 400 video_inspect_transcribe_required.
• A silent clip fails with inspect_source_has_no_audio. Check probe.has_audio first; a frames: false inspect is enough.

Why was my inspect refused?

• ffmpeg_fields_rejected: the request sent vf, filter, ffmpeg, cmd, codec, crf, or similar. The server compiles ffmpeg.
• video_inspect_frames_program_conflict: both frames.at[] and frames.fps.
• video_inspect_frames_program_required: a frames object with neither.
• frame_time_out_of_range: an at value outside [0, duration). The error names the duration.
• source_not_found: a dead or foreign media.sume.com URL.

When should I use a different media tool?

Inspect only reads a clip; it never re-encodes it. Other jobs have their own routes:

• An exact frame at time t, at source size: video frames. See How to extract frames from a video.
• Captions burned onto a public URL: video captions, covered in How to burn captions onto a video.
• A new MP4 cut: video trim. The audio track as wav or mp3: audio detach. A dim or crop pass: video filter.
• Several clips in sequence: Timeline 1.0.