How to remove silence from a video automatically

To remove silence from a video, find the pauses where nobody speaks, cut them out, and join what is left. With Sume, when the clip is already hosted on Sume, transcribe it with video inspect (transcribe: true), treat every gap between words that is longer than your threshold as silence, and render the kept ranges back to back in one Timeline 1.0 job over the clip's detached audio.

The facts come from Sume's Video inspect, Audio detach, Timeline 1.0, and Timeline audio docs and the Sume API reference, read on 2026-09-27. The render itself is walked through in remove part of a video by API; this post is about finding every pause automatically.

How do I find the silent parts automatically?

Ask for a transcript. The clip must already be your workspace's media.sume.com artifact or asset, such as an earlier Sume job's output; Sume has no public upload route for a recording on your computer (which URLs each endpoint accepts). Send it to POST /v1/video-inspect with transcribe: true, and frames: false to skip the stills. The default mode is sync: it waits up to 30 seconds and answers 200 with the result, or 202 with a job you poll at GET /v1/video-inspect/:id.

The result's transcript.words[] lists each word with start and end in seconds from the start of the video. Speech is where the words are, and every gap between one word's end and the next word's start is a pause.

curl -X POST https://api.sume.com/v1/video-inspect \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: talk-silence-001" \
  -d '{
    "video_url": "https://media.sume.com/artifacts/artf_demo/talk.mp4",
    "frames": false,
    "transcribe": true
  }'

How do I turn word timings into cuts?

Pick two numbers: the threshold, the shortest pause worth cutting, and a margin kept on each side of speech so the edges of words are not clipped. Both are your choice, not Sume settings. This Python function merges words into kept ranges; the API reference marks only word as required, so it skips entries without timings:

• Keep the threshold above twice the margin, so the margins of two ranges never overlap.
• Each pair becomes one audio.parts[] slice of the detached wav and one video[] slot of the clip, both with that source_in and duration. Each slot starts at the running total of the kept lengths, and audio.duration_seconds is their sum.
• Detach the audio with POST /v1/audio-detach for the spine. The transcript's own audio_url is a 16 kHz mono speech-to-text copy, and the API reference says it is not a timeline spine.
• Before you pay, the unbilled POST /v1/timeline-1.0/plan checks a body with dozens of slots and returns its duration_seconds, segment_count, and estimated_cost_usd_micros.

def kept_ranges(words, clip_length, threshold=0.6, margin=0.15):
    """(source_in, duration) pairs from transcript.words[]."""
    timed = [w for w in words if "start" in w and "end" in w]
    ranges = []
    for w in timed:
        if ranges and w["start"] - ranges[-1][1] <= threshold:
            ranges[-1][1] = w["end"]  # same stretch of speech
        else:
            ranges.append([w["start"], w["end"]])  # pause before this word
    kept = []
    for start, end in ranges:
        s, e = max(0.0, start - margin), min(clip_length, end + margin)
        if e - s >= 0.2:  # a Timeline slot lasts at least 0.2 s
            kept.append((round(s, 3), round(e - s, 3)))
    return kept

What counts as silence?

Here, silence means no transcribed words, not low volume. A gap between words is cut even when music or background sound fills it, and a word the transcript missed is cut along with its gap. Play the result once before you publish it.

How many pauses can one render remove?

Up to 200 kept ranges fit in one render, since video[] takes up to 200 slots. audio.parts[] stops at 20 slices, though, so with more than 20 kept ranges, join the audio slices first with Timeline audio, as remove part of a video by API shows, and pass the joined file as audio.url.

What does it cost, and what are the limits?

Each step bills on its own, plus a 5.5% agent fee by default. A clip longer than 900 seconds needs two detaches with range, because one detach writes at most 900 seconds; each part's source_in then counts from the start of its own file.