Generate music for a video with AI, then fit it to the cut

To generate music for a video with AI, describe the video in a short music brief (its mood, tempo, instruments, and the second where the energy should change), send a representative still as an image input, and ask for about the video's length. Then fit the track to the cut: loop or trim it, and fade it out. On Sume, that is one Music Router request with the still as image_url, plus an optional Timeline 1.0 render that lays the track under the video. The Music Router takes a text prompt and one image, not the video, so your brief carries the timing.

The steps come from Sume's Music 1.0, Music Router, Video frames, and Timeline 1.0 docs, read on 2026-09-27. To mix a track you already have, see Add background music to a video.

How do I describe a video in a music brief?

Watch the cut with a timer and note its length, its mood, and the moment where the energy should lift or drop. Write those into the brief the Sume docs recommend: the emotion, the genre, a tempo in BPM, the key, two to four instruments with texture, one named moment at its timestamp, and the era, closing with “Instrumental, no vocals.”

For a 45-second product video with a reveal at 0:28: “Optimistic, warm acoustic pop, 104 BPM, G major. Fingerpicked acoustic guitar, soft kick, claps, a glockenspiel hook. A sparse intro; the full beat drops in at 0:28 for the reveal and resolves on one sustained chord at 0:44. Modern, clean and bright. A 45-second track. Instrumental, no vocals.”

• Add “no spoken word” only when the music will sit under narration.
• For scenes that should sound different, vary the genre family, the tempo (at least 12 BPM apart), and the lead instrument. AI music prompt examples has more briefs.

How do I send a frame from the video with the prompt?

Pass one representative still as image_url, a public HTTPS image; the docs' prompt guidance says to pass the scene still when appropriate. The image is optional and doesn't change the price, as Image to music AI explains.

If the video is already on Sume, POST /v1/video-frames returns exact stills at the seconds you name as durable media.sume.com images, from clips up to 300 seconds, and the call is unbilled. For your own footage, export a frame in your editor and host it at a public HTTPS URL.

curl -X POST https://api.sume.com/v1/music-router/generate \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: launch-video-music-001" \
  -d '{
    "prompt": "Optimistic, warm acoustic pop, 104 BPM, G major. Fingerpicked acoustic guitar, soft kick, claps, a glockenspiel hook. A sparse intro; the full beat drops in at 0:28 for the reveal and resolves on one sustained chord at 0:44. Modern, clean and bright. A 45-second track. Instrumental, no vocals.",
    "image_url": "https://media.sume.com/artifacts/artf_demo/frame-28s.jpg"
  }'

How do I make the track as long as the video?

Ask for the length in the prompt (“A 45-second track”) or lay out sections with markers such as [0:00-0:15] Intro: …; duration and duration_seconds are rejected. The docs call a brief's details creative directions, not guaranteed output settings, so check the returned track's length before you fit it.

Then fit it in your editor, or, for a video on Sume, in a Timeline 1.0 render with the track as soundtrack. loop repeats a short bed until the spine ends, fade_out_seconds (up to 10) fades it over the last seconds, and the output always runs audio.duration_seconds, so a longer track ends there. The render's sound is the audio spine plus that bed, as Add background music to a video explains.

What does it cost, and what can't it do?

On API pricing, music generation is $0.125 per audio generation and a Timeline render reserves $0.10 per output minute, each plus a 5.5% agent fee by default; a render never captures more than its reservation. Audio detach, for the speech row, is billed per job, and video frames are unbilled.

• No sync to cuts or beats: the request carries a prompt and one image, so every timestamp comes from your brief.
• No promised mood match. The docs say to verify the generated audio.
• A render takes only this workspace's media.sume.com files, such as earlier Sume job outputs, so off-Sume footage stays in your editor.