Grok Imagine <IMAGE_0> tags shift with a first frame; Sume is 0-based
On xAI, pinning a first frame with image takes <IMAGE_0> and references start at <IMAGE_1>. Sume Omni tags are <IMAGE_REF_0>, 0-based by list order.

On xAI, if you also set image to pin the first frame, that image takes <IMAGE_0> and your reference_images start at <IMAGE_1>. On Sume's Omni route (gemini-omni-flash-1.1), tags are <IMAGE_REF_0> and <VIDEO_REF_0>, 0-based in the order of the reference lists, and the docs do not describe a first-frame shift.
Sources: xAI's reference-to-video page and the Video Router docs, read 2026-10-01.
How does xAI number reference images?
The page says to refer to reference images by their position in the list, starting at <IMAGE_0>, then <IMAGE_1>, and so on. With image set as well, the first frame takes <IMAGE_0> and the references start at <IMAGE_1>. Voices are tagged <AUDIO_0>, <AUDIO_1> and <AUDIO_2>.
| Case | xAI | Sume Omni |
|---|---|---|
| References only | First image is <IMAGE_0> | First reference_image_urls entry is <IMAGE_REF_0> |
| First frame plus references | Frame is <IMAGE_0>; references start at <IMAGE_1> | image_url and reference_image_urls are listed as separate capabilities; no combined numbering is documented |
| Video references | Not covered in the part of the page I read | <VIDEO_REF_0>, 0-based, list order |
| Audio references | <AUDIO_0> and up | No reference_audio_urls on Omni |
How does Sume number references?
The Omni row says to address media as <IMAGE_REF_0> and <VIDEO_REF_0>, 0-based, in list order. Caps are reference_image_urls up to 10 and reference_video_urls up to 3, each 3 seconds or shorter.
Can I combine a video_url with references on Sume?
No. The docs say video_url is the edit source, not a reference, and it cannot be combined with image_url, end_image_url or reference_*_urls. An edit request therefore has no tags to number.
How do I avoid an off-by-one tag?
Number from the first list you send, and test one request with a single reference before a large one. If you port a prompt written for xAI's first-frame case, subtract the shift yourself: every <IMAGE_n> that followed the pinned frame becomes <IMAGE_REF_n-1>, but only when your Sume request sends references alone. For the same tag idea on another model, see Seedance 2.5 Omni reference at-tags.
Sources
Related posts
More in Developers
- Grok Imagine base64 response_format vs a Sume hosted URL
xAI lets you request base64 instead of a temporary URL. Sume returns Sume-hosted, signed URLs in data[].url, not inline base64, for x-ai/grok-image.
- Grok Imagine video extension duration counts only the extension
In xAI video extension, duration is the added seconds only: a 10s input plus duration 5 returns 15s. Sume has no extend on Omni; chain clips from a last frame.
- Grok Imagine video input: base64 or file_id vs Sume HTTPS URLs
xAI's extension page accepts a source video as a public URL, base64 data URI or file_id. Sume's video fields take public HTTPS media URLs only.
- HeyGen 429 Retry-After vs Sume rate_limited and queue_full
HeyGen sends 429 with a Retry-After header in seconds. Sume sends 429 rate_limited with retry-after when present, and a separate 429 queue_full.
Written by Sume