Reference Images and Pinned Images

Lesson rewritten on November 2, 2026 based on the Reference-to-Video documentation from xAI: reference images make specific people, objects, or clothes appear, designated by the prompt as <IMAGE_0>, <IMAGE_1>… — not just to impose a style; on grok-imagine-video-1.5, they can now accompany image, a pinned last image, keyframes, and voices. The old version claimed that image and reference_images were incompatible. Sources: Reference-to-Video · Reference Videos.

Feeding the Model Visuals

Directly animating an image isn’t the only way to provide visuals to the model. xAI’s video API offers a second mechanism: reference images. Instead of moving the image as is, they provide elements that the video must make appear — a person, a product, an item of clothing — without freezing the foreground. Understanding the boundary between image and reference_images is the key to this lesson.

Two Parameters, Two Intentions

The image parameter provides the image to animate: it becomes the first frame of the video, and its content is set in motion. This is the image-to-video from previous lessons.

The reference_images parameter provides images whose elements the model reuses: a model’s face, a shirt, a perfume bottle. The scene itself follows your prompt. xAI cites three typical uses: virtual try-on, product placement, and stories with recurring characters. Each reference is a public HTTPS URL, base64-encoded data, or the file_id of an uploaded file, and you can mix all three forms in a single request.

{
  "model": "grok-imagine-video-1.5",
  "prompt": "Slow tracking shot on a white runway. The model from <IMAGE_0> enters from the background and walks toward the camera; they are wearing the shirt from <IMAGE_1> and black jeans. Slow motion, high quality, detailed faces.",
  "reference_images": [
    {"url": "https://exemple.com/mannequin.jpg"},
    {"url": "https://exemple.com/chemise.jpg"}
  ],
  "duration": 10,
  "aspect_ratio": "16:9",
  "resolution": "720p"
}

Citing References in the Prompt

The prompt designates each reference by its position in the list: <IMAGE_0> for the first, <IMAGE_1> for the second, and so on. If you also provide image to pin the first frame, it takes <IMAGE_0>, and your references start at <IMAGE_1>.

With multiple references, xAI recommends giving each one a single role and a fixed place in the scene, keeping the cast limited, and using static props. A reference serving as both character and decor, or ten objects moving at once, produces confusing scenes.

The Rules

A request of this type must contain at least one reference image, voice, last image, or keyframe.

Pinning Specific Images (grok-imagine-video-1.5)

On grok-imagine-video-1.5, you can fix exact images at given times in the video:

• image and last_frame together: the video starts from the first image and ends on the last, with the model interpolating between the two
• last_frame alone: the model invents the opening and lands on the pinned image
• last_frame with references or voices: the last image is pinned, references guide the rest; add image to pin the first as well
• keyframes: up to four keyframes, each with a timestamp_s strictly between 0 and the video duration; anchors snap to a one-third-second grid, so two keyframes must be spaced by at least one-third of a second

{
  "model": "grok-imagine-video-1.5",
  "prompt": "The flower opens slowly under the morning light",
  "image": {"url": "https://exemple.com/bouton-ferme.jpg"},
  "last_frame": {"url": "https://exemple.com/fleur-ouverte.jpg"},
  "keyframes": [
    {"image": {"url": "https://exemple.com/fleur-mi-ouverte.jpg"}, "timestamp_s": 3.0}
  ],
  "duration": 6
}

The same model can give voices to your characters: reference_audios accepts up to three predefined voices, designated by their voice_id (from the xAI text-to-speech voice catalog, eve or leo for example), which the prompt cites as <AUDIO_0>, <AUDIO_1>…

What References Make Possible

The first use case, and the most cost-effective in a professional context, is continuity: the same character, the same product, from one video to the next. Imagine a ten-video campaign produced over three weeks: without references, the product subtly changes shape and the mascot changes face from one clip to the next. By systematically providing the same references, you anchor what must remain identical, and only the prompt varies the scene.

{
  "model": "grok-imagine-video-1.5",
  "prompt": "The smartphone from <IMAGE_0> sits on a minimalist desk; its screen turns on gradually, soft morning light",
  "reference_images": [
    {"url": "https://exemple.com/smartphone-officiel.jpg"}
  ],
  "duration": 6
}

Product placement follows the same logic — your bottle, your shoe in an invented scene — and virtual try-on pushes it further: a person and a garment, each on its own reference, united in the same shot.

Choosing and Pairing Your References

Choose sharp, well-lit images where the element to reuse occupies most of the frame: a shirt photographed alone on a neutral background works better than a shirt worn in a crowd. One reference, one subject.

Prompt writing follows the role distribution: references bring the elements, the prompt describes the scene and the action. Name each reference by its tag where it intervenes, and stay explicit about movement, which no image carries.

When facing a request, the decision comes down to one question. If you want to animate a specific image as is, use image. If you want a new scene where given people or objects appear, use reference_images. If you want to control the start, end, or specific moments, use the pinned images of grok-imagine-video-1.5. And if none of this matters, the prompt alone suffices.

Key Points to Remember

• image animates an image that becomes the first of the video; reference_images makes elements appear without freezing the foreground
• The prompt cites references by position: <IMAGE_0>, <IMAGE_1>… (image takes <IMAGE_0> when provided)
• Up to 14 references on grok-imagine-video-1.5 (15 s), 7 on grok-imagine-video (10 s), 720p max
• On grok-imagine-video-1.5, image, last_frame, keyframes, and reference_audios complement references
• One role per reference, limited cast, static props