Guide a Video with Reference Images

Not yet available. Reference-guided video generation is tracked to be built — today, text-to-video works on POST /v1/videos, which accepts model, prompt, and seconds.

Reference-guided generation lets you constrain the output of a video model using one or more images as visual anchors. Instead of describing a character or setting purely in words, you supply a photo or illustration and the model keeps the output visually consistent with it.

When to use reference images#

• Character consistency: you have a product, person, or character that must appear recognizably in the video.
• Style transfer: you want the video to match the palette and mood of a reference photo.
• Setting fidelity: you want the generated scene to match a real location shown in an image.

Not every video model supports reference images. Filter the catalog with ?capability=image-to-video to see models that do.

Intended request shape#

This is the intended design, not a working request — the shipped API does not accept a references field yet:

# Intended design — not yet available
curl https://opendunes.com/api/v1/videos \
  -H "Authorization: Bearer $OPENDUNES_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/veo-3.1",
    "prompt": "The character walks through a busy souk, warm afternoon light",
    "seconds": 5,
    "references": [
      {
        "type": "character",
        "url": "https://your-cdn.com/character-reference.jpg",
        "weight": 0.85
      }
    ]
  }'

In the intended design, the references array accepts:

Using multiple references#

The intended design combines references to control different aspects of the output independently:

"references": [
  { "type": "character", "url": "https://cdn.example.com/hero.jpg",   "weight": 0.9 },
  { "type": "style",     "url": "https://cdn.example.com/palette.jpg","weight": 0.6 },
  { "type": "setting",   "url": "https://cdn.example.com/alger.jpg",  "weight": 0.75 }
]

Keep total weight reasonable. Very high weights on multiple references simultaneously can produce conflicting constraints and reduce prompt adherence.

Prompt tips for reference-guided generation#

When using a character reference, describe the action rather than re-describing the character's appearance — the model already has the visual:

Avoid: "A tall woman with dark hair and red jacket walks through a market"

Prefer: "She walks through a busy market, looking at stalls on either side"

For style references, you can be explicit: "In the visual style of the reference image, a desert landscape at dawn".

Cost#

Reference-guided requests are billed at the same per-second DA rate as standard text-to-video. The additional processing for reference images adds a small overhead; exact rates will be shown in the model catalog once the feature is live.