Reuse large shared prompt prefixes to reduce cost and latency on repeated requests.
Not yet available. Prompt caching is on the roadmap. The page below documents the intended behavior.
Prompt caching lets you mark a stable prefix in your prompt — a long system instruction, a reference document, a code file — so that OpenDunes can reuse the key-value cache from the first request on all subsequent calls that share that prefix.
On the first request, OpenDunes processes and caches the designated prefix at the provider level. On follow-up requests that include the same prefix, the model skips re-processing those tokens — reducing both latency and the DA cost of input tokens.
The cache is keyed on the exact byte content of the prefix up to the cache breakpoint. Even a single character change invalidates the cache for that prefix.
Annotate a message content block with "cache_control": { "type": "ephemeral" } to instruct OpenDunes to cache up to that point:
Cached prefixes expire after a period of inactivity (exact TTL will be documented when the feature launches). Infrequently used caches are evicted automatically.
Cached input tokens are billed at a reduced rate compared to uncached tokens. The exact DA discount per million tokens will be published in the model catalog when caching is live. Non-cached tokens are always billed at the standard rate.
Caching pays off when:
For short, one-off prompts, the overhead of setting up a cache breakpoint exceeds any savings.
Not all models support caching at the provider level. When the feature launches, the model catalog will include a prompt_caching capability flag. Filter with GET /api/models?capability=prompt_caching.