Identical completions are cached so you pay less and get results faster on repeated requests.
Not yet available. Response caching is on the roadmap.
When your application sends the same prompt more than once — the same model, same messages, same parameters — OpenDunes can return a cached completion instantly instead of forwarding to the underlying model. You receive the same response at a fraction of the cost and with near-zero latency.
A cache key is derived from the combination of:
modelmessages (full array, exact match)temperature, top_p, max_tokens (any parameter that affects the output distribution)If all components match a previous request's key, the cached response is returned. Cache entries are stored for a configurable TTL (default: 1 hour).
Non-deterministic by nature, higher-temperature completions are less likely to be useful when cached — set temperature: 0 for use cases where exact repeatability matters (classification, extraction, structured outputs).
A cache hit is billed at a reduced rate compared to a full model completion. The exact rate depends on the model — check the pricing column in the model catalog once caching is live.
Latency on a cache hit is typically under 50 ms regardless of model or response length, since no inference is performed.
Responses include headers that tell you whether the result came from cache:
| Header | Value |
|---|---|
X-Cache-Status | HIT or MISS |
X-Cache-Age | Seconds since the cached entry was created |
By default, caching is enabled for all requests. You can control it per-request:
| Value | Behavior |
|---|---|
"auto" | Cache if the key matches (default) |
"no-store" | Never read from or write to cache for this request |
"no-cache" | Skip the cache read but write the result to cache for subsequent requests |
Response caching is most valuable when:
"stream": true are not cached (streaming responses are not stored).temperature > 0 can still be cached, but cache misses are more likely as the key is more sensitive to slight message differences.