Prompt Caching

Prompt caching lets the model's provider reuse the part of a prompt it has already seen — a long system instruction, a reference document, a code file — instead of reading it again. The model still runs; the repeated prefix is billed at the model's cache rate, lower than its input rate, and the answer starts sooner.

Prompt caching is not the response cache, which keeps a whole answer and skips the model.

Which models cache#

Most providers cache on their own, with nothing to set: a prompt long enough that repeats a prefix is read from cache. Some — Anthropic's among them — cache only the blocks you mark. A model's endpoints show whether it caches, and at what rate:

curl https://opendunes.com/v1/models/anthropic/claude-sonnet-5/endpoints \
  -H "Authorization: Bearer $OPENDUNES_API_KEY"

Each endpoint's pricing carries input_cache_read and, where writing to the cache is billed, input_cache_write — in DA per token, beside prompt. supports_implicit_caching says whether it caches without being asked.

Marking what to cache#

Mark the end of the stable prefix with cache_control on a content block. Everything up to that block is cached:

{
  "model": "anthropic/claude-sonnet-5",
  "messages": [
    {
      "role": "system",
      "content": [
        {
          "type": "text",
          "text": "You are a legal assistant specialising in Algerian commercial law. [... long reference text ...]",
          "cache_control": { "type": "ephemeral" }
        }
      ]
    },
    { "role": "user", "content": "Summarise Article 42 of the Commercial Code." }
  ]
}

cache_control and prompt_cache_key travel to the provider as you send them, on /v1/chat/completions, /v1/messages and /v1/responses, and a preset keeps them. The prefix must match byte for byte: one changed character starts a new cache.

What caching saved#

Every answer's usage says how much of the prompt came from cache:

{
  "usage": {
    "prompt_tokens": 10339,
    "completion_tokens": 60,
    "total_tokens": 10399,
    "prompt_tokens_details": {
      "cached_tokens": 10318,
      "cache_write_tokens": 0
    }
  }
}

When it pays#

• A system prompt of a thousand tokens or more, the same on every request.
• A document, a code file or a knowledge base sent with every call.
• Many similar requests close together — an agent loop, a batch of questions over one document.

A short, one-off prompt gains nothing.