Streaming

Streaming shows the answer as it arrives instead of waiting for the whole of it. "stream": true sends it as server-sent events; any library that reads OpenAI streaming reads ours unchanged.

Enabling streaming#

Add "stream": true to the request body. The answer switches from application/json to text/event-stream, and the connection stays open until the model finishes.

{
  "model": "meta-llama/llama-4-maverick",
  "messages": [{ "role": "user", "content": "Write a short poem about the Sahara." }],
  "stream": true
}

The event format#

: OPENDUNES PROCESSING          a keep-alive while the model works
data: <chunk>                   one piece of the answer
data: <chunk>
data: <last chunk>              the finish reason, the usage and the cost
data: [DONE]

Each event is a line opening data: followed by a JSON object, then a blank line. A chunk carries only the new text, in choices[0].delta.content; build the full answer by joining them.

data: {"id":"gen-01938f","object":"chat.completion.chunk","created":1750000000,"model":"meta-llama/llama-4-maverick","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"gen-01938f","object":"chat.completion.chunk","created":1750000000,"model":"meta-llama/llama-4-maverick","choices":[{"index":0,"delta":{"content":"Sands"},"finish_reason":null}]}

data: {"id":"gen-01938f","object":"chat.completion.chunk","created":1750000000,"model":"meta-llama/llama-4-maverick","choices":[{"index":0,"delta":{"content":" drift"},"finish_reason":null}]}

data: {"id":"gen-01938f","object":"chat.completion.chunk","created":1750000000,"model":"meta-llama/llama-4-maverick","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":18,"completion_tokens":42,"total_tokens":60,"cost":0.0231}}

data: [DONE]

• The usage rides the closing event. usage.cost is the request's charge, in DA.
• A line opening with : is a keep-alive and carries nothing. Ignore it; the libraries already do.
• /v1/messages streams the Anthropic events, and /v1/responses the Responses events, each with its usage on its closing event.

Headers on a stream#

The answer's headers leave before the model has written a word, so they carry what is known then: X-Generation-Id, the handle of the request's receipt. They carry no X-Balance-Available — the charge does not exist yet. Read your balance at GET /v1/key, or at GET /v1/credits with a management key.

Consuming the stream#

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://opendunes.com/v1",
    api_key=os.environ["OPENDUNES_API_KEY"],
)

stream = client.chat.completions.create(
    model="meta-llama/llama-4-maverick",
    messages=[{"role": "user", "content": "Write a short poem about the Sahara."}],
    stream=True,
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
print()

Stream cancellation#

Closing the connection ends the stream. In Python, call stream.close(); in Node, stream.controller.abort(); with fetch, abort() on the AbortController you passed.

A cut stream is billed on what the provider actually produced for it, read from the provider's record of the request — never an estimate. That record is written after the stream ends, so the charge lands a few seconds later. A receipt or a balance read in those seconds does not show it yet; read it again and the charge is there.

Handling errors during streaming#

Errors before the stream opens#

A request refused before its first chunk — a bad key (401), an empty wallet (402), an unknown model (404), a provider that fails before it starts — gets an ordinary error response, with its status and the error object in the route's shape — the Anthropic shape on /v1/messages, the Responses shape on /v1/responses. Your event loop never starts.

Errors mid-stream#

Once the answer is under way its 200 is already sent. An error after that point arrives as the last event, carrying error and finish_reason: "error", then [DONE]:

data: {"object":"chat.completion.chunk","error":{"code":502,"message":"The model is down, or answered with nothing usable.","metadata":{"error_type":"provider_unavailable"}},"choices":[{"index":0,"delta":{"content":""},"finish_reason":"error"}]}

data: [DONE]

Check the last chunk for error or finish_reason: "error", show its message, and offer a retry. What the model wrote before the error is billed. See Errors for the statuses and error_type words.