Streaming

Streaming lets you display output as it arrives rather than waiting for the full completion. The OpenDunes API implements the standard SSE (server-sent events) format, so any library that handles OpenAI streaming works out of the box.

Enabling streaming#

Add "stream": true to your request body. The response switches from application/json to text/event-stream and the connection stays open until the model finishes.

{
  "model": "meta-llama/llama-4-maverick",
  "messages": [{ "role": "user", "content": "Write a short poem about the Sahara." }],
  "stream": true
}

The event format#

Each event arrives as a line starting with data: followed by a JSON object, then a blank line. The chunks carry delta content — only the new token(s) in choices[0].delta.content, not the full text so far.

data: {"id":"chatcmpl-01938f","object":"chat.completion.chunk","created":1750000000,"model":"meta-llama/llama-4-maverick","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"chatcmpl-01938f","object":"chat.completion.chunk","created":1750000000,"model":"meta-llama/llama-4-maverick","choices":[{"index":0,"delta":{"content":"Sands"},"finish_reason":null}]}

data: {"id":"chatcmpl-01938f","object":"chat.completion.chunk","created":1750000000,"model":"meta-llama/llama-4-maverick","choices":[{"index":0,"delta":{"content":" drift"},"finish_reason":null}]}

data: {"id":"chatcmpl-01938f","object":"chat.completion.chunk","created":1750000000,"model":"meta-llama/llama-4-maverick","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":18,"completion_tokens":42,"total_tokens":60}}

data: [DONE]

The final data chunk (before [DONE]) carries the usage object with token counts for the full request. Accumulate your own buffer client-side by concatenating delta.content from each chunk.

Consuming the stream#

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://opendunes.com/api/v1",
    api_key=os.environ["OPENDUNES_API_KEY"],
)

stream = client.chat.completions.create(
    model="meta-llama/llama-4-maverick",
    messages=[{"role": "user", "content": "Write a short poem about the Sahara."}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)
print()

Stream cancellation#

Close the connection at any point to cancel the stream. The server stops generating as soon as it detects the disconnect.

In Python, call stream.close(). In Node, call stream.controller.abort(). With raw fetch, call controller.abort() on the AbortController you passed to the request.

Billing settles on the tokens already sent — you are not charged for tokens that were never transmitted.

Handling errors during streaming#

Errors before the stream opens#

If the request is rejected before the stream opens — 401 invalid_api_key, or 402 insufficient_credits caught by the balance check — the server returns a normal non-streaming error response with the appropriate HTTP status code and JSON body. Your SSE loop never starts.

Errors mid-stream#

Once the response has been sent as 200 text/event-stream, the HTTP status can no longer change. If something fails after that point — the upstream provider erroring, or a balance reservation failing right as the stream opens — the server emits an error event followed by [DONE]:

data: {"error": "upstream_error", "message": "AI provider returned an error."}

data: [DONE]

A reservation failure surfaces the same way with "error": "insufficient_credits". Detect either by checking whether the final data chunk (before [DONE]) has an error field instead of choices. Handle it by surfacing the message to the user and offering a retry.

Billing settles on the tokens received before the error. See Errors for the full list of error codes.