Uptime Optimization

AI inference is probabilistic infrastructure — upstream providers occasionally return errors, rate limits enforce usage ceilings, and networks misbehave. A robust integration handles these conditions without surfacing failures to your users.

Retry with Exponential Back-Off#

The 429 rate_limit_exceeded and 503 provider_overloaded responses both include a Retry-After header. Honor it — retrying sooner only meets the provider's limit again. OpenDunes itself puts no cap on the number of requests.

import time
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://opendunes.com/v1",
    api_key=os.environ["OPENDUNES_API_KEY"],
)

def chat_with_retry(messages, model="anthropic/claude-sonnet-5", max_attempts=4):
    delay = 1
    for attempt in range(max_attempts):
        try:
            return client.chat.completions.create(model=model, messages=messages)
        except Exception as e:
            status = getattr(e, "status_code", None)
            if status in (429, 503) and attempt < max_attempts - 1:
                retry_after = int(getattr(e, "response", None) and
                                  e.response.headers.get("Retry-After", delay) or delay)
                time.sleep(retry_after)
                delay = min(delay * 2, 30)  # cap at 30 s
            else:
                raise

Add a small random jitter (e.g. random.uniform(0, 1) seconds) to prevent a thundering herd when multiple workers retry simultaneously.

Idempotency#

Inference requests are not idempotent by nature — retrying a request may produce a different response. For workflows where duplicate processing matters (billing actions, database writes triggered by AI output), add your own deduplication layer keyed on a request ID that you generate client-side.

Pass your ID in the X-Request-Id request header. OpenDunes will echo it in the response header, making it easy to correlate logs.

Graceful Degradation#

Design for failure states so your product keeps working:

• Timeouts: set a client-side timeout (30–60 s for streaming). If the stream stalls, cancel and surface a retry prompt to the user rather than hanging indefinitely.
• 402 payment_required: catch this status explicitly — error.metadata.limit_source says whether the wallet, the key's cap or a guardrail's budget stopped it — and redirect the user to deposit into their balance, rather than showing a generic error.
• 504 provider_timeout: show a user-friendly message and offer to retry. Do not retry automatically in a tight loop — back off first.

try {
  const resp = await client.chat.completions.create({ model, messages });
  return resp.choices[0].message.content;
} catch (err: any) {
  if (err.status === 402) {
    return { error: "top_up_required", redirect: "/dashboard/billing" };
  }
  if (err.status === 504) {
    return { error: "timeout", message: "The model took too long. Try again." };
  }
  throw err;
}

Check Your Balance Proactively#

Each response includes the X-Balance-Available header (remaining micro-DA). Poll it, or watch the low-balance state on your dashboard, to avoid surprise 402 errors in production.

Model Fallbacks#

Pass an ordered models array (up to 4 slugs) instead of a single model — if the primary errors or is unavailable, OpenDunes retries the next model in the list automatically. See Model Fallbacks. Provider-level failover within a model rides on provider.allow_fallbacks (on by default).

Error Reference#

For the full list of status codes and their meanings, see the Errors reference page.