opendunes
Requests · 4 min
Copy page ▾

Batches

Many requests now, the answers later.

  1. Build the batch.
  2. Send it.
  3. Ask until it ends.
  4. Read the answers.

1. Build the batch

A batch is a list of requests sent in one call and answered later, when the provider has worked through them. It suits work nobody waits on: a shop's catalogue to describe, a month of reviews to sort.

One batch holds requests for one route and one model.

You sendHolds
endpointthe route every request in the batch takes: /v1/chat/completions, /v1/responses, /v1/messages or /v1/embeddings
modelthe model's ID, for every request
requeststhe requests: each {"custom_id": ..., "body": ...}
custom_idin each request: your name for it, unique in the batch; its answer comes back under it
bodyin each request: the request as that route takes it; model may be left out, and if sent, it must be the batch's
completion_windowoptional: 24h, the only window; a batch not done within it ends expired

A batch runs on the model's batch entry, the one its ID carries with :batch in Models. Name the model by its plain ID: the batch picks that entry on its own.

2. Send it

POST /v1/batches takes the batch and answers 202 at once, with the batch's id. The code below runs every step: it sends three product descriptions to write, asks every minute until the batch ends, and prints each answer.

import json
import os
import time
import urllib.request

API_KEY = os.environ["OPENDUNES_API_KEY"]
ADDRESS = "https://opendunes.com/v1/batches"
MODEL = "anthropic/claude-sonnet-4.5"
PROMPT = "Two lines of shop copy, in French, for: "
ENDED = ("completed", "failed", "expired", "cancelled")


def call(url, body=None):
    request = urllib.request.Request(
        url,
        data=json.dumps(body).encode() if body else None,
        headers={
            "Authorization": "Bearer " + API_KEY,
            "Content-Type": "application/json",
        },
    )
    with urllib.request.urlopen(request) as response:
        return json.load(response)


products = {
    "burnous-01": "Burnous en laine, blanc cassé",
    "tapis-07": "Tapis berbère, motifs Ouarsenis",
    "tajine-03": "Tajine de Nedroma, terre cuite",
}

requests = [
    {
        "custom_id": sku,
        "body": {"messages": [{"role": "user", "content": PROMPT + name}]},
    }
    for sku, name in products.items()
]
batch = call(
    ADDRESS,
    {"endpoint": "/v1/chat/completions", "model": MODEL, "requests": requests},
)

while batch["status"] not in ENDED:
    time.sleep(60)
    batch = call(ADDRESS + "/" + batch["id"])

print(batch["status"])

if batch["status"] == "completed":
    for result in batch["results"]:
        if result["response"]:
            answer = result["response"]["body"]["choices"][0]["message"]
            print(result["custom_id"], answer["content"])
        else:
            print(result["custom_id"], result["error"])
    print(batch["usage"]["cost"])
else:
    print(batch["error"])

3. Ask until it ends

GET /v1/batches/{id} answers where the batch stands. Asking is free, however often you ask; once a minute is enough.

StatusMeans
validatingreceived; its requests are being checked
in_progressthe requests are running
finalizingthe answers are being gathered
cancellinga cancel is under way
completedthe end: results holds every answer, and usage.cost the charge
failedthe end, with no answers; error.message says why
expiredthe end: the batch lapsed before it finished
cancelledthe end: the batch was cancelled

completed, failed, expired and cancelled are the end: the batch never moves again. request_counts says how far it has gone: total, completed, failed.

Watch out

A 202 means the batch arrived, not that every request in it passed. One request that fails a check after the 202 fails the whole batch, and error.message names why.

4. Read the answers

On completed, results holds one entry per request, in no fixed order: match each to your request by custom_id.

FieldHolds
custom_idyour name for the request
responsethe answer: status_code, and body, the answer as the route gives it; empty when the request failed
errorwhy the request failed; empty when it answered

usage.cost on the batch is its charge in DADAThe Algerian dinar, the one currency the docs show.More on Balance and charges →, taken from your balanceBalanceWhat you can spend, in DA.More on Balance and charges →: every request's tokens, at the model's batch rate. The model's page in Models gives the rate.

Every batch, and deleting one

RouteDoes
GET /v1/batcheslists your batches, newest first, without their answers
GET /v1/batches/{id}one batch, with its answers once it has ended
DELETE /v1/batches/{id}deletes a batch that has ended, with its requests and answers

All three are free. Deleting keeps the charge and its receipt.

What a batch refuses

The requestThe batch
a model with no batch entryrefused
"stream": truerefused: a batch never streams
a picture or a file sent as bytesrefused: each goes as a public link, an http or https address
audio or video, sent or asked forrefused

A request the batch refuses is refused whole, before the 202: the answer is a 400 whose message names the request by its custom_id. A request that passes this check and fails afterwards fails the whole batch.

Do

Send one batch per model. Anything a batch refuses goes as an ordinary request instead: Text.

Errors

The body says why.

StatusWhen
400the body is malformed, two requests share a custom_id, the model has no batch entry, or a request is one the batch refuses; the message names the request by its custom_id
401the key is missing, wrong, disabled or expired
402your balance is at zero or below, cannot pay the request's hold, or the key's cap is spent; error.metadata.limit_source names which, and deposit_url or key_url says where to lift it. After up to 60 seconds of waiting for money your running requests hold, the 402 carries Retry-After: 60
404the batch, or the model, does not exist
409a delete on a batch that has not ended
413the body is too large

What to do on each: Errors and retries.

Last modified on