Batches
Many requests now, the answers later.
1. Build the batch
A batch is a list of requests sent in one call and answered later, when the provider has worked through them. It suits work nobody waits on: a shop's catalogue to describe, a month of reviews to sort.
One batch holds requests for one route and one model.
| You send | Holds |
|---|---|
endpoint | the route every request in the batch takes: /v1/chat/completions, /v1/responses, /v1/messages or /v1/embeddings |
model | the model's ID, for every request |
requests | the requests: each {"custom_id": ..., "body": ...} |
custom_id | in each request: your name for it, unique in the batch; its answer comes back under it |
body | in each request: the request as that route takes it; model may be left out, and if sent, it must be the batch's |
completion_window | optional: 24h, the only window; a batch not done within it ends expired |
A batch runs on the model's batch entry, the one its ID carries with :batch in Models. Name the model by its plain ID: the batch picks that entry on its own.
2. Send it
POST /v1/batches takes the batch and answers 202 at once, with the batch's id. The code below runs every step: it sends three product descriptions to write, asks every minute until the batch ends, and prints each answer.
import json
import os
import time
import urllib.request
API_KEY = os.environ["OPENDUNES_API_KEY"]
ADDRESS = "https://opendunes.com/v1/batches"
MODEL = "anthropic/claude-sonnet-4.5"
PROMPT = "Two lines of shop copy, in French, for: "
ENDED = ("completed", "failed", "expired", "cancelled")
def call(url, body=None):
request = urllib.request.Request(
url,
data=json.dumps(body).encode() if body else None,
headers={
"Authorization": "Bearer " + API_KEY,
"Content-Type": "application/json",
},
)
with urllib.request.urlopen(request) as response:
return json.load(response)
products = {
"burnous-01": "Burnous en laine, blanc cassé",
"tapis-07": "Tapis berbère, motifs Ouarsenis",
"tajine-03": "Tajine de Nedroma, terre cuite",
}
requests = [
{
"custom_id": sku,
"body": {"messages": [{"role": "user", "content": PROMPT + name}]},
}
for sku, name in products.items()
]
batch = call(
ADDRESS,
{"endpoint": "/v1/chat/completions", "model": MODEL, "requests": requests},
)
while batch["status"] not in ENDED:
time.sleep(60)
batch = call(ADDRESS + "/" + batch["id"])
print(batch["status"])
if batch["status"] == "completed":
for result in batch["results"]:
if result["response"]:
answer = result["response"]["body"]["choices"][0]["message"]
print(result["custom_id"], answer["content"])
else:
print(result["custom_id"], result["error"])
print(batch["usage"]["cost"])
else:
print(batch["error"])3. Ask until it ends
GET /v1/batches/{id} answers where the batch stands. Asking is free, however often you ask; once a minute is enough.
| Status | Means |
|---|---|
validating | received; its requests are being checked |
in_progress | the requests are running |
finalizing | the answers are being gathered |
cancelling | a cancel is under way |
completed | the end: results holds every answer, and usage.cost the charge |
failed | the end, with no answers; error.message says why |
expired | the end: the batch lapsed before it finished |
cancelled | the end: the batch was cancelled |
completed, failed, expired and cancelled are the end: the batch never moves again. request_counts says how far it has gone: total, completed, failed.
A 202 means the batch arrived, not that every request in it passed. One request that fails a check after the 202 fails the whole batch, and error.message names why.
4. Read the answers
On completed, results holds one entry per request, in no fixed order: match each to your request by custom_id.
| Field | Holds |
|---|---|
custom_id | your name for the request |
response | the answer: status_code, and body, the answer as the route gives it; empty when the request failed |
error | why the request failed; empty when it answered |
usage.cost on the batch is its charge in DADAThe Algerian dinar, the one currency the docs show.More on Balance and charges →, taken from your balanceBalanceWhat you can spend, in DA.More on Balance and charges →: every request's tokens, at the model's batch rate. The model's page in Models gives the rate.
Every batch, and deleting one
| Route | Does |
|---|---|
GET /v1/batches | lists your batches, newest first, without their answers |
GET /v1/batches/{id} | one batch, with its answers once it has ended |
DELETE /v1/batches/{id} | deletes a batch that has ended, with its requests and answers |
All three are free. Deleting keeps the charge and its receipt.
What a batch refuses
| The request | The batch |
|---|---|
| a model with no batch entry | refused |
"stream": true | refused: a batch never streams |
| a picture or a file sent as bytes | refused: each goes as a public link, an http or https address |
| audio or video, sent or asked for | refused |
A request the batch refuses is refused whole, before the 202: the answer is a 400 whose message names the request by its custom_id. A request that passes this check and fails afterwards fails the whole batch.
Send one batch per model. Anything a batch refuses goes as an ordinary request instead: Text.
Errors
The body says why.
| Status | When |
|---|---|
| 400 | the body is malformed, two requests share a custom_id, the model has no batch entry, or a request is one the batch refuses; the message names the request by its custom_id |
| 401 | the key is missing, wrong, disabled or expired |
| 402 | your balance is at zero or below, cannot pay the request's hold, or the key's cap is spent; error.metadata.limit_source names which, and deposit_url or key_url says where to lift it. After up to 60 seconds of waiting for money your running requests hold, the 402 carries Retry-After: 60 |
| 404 | the batch, or the model, does not exist |
| 409 | a delete on a batch that has not ended |
| 413 | the body is too large |
What to do on each: Errors and retries.
Last modified on