# Benchmark scores

`GET /v1/benchmarks` answers benchmark scores, one row per model and benchmark, from three
sources.

## Request

Send it with an API key. Every field goes in the address, after `?`, and each one narrows the
list; with none, every row of every source comes back. The [models guide](/docs/models) shows
how to pick a model and read its ID.

Yacine wants the model that writes the best code for Souk Bot's order tools. He asks for one
source's rows with `source=artificial-analysis` and reads each model's `coding_index`.

| Field | Needed | You send |
|---|---|---|
| `source` | no | `artificial-analysis`, `design-arena` or `opendunes`: only that source's rows, each in its own shape |
| `task_type` | no | `coding`, `intelligence`, `agentic` or `search`: the rows scored on that kind of task |
| `benchmark_type` | no | one benchmark of the `opendunes` source: `gpqa_diamond`, `tau_bench_verified_airline`, `search_browsecomp`, `search_hle`, `search_dsqa` or `search_widesearch` |
| `max_results` | no | the most rows to return, `1` or more; every matching row when left out |
| `arena` | no | on `design-arena`: `models`, `builders` or `agents`; `models` when left out |
| `category` | no | on `design-arena`: one category of the arena, such as `uicomponent`, `gamedev` or `dataviz`; every category when left out |
| `search_engine` | no | on a search benchmark: only the runs that used this search engine |
| `search_surface` | no | on a search benchmark: `server-tool` for the search helper, `plugin` for the web plugin |
| `include_run_config` | no | `true` to add each search row's run settings, `run_config`; `false` when left out |

```python
import json
import os
import urllib.request

API_KEY = os.environ["OPENDUNES_API_KEY"]
SOURCE = "artificial-analysis"
ADDRESS = "https://opendunes.com/v1/benchmarks?source=" + SOURCE

request = urllib.request.Request(
    ADDRESS,
    headers={"Authorization": "Bearer " + API_KEY},
)
with urllib.request.urlopen(request) as response:
    scores = json.load(response)["data"]

for score in scores:
    print(score["model_permaslug"], score["coding_index"])
```

```javascript
const API_KEY = process.env.OPENDUNES_API_KEY;
const SOURCE = "artificial-analysis";
const ADDRESS = `https://opendunes.com/v1/benchmarks?source=${SOURCE}`;

const response = await fetch(ADDRESS, {
  headers: { Authorization: `Bearer ${API_KEY}` },
});
const { data } = await response.json();

for (const score of data) {
  console.log(score.model_permaslug, score.coding_index);
}
```

```go
package main

import (
	"encoding/json"
	"fmt"
	"net/http"
	"os"
)

const (
	source  = "artificial-analysis"
	address = "https://opendunes.com/v1/benchmarks?source=" + source
)

func main() {
	apiKey := os.Getenv("OPENDUNES_API_KEY")

	req, err := http.NewRequest("GET", address, nil)
	if err != nil {
		panic(err)
	}
	req.Header.Set("Authorization", "Bearer "+apiKey)

	res, err := http.DefaultClient.Do(req)
	if err != nil {
		panic(err)
	}
	defer res.Body.Close()

	var body struct {
		Data []struct {
			Model       string   `json:"model_permaslug"`
			CodingIndex *float64 `json:"coding_index"`
		} `json:"data"`
	}
	if err := json.NewDecoder(res.Body).Decode(&body); err != nil {
		panic(err)
	}
	for _, score := range body.Data {
		if score.CodingIndex != nil {
			fmt.Println(score.Model, *score.CodingIndex)
		}
	}
}
```

```bash cURL
SOURCE="artificial-analysis"
ADDRESS="https://opendunes.com/v1/benchmarks?source=$SOURCE"

curl "$ADDRESS" \
  -H "Authorization: Bearer $OPENDUNES_API_KEY"
```

## Response

`200 OK`. `data` is the list of rows; `source` in each says which source scored it and which
fields it carries.

| Field | Holds |
|---|---|
| `data` | the rows, one per model and benchmark |
| `source` | in each: `artificial-analysis`, `design-arena` or `OpenDunes` |
| `model_permaslug` | in each: the model's ID fixed to one release; on a `design-arena` row that maps to no model, that source's own name for it |
| `display_name` | in each: the model's name, as the source writes it |
| `pricing` | in each: the model's prompt and completion prices in DA, each a string per token, or `null` |

An `artificial-analysis` row adds:

| Field | Holds |
|---|---|
| `intelligence_index` · `coding_index` · `agentic_index` | the model's score on each index, higher is better, or `null` |

A `design-arena` row adds:

| Field | Holds |
|---|---|
| `arena` · `category` | the arena and the category the rating belongs to |
| `elo` | the rating from head-to-head matches |
| `win_rate` | the share of matches it won, in percent |
| `tournament_stats` | how often it placed first to fourth, and the matches played |
| `avg_generation_time_ms` | how long it took to answer, on average, in milliseconds, or `null` |

An `OpenDunes` row adds:

| Field | Holds |
|---|---|
| `benchmark_type` | the benchmark, one of the six `benchmark_type` takes |
| `accuracy` · `accuracy_stddev` | on `gpqa_diamond` and `tau_bench_verified_airline`: the share of tasks answered right, from 0 to 1, and its standard deviation across runs, or `null` for one run |
| `primary_score` · `primary_metric` | on a search benchmark: its headline score, from 0 to 1, and what it measures: `accuracy`, or `f1_by_item` on `search_widesearch` |
| `avg_cost_per_task` | what one task cost on average, in DA, or `null` |
| `avg_latency_per_task_ms` | on a search benchmark: how long one task took on average, in milliseconds, or `null` |
| `search_engine` · `search_surface` | on a search benchmark: the engine and the way the search ran |
| `run_config` | on a search benchmark, when you sent `include_run_config=true`: its `max_agent_turns`, `reasoning_effort` and `temperature` |
| `total_tasks` | the tasks counted across its runs |
| `last_run_timestamp` | when it last ran |

## What it cost

Free. Reading the catalogue is never charged.

## Errors

`error.code` carries the status, and `error.message` says why.

| Status | When |
|---|---|
| 400 | a field is refused, such as a `source` that is not one of the three; `error.metadata.errors` names each field and the values it takes |
| 401 | the key is missing, wrong, disabled or expired |
| 403 | the key answers only from the addresses set on it, and this request came from another |
| 503 | the scores could not be read just now; `Retry-After` says how many seconds to wait |
