Not every piece of work needs your most capable model. A token-efficient review agent uses a lightweight model as a first-pass filter — discarding obviously acceptable items, flagging the few that need deeper scrutiny, and only then routing to the expensive reviewer. The result is the same quality bar at a fraction of the DA cost.
Input → Triage model (cheap, fast) → Pass / Flag
│
Flag only
│
Review model (capable)
│
Final verdict
The triage model handles the easy cases (roughly 80–90% of inputs in most workloads). The review model only sees the hard ones.
import os
from openai import OpenAI
client = OpenAI(
base_url="https://opendunes.com/api/v1",
api_key=os.environ["OPENDUNES_API_KEY"],
)
TRIAGE_MODEL = "meta-llama/llama-4-maverick"
REVIEW_MODEL = "anthropic/claude-opus-4.8"
def triage(item: str) -> tuple[str, str]:
"""Returns ('pass'|'flag', reason)."""
resp = client.chat.completions.create(
model=TRIAGE_MODEL,
messages=[
{"role": "system", "content": TRIAGE_PROMPT},
{"role": "user", "content": item},
],
max_tokens=64,
temperature=0,
)
raw = (resp.choices[0].message.content or "").strip().lower()
if raw.startswith("pass"):
return "pass", raw
return "flag", raw
def deep_review(item: str) -> str:
"""Full review by the capable model."""
resp = client.chat.completions.create(
model=REVIEW_MODEL,
messages=[
{"role": "system", "content": REVIEW_PROMPT},
{"role": "user", "content": item},
],
max_tokens=512,
)
return resp.choices[0].message.content or ""
def review_batch(items: list[str]) -> list[dict]:
results = []
for item in items:
verdict, triage_note = triage(item)
if verdict == "pass":
results.append({"item": item, "verdict": "pass", "note": triage_note, "model": TRIAGE_MODEL})
else:
full_review = deep_review(item)
results.append({"item": item, "verdict": "flag", "note": full_review, "model": REVIEW_MODEL})
return results
Keep the triage prompt extremely tight — the model should output at most two words:
TRIAGE_PROMPT = """
You are a triage filter. Read the input and respond with exactly one of:
pass — the item is clearly fine and needs no further review
flag — the item is ambiguous, risky, or needs a closer look
No other output. No explanation.
"""
REVIEW_PROMPT = """
You are a careful reviewer. Analyze the item in detail.
State: verdict (approve / reject / revise), confidence (0–1), and a one-paragraph rationale.
"""
Assume a batch of 100 items, each ~200 tokens of input:
| Approach | Model | Tokens | Approx. cost |
|---|
| Review all items | anthropic/claude-opus-4.8 | 20,000 input + output | ~45 DA |
| Triage 100 → review 15 flagged | Mixed | ~22,000 triage + 3,000 review | ~9 DA |
The exact breakdown depends on your triage pass rate and the current DA pricing shown in the model catalog. At an 85% pass rate, you're spending roughly 80% less on the review workload.
The triage prompt is the lever that controls the pass rate. If your downstream review is finding too many approves (the triage is too strict), relax the language. If it's missing real issues (triage passes too much), tighten it.
A quick calibration loop:
def calibrate(sample: list[tuple[str, str]], target_recall: float = 0.95):
"""
sample: list of (item, ground_truth) where ground_truth is 'pass' or 'flag'
Returns: pass rate and recall on flags
"""
triaged = [(item, triage(item)[0], truth) for item, truth in sample]
flags_in_truth = sum(1 for _, _, t in triaged if t == "flag")
flags_caught = sum(1 for _, v, t in triaged if t == "flag" and v == "flag")
recall = flags_caught / flags_in_truth if flags_in_truth else 1.0
pass_rate = sum(1 for _, v, _ in triaged if v == "pass") / len(triaged)
print(f"Pass rate: {pass_rate:.0%} | Flag recall: {recall:.0%}")
if recall < target_recall:
print("Triage is too permissive — tighten the prompt.")
return pass_rate, recall
Run calibration on a labeled sample of 50–100 real inputs before deploying to production.
- Three tiers: add a mid-tier model between triage and deep review for items that are neither obvious passes nor clear flags.
- Async review: triage synchronously and queue flagged items for async deep review, returning the triage verdict immediately to the caller.
- Confidence threshold: instead of a binary pass/flag, have the triage model output a score (0–1) and flag anything below 0.8.