opendunes
Evaluate and optimise
Copy page ▾

Search with embeddings

Find the passages of your own documents that answer a question, then answer from them.

  1. Cut your documents into passages.
  2. Turn passages and question into vectors.
  3. Rank the passages.
  4. Answer from the best passages.

Yacine's Souk Bot answers the customers of a clothing shop in Oran. Its answers about delivery, payment and returns must come from the shop's own rules, not from what a model guesses.

1. Cut your documents into passages

A passage is the piece of text a search returns: one idea, a paragraph or a few.

RuleWhy
cut at paragraphs and headings, not at a fixed lengtha passage holds one whole idea
repeat the last sentence of a passage at the start of the nextan answer that spans two passages is still found
keep the source beside each passagethe answer can say where it comes from

2. Turn passages and question into vectors

Send the passages to /v1/embeddings once, with input_type: "search_document", and store each vector beside its passage: a file, or a database that searches vectors. Each question then goes to the same route with input_type: "search_query". How the route works: Embeddings.

The sample does steps 2 and 3 in one run: it sends the passages, then the question, and prints the two passages closest to it.

import json
import math
import os
import urllib.request

API_KEY = os.environ["OPENDUNES_API_KEY"]
ADDRESS = "https://opendunes.com/v1/embeddings"
MODEL = "voyageai/voyage-multimodal-3.5"
QUESTION = "Vous livrez à Constantine ?"

passages = [
    "Livraison partout en Algérie, sous 2 à 5 jours ouvrables.",
    "Paiement à la livraison, ou par carte Edahabia ou CIB.",
    "Retour accepté sous 7 jours, article non porté.",
    "Boutique ouverte du samedi au jeudi, de 9 h à 19 h.",
]


def embed(texts, input_type):
    request = urllib.request.Request(
        ADDRESS,
        data=json.dumps(
            {
                "model": MODEL,
                "input": texts,
                "input_type": input_type,
            }
        ).encode(),
        headers={
            "Authorization": "Bearer " + API_KEY,
            "Content-Type": "application/json",
        },
    )
    with urllib.request.urlopen(request) as response:
        answer = json.load(response)
    return [item["embedding"] for item in answer["data"]]


def cosine(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    return dot / (math.hypot(*a) * math.hypot(*b))


index = embed(passages, "search_document")
query = embed([QUESTION], "search_query")[0]

scores = [cosine(query, vector) for vector in index]
ranked = sorted(zip(scores, passages), reverse=True)
for score, text in ranked[:2]:
    print(round(score, 3), text)
Don't

Store the passages' vectors from one model and search them with another. Vectors from two models do not compare.

Do

Send the passages and every question through the same model. A new model means sending every passage again.

3. Rank the passages

Two vectors that point the same way hold texts that mean the same. Their cosine measures it: close to 1 for the same meaning, lower as meanings part. The samples score every passage against the question and keep the two highest. Illustrative: the scores depend on the model.

Text
0.612 Livraison partout en Algérie, sous 2 à 5 jours ouvrables.
0.431 Paiement à la livraison, ou par carte Edahabia ou CIB.

Keep the passages that score well above the rest; how many depends on their length and on the room in your prompt. Where a search returns many candidates, a rerank model can reorder them before the answer: see Rerank.

4. Answer from the best passages

Send the passages and the question to a text model, with a rule: answer from these passages only, and say so when they do not hold the answer. The samples carry the two passages the search found.

import os
from openai import OpenAI

API_KEY = os.environ["OPENDUNES_API_KEY"]
BASE_URL = "https://opendunes.com/v1"
MODEL = "anthropic/claude-sonnet-4.5"
QUESTION = "Vous livrez à Constantine ?"
RULE = "Answer from these passages only. If they lack it, say so."

passages = [
    "Livraison partout en Algérie, sous 2 à 5 jours ouvrables.",
    "Paiement à la livraison, ou par carte Edahabia ou CIB.",
]

client = OpenAI(base_url=BASE_URL, api_key=API_KEY)

system = RULE + "\n" + "\n".join("- " + p for p in passages)
reply = client.chat.completions.create(
    model=MODEL,
    messages=[
        {"role": "system", "content": system},
        {"role": "user", "content": QUESTION},
    ],
)

print(reply.choices[0].message.content)
print(reply.usage.cost)
Good to know

A question no passage answers gets "I don't know" from the rule, not a guess. Log those questions: they show what your documents are missing.

What it cost

Each request is charged in DADAThe Algerian dinar, the one currency the docs show.More on Balance and charges → from your balanceBalanceWhat you can spend, in DA.More on Balance and charges →, and usage.cost on each answer gives it. Sending the passages is charged once, on the tokens sent; each question then costs one short vectors request and one text answer. The model's page in Models gives each rate. Every field of both routes: Vectors · Chat.

Last modified on