Embeddings

An embedding is a list of numbers that stands in for a piece of text. Texts that mean similar things get vectors that point in similar directions, which is what makes search-by-meaning and clustering possible.

from trace_sdk import Trace

client = Trace()

response = client.embeddings.create(input="The cat sat on the mat.")

vector = response.data[0].embedding
print(len(vector))                    # 1536
print(response.usage.credits_spent)

OpenAI's method and OpenAI's response shape. model is optional, as everywhere else in this SDK.

Parameters#

A single string is sent as a one-item batch, so response.data is always a list.

Batching#

Batching is cheaper in wall-clock time and identical in credits — you pay per token either way — so batch whenever you have more than one text.

documents = [
    "Photosynthesis converts light into chemical energy.",
    "The mitochondrion produces ATP.",
    "Rain forms when water vapour condenses.",
]

response = client.embeddings.create(input=documents)

for item in response.data:
    print(item.index, len(item.embedding))

data comes back in the same order as input, and each item carries its index, so you can match vectors to texts either way.

More than 128 texts in one request is a 400 with code: "too_many_inputs". Chunk your corpus:

def embed_all(texts, size=128):
    vectors = []
    for start in range(0, len(texts), size):
        response = client.embeddings.create(input=texts[start : start + size])
        vectors.extend(item.embedding for item in response.data)
    return vectors

The response#

response.object                  # "list"
response.model                   # the model that produced these
response.data                    # list[Embedding]
response.data[0].index           # position in the input
response.data[0].embedding       # list[float]
response.usage.prompt_tokens
response.usage.total_tokens
response.usage.credits_spent     # Trace's addition

Trace's shortcuts, beside the mirrored surface:

response.vectors          # every embedding as a plain list of floats, in order
response.credits_spent    # usage.credits_spent
len(response)             # how many vectors came back
for item in response: ...  # iterating yields Embedding objects

Comparing vectors#

Vectors are not guaranteed to be unit length, so compute cosine similarity by dividing the dot product by the norms. No library required:

import math


def similarity(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    norm = math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b))
    return dot / norm if norm else 0.0


query = client.embeddings.create(input="How do plants make energy?").vectors[0]
corpus = client.embeddings.create(input=documents).vectors

ranked = sorted(zip(documents, corpus), key=lambda pair: similarity(query, pair[1]), reverse=True)
for text, _ in ranked:
    print(text)

Similarity runs from -1 to 1. The number on its own means little; what matters is the ranking against other candidates in the same corpus.

The model#

Trace approves one embedding model, and it is the default. One is deliberate: a tutorial should not have to justify a choice between five.

Confirm the current numbers at runtime rather than trusting this table:

model = client.models.retrieve("openai/text-embedding-3-small")
print(model.dimensions, model.credits_per_1k_tokens)

Passing a chat model id raises ModelNotFoundError listing the embedding models. The two rosters do not overlap.

Cost#

Embeddings settle against provider-reported cost. At roughly 0.02 credits per 1K tokens, a 500-word document costs about 0.013 credits. The Free plan's monthly 2,000-credit allowance covers a corpus far larger than any lesson needs.

Redaction#

Text sent for embedding passes through the same server-side redaction as chat messages — see Authentication. An embedding of a redacted string is not the same vector as an embedding of the original, so do not embed personal data and expect it to match later.