AI, Actually

Embeddings Are Why AI Finds the Right Answer Without Your Exact Words

Published November 11, 2026 — 3 min read

TL;DR: Embeddings turn text into coordinates, so a search for "refund for a broken laptop" can find a document titled "Returning defective hardware." If your team uses any "smart search" at work, this one idea explains both why it works and why it sometimes confidently returns the wrong thing.

Key Insight

Most people picture AI search as a model that reads your question and understands it. The part that finds your document usually does something humbler. It converts your question into a list of numbers, converts every document into a list of numbers, and returns the documents whose numbers sit closest to yours.

That list of numbers is an embedding. Think of it as a street address for meaning. "Invoice dispute" and "billing disagreement" share almost no words, but they land on the same block. "Invoice dispute" and "invoice template" share a word and land in different neighborhoods.

An embedding does not understand anything, though. It records which kinds of text tend to show up in similar contexts. That is enough to be extremely useful, and it is also why the technology fails in specific, predictable ways. Treating it as comprehension is how teams end up trusting it in places it was never good at.

Why Teams Miss This

Two mistakes show up again and again in enterprise rollouts.

The first is assuming a smart search replaces keyword search. It does not. Embeddings are weak at exact matches. Search for a part number like "KX-4471-B" or a contract clause ID and the embedding of that string is close to many other part numbers, because to the model they all look like the same sort of thing. Keyword search nails this case in milliseconds. Teams that switch entirely to embeddings often see their power users complain first, because power users search with exact identifiers.

The second is assuming closeness means correctness. Embedding search always returns the nearest results, even when nothing nearby is relevant. Ask your internal knowledge base about a policy that does not exist and it will still hand back the three closest paragraphs, with a similarity score that looks reassuring. A downstream model then writes a fluent answer on top of irrelevant text. The failure looks like a hallucination, but the cause is a retrieval step that was never told it was allowed to find nothing.

How to Actually Do It

You do not need to train anything. Hosted embedding models from OpenAI, Google, Cohere and others turn text into vectors with one API call. The working pattern in production looks like this:

  1. Split documents into chunks of a few hundred words, so each vector represents one idea instead of a whole handbook.
  2. Embed each chunk once and store the vectors next to the original text in a vector database or a Postgres table with the pgvector extension.
  3. At query time, embed the question with the same model and fetch the closest chunks.
  4. Run a keyword search in parallel and merge both result lists. This is called hybrid search, and it covers the exact-match gap.
  5. Set a minimum similarity cutoff and let the system answer "I did not find anything" when nothing clears it.

The core of steps 2 and 3 fits in a few lines:

from openai import OpenAI
import numpy as np

client = OpenAI()

def embed(texts):
    resp = client.embeddings.create(model="text-embedding-3-small", input=texts)
    return np.array([d.embedding for d in resp.data])

docs = ["Returning defective hardware", "Invoice template guidelines", "Parental leave policy"]
doc_vecs = embed(docs)

query_vec = embed(["refund for a broken laptop"])[0]
scores = doc_vecs @ query_vec / (
    np.linalg.norm(doc_vecs, axis=1) * np.linalg.norm(query_vec)
)

best = scores.argmax()
if scores[best] > 0.35:
    print(docs[best], scores[best])
else:
    print("No confident match")

The cutoff of 0.35 is a placeholder. Score scales differ between embedding models, so pick yours by running 30 real questions your users ask and looking at where relevant results stop and noise starts.

What We've Learned

Before you buy or build a smart search, run one cheap experiment. Collect 30 real queries from your team, including a few with exact identifiers and a few about topics your documents do not cover. Run them through keyword search, embedding search, and a hybrid of the two. Count how many each one gets right and how many it answers confidently when it should have said nothing.

In our experience the hybrid wins on the first count and the cutoff wins on the second. Neither requires a bigger model. Both require someone to look at the actual results for an afternoon.

Sources

Have a specific workflow in mind?

Bring it to a Quick Scan — a live working session where we'll tell you honestly whether it should be an agent, a workflow, or left alone, before you spend a dollar building it. You get 3 prioritized recommendations on the call, a one-page summary after, and the $500 credited toward any engagement within 30 days.

Tell us what you'd hand off →

See AI Agent Consulting·Book an intro call

Get new posts + practical agent-ops notes

One email when something new goes up. No nurture sequence, no spam — unsubscribe whenever you want.

Thanks — you're on the list.