Embeddings: model-scoped representations, not semantic truth

An embedding is a learned representation produced by one model and input contract. Distance is meaningful only under the model's documented preprocessing, similarity function, version, and evaluated task distribution.

Mental model

• Embeddings support efficient semantic comparison only within the learned representation and task setup. Describe embeddings as learned, model-scoped signals rather than coordinates of objective meaning.
• Cosine similarity is used by Sentence-BERT for evaluated sentence comparison; it is not a calibrated probability. Remove universal score bands and require task-specific labeled calibration.
• Provider embedding models and dimensions are versioned product contracts that can change. Pin model and preprocessing metadata and treat migration as re-indexing.
• Vector and embedding systems introduce access-control, poisoning, and sensitive-information risks. Put policy before encoding and apply authorization again at retrieval.

Guarantees and boundaries

• A stored vector is tied to one explicit schema and source version.
• Normalized vectors make cosine ranking equivalent to dot-product ranking only when both sides are normalized.
• Approximate search quality must be measured against labeled or exact-search ground truth.

Diagram scenarios

Versioned document encoding. The stored vector is meaningful only with its pinned preprocessing, model, dimension, and metric.

Similarity is an evaluated ranking signal. Cosine, dot product, and Euclidean distance are mathematical choices, not universal relevance thresholds.

Shadow re-embedding migration. A model change creates a new vector schema and index; it does not mutate old vectors in place.

Privacy, truncation, and drift failure path. Encoding can leak sensitive content, silently truncate evidence, or drift after a model change.

Architecture decision

Version the model, preprocessing, vector dimension, normalization, and distance metric as one schema. Build task-specific retrieval evaluations before choosing thresholds. Redact or reject sensitive inputs before an external embedding call, and re-embed into a new index before changing the schema.

Failure modes

• Silent truncation removes evidence while producing a syntactically valid vector.
• A mixed-version index can return meaningless rankings without throwing an error.
• Derived vectors can still expose sensitive attributes and must follow retention and access policy.

Operational checklist

• Pin model, tokenizer, input role, dimension, normalization, and metric.
• Keep source IDs and content versions for deletion and re-embedding.
• Evaluate thresholds and migrations on representative slices.
• Never log raw sensitive inputs merely to debug vectors.

Related concept

Primary sources

1. https://arxiv.org/abs/1908.10084
2. https://platform.openai.com/docs/guides/embeddings
3. https://genai.owasp.org/llm-top-10/