Applications
RAG & Vector Search
IntermediateA trained model only knows what it saw before its cutoff. Retrieval-Augmented Generation fixes that: look up the right facts from your own data, then let the model answer with them in hand. Under the hood it is embeddings + vector search - turning meaning into numbers and search into geometry. This page builds from why RAG exists, to the real cosine numbers behind it, to billion-scale infrastructure.
Why RAG exists
A language model is a closed-book exam: it answers from memory frozen at its training cutoff. Ask about yesterday's news, an internal wiki, or a customer's contract and it has three options - say it doesn't know, or worse, confidently make something up (a hallucination). RAG turns it into an open-book exam: before answering, the system retrieves the most relevant passages from a trusted source and pastes them into the prompt. The model then answers from the page in front of it, with citations.
Knowledge cutoff
Models don't know anything after training. RAG injects fresh, current facts at query time.
Hallucination
Ungrounded models invent plausible-sounding answers. Retrieved sources keep them honest and citable.
Private data
Your contracts, tickets, and wikis were never in training. RAG reaches them without retraining.
The one-line mental model: RAG = look it up first, then answer.
Embeddings: turning meaning into numbers
To “look something up” by meaning, you first have to make meaning measurable. An embedding model reads a piece of text (or an image) and outputs a list of numbers - a vector - that captures what it's about. The magic property: texts with similar meaning produce vectors that sit close together in that space, even when they share no words. So “car” and “automobile” land near each other; “car” and “banana” land far apart.
How long is the list? It depends on the model. Below are the dimension counts you'll meet most often.
| Embedding model | Dimensions | Notes |
|---|---|---|
| OpenAI text-embedding-3-small | 1536 | default API embedder; Matryoshka-shortenable |
| OpenAI text-embedding-3-large | 3072 | ~64.6% MTEB; highest quality OpenAI |
| Cohere embed v3 | 1024 | strong multilingual + reranker pairing |
| BGE-large / MPNet | 1024 / 768 | top open-weight encoders |
| all-MiniLM-L6-v2 | 384 | tiny, fast, runs in the browser |
To compare two vectors we use cosine similarity: the cosine of the angle between them, from −1 (opposite) through 0 (unrelated) to 1 (identical meaning). For example cos(king, queen) ≈ 0.86 (related) versus cos(king, banana) ≈ 0.12 (unrelated). The geometry is so faithful that vector arithmetic works: king − man + woman ≈ queen. Pick a query in the demo and watch the real numbers.
Search is geometry: pick a query, find its nearest neighbors
Every word below is a tiny 4-D vector. Lines show cosine similarity to your query, computed live from the vectors. Green = close meaning, gray = unrelated.
Cosine similarity to “king”
- mantop-30.986
- doctortop-30.895
- queentop-30.836
- nurse0.734
- woman0.537
- truck0.217
- apple0.209
- car0.145
- banana0.133
cos(A,B) = A·B / (‖A‖ ‖B‖)
Range −1 to 1: 1 = same meaning, 0 = unrelated, −1 = opposite. Most real text pairs land between 0.0 and 0.9.
Notice king sits closest to queen and far from banana - no keywords matched, only geometry. That is the whole idea behind vector search: similar meaning lands close together, so retrieval becomes a nearest-neighbor lookup.
Vector search fundamentals
Once everything is a vector, retrieval is just nearest-neighbor search: embed the query, then find the stored vectors closest to it. The first question is how you measure “close.”
Cosine similarity
Angle between vectors, ignoring length. Range −1..1. The default for text embeddings because magnitude carries little meaning.
Dot product
Cosine × the two magnitudes. For L2-normalized vectors it equals cosine exactly - which is why most engines normalize first.
Euclidean (L2)
Straight-line distance. For normalized vectors it is just a monotonic re-mapping of cosine, so the ranking is identical.
The second question is how you find those neighbors without checking all billion vectors every time. That is the difference between exact and approximate search.
Brute-force kNN (exact)
Compare the query to every vector. 100% recall, but O(N·d) per query - only viable below roughly 1M vectors.
HNSW (graph)
A multi-layer proximity graph you greedily walk toward the query. Best recall + latency, but the whole graph lives in RAM. Knobs: M (links per node, ~16–64) and ef_search (higher = better recall, slower).
IVF (cluster + probe)
Cluster vectors into buckets; at query time only probe the nprobe nearest buckets. Cheaper memory than HNSW; nprobe trades recall for speed.
Quantization (compress)
Shrink each vector. Scalar int8 ≈ 4× smaller; product quantization (PQ) 10–30×+ - a 1536-d float (~6 KB) drops to ~96 bytes. Layers on top of HNSW/IVF.
Approximate Nearest Neighbor (ANN) indexes give up a tiny bit of accuracy to go orders of magnitude faster - typically 95–99% recall at sub-10ms. The demo below lets you feel that tradeoff at different corpus sizes.
The recall ↔ latency ↔ cost triangle
Exact search is always 100% accurate but scans every vector. ANN trades a sliver of recall for orders-of-magnitude speed. Drag the corpus up and the index knob right to feel the squeeze. Illustrative model.
ANN recall
Exact = 100% · aggressive ANN dips toward ~94%
Query latency (log scale)
Under ~1M vectors, exact is still fast enough.
Index memory (1536-d)
- float326.1 GB
- int8 (4×)1.5 GB
- PQ (~20×)307 MB
1536 × 4 B = 6 KB per vector. Quantization is what keeps billion-scale indexes affordable.
Pick any two corners of the triangle and the third gives way. Billion-scale production indexes lean on ANN + quantization, and GPU-accelerated builds (NVIDIA cuVS / CAGRA) to make the index in hours instead of days.
The RAG pipeline, end to end
RAG has two halves. Offline (ingest): documents are split into chunks (~200–500 tokens with 10–20% overlap so ideas aren't cut mid-sentence), each chunk is embedded, and the vectors are stored in the index. Online (query): the question is embedded, the top-k chunks (usually 3–20) are retrieved, an optional reranker - a cross-encoder that reads query and chunk together - reorders them for precision, and the survivors are stuffed into the prompt for the LLM to answer with citations.
One query, end to end
User query
“How does the KV cache lower inference cost?”
Query vector (1536-d, truncated)
system 40 + query 9 + chunks 0. Longer chunks → longer prefill → a bigger KV cache.
Retrieved chunks (vector score)
Decode is memory-bandwidth bound; reusing cached KV turns an O(n^2) prefill into O(1) per new token.
140 tokens
The KV cache stores the keys and values of every past token so they are not recomputed each step.
120 tokens
HBM bandwidth and VRAM capacity set the ceiling on batch size and context length for a given GPU.
110 tokens
Watch the chunk order change when rerank runs - the cross-encoder demotes the loosely-related GPU chunk that vector search ranked highly. Better retrieval means fewer, sharper chunks, which directly lowers prefill cost and KV-cache size downstream.
Why an ML engineer cares about the token meter: every retrieved chunk lengthens the prompt, which lengthens prefill and grows the KV cache. Better retrieval - fewer, sharper chunks - is not just more accurate, it is cheaper to serve.
The technology landscape
A RAG stack is assembled from four kinds of building block. Here are the names you'll hear in vendor calls - what each one is, and why it matters.
Vector databases
- Pinecone - Fully-managed serverless vector DB; zero-ops scaling.
- Weaviate - Open-source DB with built-in hybrid search & modules.
- Qdrant - Rust-based, fast HNSW with payload filtering.
- Milvus - Cloud-native, billion-scale; GPU indexing via cuVS.
- pgvector - Postgres extension - vectors next to your relational data.
- FAISS - Meta's library; the de-facto ANN engine many DBs embed.
Embedding models
- OpenAI - text-embedding-3 small/large; simple API, strong quality.
- Cohere - embed v3, multilingual; pairs with Cohere Rerank.
- sentence-transformers - Open framework; MiniLM, MPNet, and friends.
- BGE (BAAI) - Top open-weight encoders for retrieval.
Rerankers
- Cohere Rerank - Hosted cross-encoder; re-scores top-k for precision.
- Cross-encoders - Read query + chunk together; accurate but slower, so used only on top-k.
GPU acceleration
- NVIDIA cuVS - CUDA vector search (successor to RAFT); GPU IVF-Flat/IVF-PQ.
- CAGRA - GPU-native graph index; ~10–95× faster billion-scale builds vs CPU.
- Integrations - Plugs into Milvus, FAISS, Lucene, Elasticsearch, Weaviate.
Scale & cost
Every production decision lives inside one triangle: recall ↔ latency ↔ cost. You can have any two; the third bends. The dominant cost driver is memory, because the highest-recall index (HNSW) wants the whole graph resident in RAM.
When to use what
- < ~1M vectors: exact brute force or pgvector is simple, 100% recall, fast enough.
- 1M–1B+ vectors: HNSW or IVF + quantization; ANN p99 in single-digit to low-tens of ms.
- Billion-scale builds: GPU ANN (cuVS / CAGRA) cuts index build from days to hours.
Use case: when and where to position RAG
Theory lands when you can see it on a real desk. Here's a composite enterprise scenario - a large insurer's claims-and-service copilot - that shows the moment RAG earns its place: the same model answers a customer question with and without retrieval over the company's own corpus, data lake, and data warehouse. Toggle it and watch a plausible-but-wrong answer become a grounded, citable one.
In the wild - a P&C insurer's claims & service copilot
A top-10 property & casualty insurer's contact-center agents and adjusters were drowning in policy language and giving inconsistent, sometimes wrong answers. RAG over their own data turns a generic chatbot into a grounded, auditable expert. Toggle RAG to see the difference on a live question.
The data foundation it retrieves from
Policy wordings, endorsements, underwriting guidelines, 10 yrs of adjuster notes (PDF/Word in SharePoint) - chunked & embedded.
FNOL call transcripts, claim documents, telematics, photo metadata (Parquet on object store) - semi-structured.
Policy master, coverage limits, deductibles, claim status, customer tier, state - the authoritative numbers.
Current claim stage and fraud flags, joined in at query time.
Pick a question an agent asks
“A policyholder's basement flooded after a pipe burst. Is it covered, and what's their deductible?”
Yes - sudden, accidental discharge from a burst pipe is covered under Section I, Coverage A of the HO-3 form (¶12, “Water Damage”). This policyholder also carries the Water Backup & Sump Overflow endorsement (HO-04-95), adding $10,000 of coverage. Their all-perils deductible is $1,500 - not the $2,500 wind/hail deductible. External surface flooding remains excluded. The claim is open and currently in “inspection.”
Retrieved from
- Enterprise corpusHO-3 wording ¶12 · endorsement HO-04-95
- Data warehousedeductible $1,500 · coverage limits · form on file
- Data lakeFNOL transcript · first adjuster note
- Live statusclaim #CLM-48217 → status: inspection
Outcome after rollout
Average handle time
−38%
11.5 min → 7.1 min
First-contact resolution
+14 pts
66% → 80%
New-adjuster ramp
−50%
6 mo → 3 mo
Incorrect coverage statements (QA)
−72%
baseline → −72%
The point: the model didn't get smarter - the data got closer. Answers became specific, correct, and auditable because every claim is grounded in the company's own corpus, lake, and warehouse. That is why, at enterprise scale, RAG is a data-platform decision, not just a prompt.
Composite, illustrative scenario; figures are representative of published enterprise RAG deployments, not a specific customer.
The separate vector DB problem
Notice what the stack above quietly assumes: the embeddings live in a dedicated vector database - Pinecone, Milvus, Weaviate, or pgvector - that is separate from where the source records actually live (your data lake, warehouse, or object store). That split is the default today, and it quietly creates work.
A second system to run
You now operate, scale, secure, and pay for a vector store alongside your system of record - two clusters, two permission models.
Sync drift
Every time a source record changes, its embedding must be re-computed and re-loaded. Until it is, retrieval returns stale results.
Copy + ETL
Text is copied out, embedded, and copied into the index - a pipeline that can break, lag, or silently fall out of step with the source.
The obvious question is whether vectors could just live next to the data they describe - one system, no copy, no drift. That is exactly the step the VAST VectorStore page picks up.
Retrieval is upstream of everything you serve
The chunks you retrieve become the prompt you serve, so RAG quality flows straight into prefill cost and KV-cache size. And at scale, vector search is a data-platform problem - which is exactly where VAST VectorStore runs search native to the data it lives on.