Learn RAG step by step with real embeddings and visual similarity search. Explore chunking, retrieval, prompts and cited answers in an offline teaching playground.
Find first. Answer with evidence.
Give the AI an open-book exam.
A language model may not know your shop’s rules. RAG finds useful passages from your documents, adds them to the question, and asks a language model to answer using that evidence.
Retrieval finds passages. Augmentation adds them to the prompt. Generation writes the answer.
1Prepare a small library
Split your documents into passages. An embedding model gives each passage a vector: numbers used to compare meaning.
2Find useful passages
Embed the question too. Compare its vector with the passage vectors and retrieve the closest matches.
3Answer with the book open
Give the language model the question and retrieved text. Ask for a supported answer with sources, or an honest “not enough information”.
One example, all the way through
Follow a question through the pipeline
We will use a small fictional shop handbook. Start with the documents and move one step at a time, or choose See full result to experiment directly.
Six curated questions have real bundled embeddings. Arbitrary questions would need a running embedding model.
How many days do I have to return an unused purchase?
How much text each passage keeps.
Keep a little context between neighbours.
How many matches can enter the prompt.
A filter, not an answer-confidence test.
20 prepared chunks · 384 dimensions per vector. Changing chunk settings uses a different prepared index; changing the question reuses it. The walkthrough rewinds whenever inputs change.
Watch the data move
Select a component to inspect it. Play moves data along the active connections; Pause stops the flow.
Scroll the diagram sideways to follow the complete architecture.
Start with your documents. The upper path prepares the index; the lower paths reuse it for each question.
Read the handbook, then press Start or Step forward. Each stage shows only what has been produced so far.
1. Start with source documents
Document preparation
Our entire knowledge source is this fictional Luma Shop handbook. Read the sentences now; the same source labels follow them through the pipeline.
Luma Shop · Returns · 6 sentences
- S1You can return an unused purchase within 30 days of delivery.
- S2The item must be unused, but its packaging may have been opened.
- S3Return shipping is free only when the item arrived damaged.
- S4For all other returns, the customer pays the return shipping cost.
- S5Start a return by emailing support with your order number.
- S6Keep your shipping receipt until the returned package reaches our warehouse.
Luma Shop · Refunds · 6 sentences
- S1A refund starts after our warehouse receives and checks the returned item.
- S2The money reaches your original payment method within five business days after that check.
- S3We send a confirmation email when the refund is issued.
- S4An exchange sends a replacement item instead of returning money.
- S5Gift purchases can be refunded to the original buyer only.
- S6Our support team can look up a refund using the order number.
Luma Shop · Delivery · 6 sentences
- S1Standard delivery takes three to five business days.
- S2Express delivery takes one to two business days.
- S3Orders above 50 dollars qualify for free standard delivery.
- S4Delivery estimates begin when the package leaves our warehouse.
- S5A tracking link is emailed when your order ships.
- S6Contact support if tracking says delivered but the package is missing.
Luma Shop · Warranty · 6 sentences
- S1The warranty covers manufacturing defects for one year after purchase.
- S2Accidental damage and ordinary wear are not covered by the warranty.
- S3Email support a photo and the order number to request a warranty assessment.
- S4A warranty assessment determines whether a repair or replacement is available.
- S5A warranty claim is different from returning an unused item.
- S6Warranty requests do not change the shop's 30-day return window.
Change one decision
Make retrieval succeed—and fail
Load a setup below, then choose See full result. Change the suggested control and compare the evidence, not just the answer’s wording.
Keep a rule together
With 3-sentence chunks and top-k 1, a shipping rule is split. Start with overlap 0, then change overlap to 1: the top passage now carries both parts.
Load the boundary exampleFind both pieces
The return deadline and refund timing live in different documents. Defaults retrieve only part of the answer. Try chunk size 3 and top-k 3 to recover both.
Load the two-document questionA matching topic still cannot supply a policy the handbook never states.
Similar is not sufficient
The handbook discusses delivery and refunds, but never promises a refund for late express delivery. Related passages can score well and still leave the question unanswered.
Load the misleading matchOpen when you are curious
Look inside the numbers
The walkthrough is enough to explain RAG. These sections show where the numbers come from and the decisions a developer makes.
How does text become an embedding?
Think of an embedding as coordinates learned from examples. Related texts often point in similar directions, even when their words differ. The coordinates are not a secret list of labels such as “refund” or “happy”. A single dimension usually has no simple human name.
1. Train the model earlier
Training changes weights. MiniLM’s sentence model was fine-tuned with a contrastive objective: identify paired texts among other examples.
2. Encode new text
The trained weights stay fixed. Running a passage through the model produces its vector. Adding a document does not retrain the model.
3. Store the result
Indexing keeps that vector beside the original text and source. Retrieval can then reuse it for many questions.
This lesson uses all-MiniLM-L6-v2, an embedding model with 384 output numbers. It is separate from the language model that would write an answer. Its encoder follows this path; other models can use different pooling or dimensions.
1Tokenize
Split text into pieces; look up each piece’s token ID.
2Look up
Learned token and position representations form the model’s input.
3Add context
Transformer layers let token representations depend on surrounding text.
4Pool
Combine token vectors into one passage vector using a masked mean.
5Normalize
Divide by the vector’s length so its new length is 1.
A recorded run of the real model
You can return an unused purchase within 30 days of delivery.
The table shows actual token pieces and IDs, followed by the first 4 of 384 final contextual components for each token. The transformer stages above are an illustration; these values were recorded during real inference.
| Token | ID | Mask | d₀ | d₁ | d₂ | d₃ |
|---|---|---|---|---|---|---|
| [CLS] | 101 | 1 | -0.0141 | 0.2861 | 0.1183 | 0.0780 |
| you | 2017 | 1 | 0.1283 | 0.2383 | 0.2545 | -0.0104 |
| can | 2064 | 1 | -0.0727 | -0.0721 | 0.1302 | 0.1085 |
| return | 2709 | 1 | -0.0922 | -0.5472 | 0.3429 | 0.4175 |
| an | 2019 | 1 | -0.1025 | 0.3727 | 0.2451 | 0.0054 |
| unused | 15171 | 1 | -0.2195 | 0.6221 | 0.0982 | 1.2930 |
| purchase | 5309 | 1 | -0.3224 | -0.0232 | 0.0235 | -0.5141 |
| within | 2306 | 1 | -0.0292 | -0.1512 | 0.3072 | 0.0519 |
| 30 | 2382 | 1 | -0.0492 | 0.2753 | 0.4870 | 0.4214 |
| days | 2420 | 1 | -0.2370 | 0.2900 | 0.7704 | 0.6489 |
| of | 1997 | 1 | -0.1486 | 0.0280 | 0.4567 | 0.1366 |
| delivery | 6959 | 1 | -0.4532 | 0.0668 | 0.2915 | -0.0136 |
| . | 1012 | 1 | 0.0612 | 0.2939 | 0.3248 | 0.2190 |
| [SEP] | 102 | 1 | 0.2612 | 0.2043 | 0.5123 | 0.2693 |
The mask is 1 for tokens included in the mean and 0 for padding. This short recorded input needed no padding. Non-padding special tokens such as [CLS] and [SEP] are included, matching this model’s pooling operation.
pooled[d] = sum(mask[t] × token[t][d]) / sum(mask[t])
first four pooled values = [-0.0921, 0.1345, 0.3116, 0.2222]
length across all 384 pooled values = 5.4588
normalized[d] = pooled[d] / 5.4588
first four normalized values = [-0.0169, 0.0246, 0.0571, 0.0407]
For example, component 0 is -0.0921 ÷ 5.4588 = -0.0169. The search still compares all 384 components.
Query means “your search question” in this lesson. It is not the same object as an attention layer’s internal query, key, and value vectors. Explore self-attention separately.
Recorded model and reproducibility
- Model
- sentence-transformers/all-MiniLM-L6-v2
- Artifact revision
- onnx-sha256:4f148ba8ae9c2c7fbee4af2b132db8d06c6a6545b47fc83bbb98c3d22b8393e6
- Corpus fingerprint
- 5631ca00533391824d77bbb1362937be47f93c075f7b23849f562cbc5d2d2224
- Generation packages
- numpy 2.5.5 · onnxruntime 1.32.0 · tokenizers 0.23.3
The vectors are bundled. No model is downloaded or executed in your browser. The generation script verifies the model artifacts before creating the fixture.
How does cosine similarity compare two vectors?
Cosine compares directions. Arrows pointing the same way score 1, perpendicular arrows score 0, and opposite arrows score −1. For embeddings, a higher score often signals related meaning; it does not prove that a passage answers the question.
cosine(q, c) = dot(q, c) / (length(q) × length(c))
dot = 1 × 3.00 + 0 × 4.00 = 3.00
length(q) = 1; length(c) = 5
cosine = 3.00 / (1 × 5) = 0.60
Normalizing divides a vector by its length without changing its direction. If both vectors have length 1, their dot product already equals cosine similarity. The real search uses complete 384-dimensional vectors, not the four coordinates shown in the preview.
0.6 is a similarity score, not “60% certain”. The scale is model-dependent. A minimum cutoff may remove weak matches but cannot establish that the remaining passages contain the answer.
What would an experienced developer check?
- Keep enough context
- Tiny chunks can separate a rule from its condition. Large chunks can mix several topics. This lesson counts sentences; real splitters also consider headings, structure, and model token limits. Overlap preserves continuity but stores duplicate text and can crowd useful evidence out of top-k.
- Retrieve enough, without filling the prompt with noise
- Increasing top-k can recover another part of the answer, but also adds irrelevant text and token cost. Respect the answer model’s context limit; reserve room for the question, instructions, and output.
- Use a compatible embedding space
- Questions and documents must use a compatible model and configuration. Some retrieval models require different query and document prefixes. Changing models usually means re-embedding the documents. This MiniLM setup limits input to 256 word pieces; our short examples fit without truncation.
- Choose retrieval for the job
- This demo compares every chunk exactly. Large indexes often use approximate nearest-neighbour search, trading some recall for faster searches. Exact product IDs can benefit from keyword search; hybrid retrieval combines lexical and semantic results. A reranker can reorder retrieved candidates, but cannot recover a passage absent from its candidate set.
- Keep sources current and access controlled
- Update or delete index entries when source documents change. Filter documents by the user’s permissions before search and ranking, so inaccessible passages cannot enter the results or prompt. Treat retrieved documents as untrusted evidence, never as instructions that override the application.
- Evaluate finding and answering separately
- For questions with labeled evidence, recall@k measures how much expected evidence appears in the retrieved top-k. Then check whether the answer’s claims are supported and citations point to the right passages. Also measure latency and cost. A fluent answer and a high similarity score are not quality tests.
Sources: MiniLM model card · Cosine and normalization · Semantic search · Retrieve and rerank.
How it works: RAG: Retrieval-Augmented Generation
Step by step
- Current pipeline stage
- Vector comparisons
- Selected evidence
- Not selected
Press Start or Step: the step the animation is on lights up below.
- 1
Start with source documents
The shop's handbook holds its rules. RAG can only retrieve from information made available to it.
documents = load_sources() - 2
Split each document into passages
A small passage is easier to retrieve precisely. Overlap repeats some text so a rule is less likely to be separated from its explanation.
chunks = split(documents, size, overlap) - 3
Turn passages into vectors
Run the already trained embedding model on each chunk. Here MiniLM produces 384 numbers per passage; its learned dimensions do not have simple names.
vectors = embed(chunks.text) - 4
Store vectors beside their sources
Keep each vector with the passage text and its source identity. Reuse this prepared index for new questions; update it when its documents or embedding setup change.
index = store(chunks, vectors, sources) - 5
Embed the question
Represent the search question in a compatible embedding space. This search question is different from the internal query, key, and value vectors used by attention.
question_vector = embed_query(question) - 6
Compare every chunk
Cosine similarity compares vector directions. This demo uses every dimension, then sorts the scores. A high score is a useful match, not a probability that the passage answers the question.
ranked = sort_by_cosine(question_vector, index) - 7
Keep the best eligible passages
Apply the optional minimum score and keep up to top-k passages. A stricter cutoff or smaller top-k can remove useful evidence as well as irrelevant text.
context = eligible(ranked, cutoff)[:top_k] - 8
Put evidence beside the question
Send the selected text, source labels, question, and instructions to the answer model. Treat document text as evidence, not instructions to follow.
prompt = instructions + context + question - 9
Answer what the evidence supports
This simulation displays only authored claims whose required sentences were retrieved. It cites those passages and says when evidence is partial or missing. Real LLMs do not automatically have this evidence check.
answer = generate_with_sources(prompt) # simulated here
Think of an open-book exam. Before answering a question, find the useful pages and place them beside the question. RAG does those two jobs: retrieve useful passages, then ask a language model to answer using them.
First prepare the documents: split them into passages called chunks, turn each chunk into a vector (a list of numbers), and store that vector with its text and source. A trained embedding model creates the vectors. Adding a document runs that model; it does not train it again.
For each question, create a compatible question vector, compare it with the chunk vectors, and retrieve the best matches. Put the original passages into the answer model's prompt. The answer model reads text, rather than the embedding numbers.
This page searches real, recorded MiniLM embeddings in all 384 dimensions. The answer is an authored simulation that checks known evidence for these six questions. It demonstrates full, partial, and missing evidence; a real language model still needs evaluation because it can make unsupported claims.
Where you'll meet it
- Answering questions about a company's handbook, product documentation, or a collection of research papers.
- Using information that changes without retraining the answer model: update the documents and their index instead.
- Giving readers a route back to the source of an answer. A citation helps them check a claim; the citation alone does not prove it is supported.
For n chunks and d vector dimensions, this exact search compares every vector and sorts all n scores. Embedding computation and answer generation have their own model costs. Large systems often use approximate indexes; RAG does not require a particular vector database.