RAG: Retrieval-Augmented Generation

Learn RAG step by step with real embeddings and visual similarity search. Explore chunking, retrieval, prompts and cited answers in an offline teaching playground.

Find first. Answer with evidence.

Give the AI an open-book exam.

A language model may not know your shop’s rules. RAG finds useful passages from your documents, adds them to the question, and asks a language model to answer using that evidence.

Retrieval finds passages. Augmentation adds them to the prompt. Generation writes the answer.

Real embedding-based searchSimulated answer generationNo keys or model downloads
HandbookPassage 1chunkPassage 2↺ overlapPassage 3↺ overlap

1Prepare a small library

Split your documents into passages. An embedding model gives each passage a vector: numbers used to compare meaning.

Your questionquery vector✓ 0.84Retrieved✓ 0.71Retrieved0.18Out

2Find useful passages

Embed the question too. Compare its vector with the passage vectors and retrieve the closest matches.

QuestionPassages[1] [2]Language modelgrounded synthesisAnswer + citations [1] [2]

3Answer with the book open

Give the language model the question and retrieved text. Ask for a supported answer with sources, or an honest “not enough information”.

One example, all the way through

Follow a question through the pipeline

We will use a small fictional shop handbook. Start with the documents and move one step at a time, or choose See full result to experiment directly.

Six curated questions have real bundled embeddings. Arbitrary questions would need a running embedding model.

How many days do I have to return an unused purchase?

How much text each passage keeps.

Keep a little context between neighbours.

How many matches can enter the prompt.

A filter, not an answer-confidence test.

20 prepared chunks · 384 dimensions per vector. Changing chunk settings uses a different prepared index; changing the question reuses it. The walkthrough rewinds whenever inputs change.

Watch the data move

Select a component to inspect it. Play moves data along the active connections; Pause stops the flow.

Document indexingQuestion & retrievalAnswer generation

Scroll the diagram sideways to follow the complete architecture.

Your questionHow many days do I have to return an unused purchase?

Start with your documents. The upper path prepares the index; the lower paths reuse it for each question.

Step 0 / 9
1500 ms

Read the handbook, then press Start or Step forward. Each stage shows only what has been produced so far.

1. Start with source documents

Document preparation

Our entire knowledge source is this fictional Luma Shop handbook. Read the sentences now; the same source labels follow them through the pipeline.

Luma Shop · Returns · 6 sentences
  1. S1You can return an unused purchase within 30 days of delivery.
  2. S2The item must be unused, but its packaging may have been opened.
  3. S3Return shipping is free only when the item arrived damaged.
  4. S4For all other returns, the customer pays the return shipping cost.
  5. S5Start a return by emailing support with your order number.
  6. S6Keep your shipping receipt until the returned package reaches our warehouse.
Luma Shop · Refunds · 6 sentences
  1. S1A refund starts after our warehouse receives and checks the returned item.
  2. S2The money reaches your original payment method within five business days after that check.
  3. S3We send a confirmation email when the refund is issued.
  4. S4An exchange sends a replacement item instead of returning money.
  5. S5Gift purchases can be refunded to the original buyer only.
  6. S6Our support team can look up a refund using the order number.
Luma Shop · Delivery · 6 sentences
  1. S1Standard delivery takes three to five business days.
  2. S2Express delivery takes one to two business days.
  3. S3Orders above 50 dollars qualify for free standard delivery.
  4. S4Delivery estimates begin when the package leaves our warehouse.
  5. S5A tracking link is emailed when your order ships.
  6. S6Contact support if tracking says delivered but the package is missing.
Luma Shop · Warranty · 6 sentences
  1. S1The warranty covers manufacturing defects for one year after purchase.
  2. S2Accidental damage and ordinary wear are not covered by the warranty.
  3. S3Email support a photo and the order number to request a warranty assessment.
  4. S4A warranty assessment determines whether a repair or replacement is available.
  5. S5A warranty claim is different from returning an unused item.
  6. S6Warranty requests do not change the shop's 30-day return window.

Change one decision

Make retrieval succeed—and fail

Load a setup below, then choose See full result. Change the suggested control and compare the evidence, not just the answer’s wording.

Overlap 0Missing evidence
Overlap 1Supported

Keep a rule together

With 3-sentence chunks and top-k 1, a shipping rule is split. Start with overlap 0, then change overlap to 1: the top passage now carries both parts.

Load the boundary example
Default retrievalPartial evidence
Chunk 3 · top-k 3Supported

Find both pieces

The return deadline and refund timing live in different documents. Defaults retrieve only part of the answer. Try chunk size 3 and top-k 3 to recover both.

Load the two-document question
Best cosine 0.627Missing evidence

A matching topic still cannot supply a policy the handbook never states.

Similar is not sufficient

The handbook discusses delivery and refunds, but never promises a refund for late express delivery. Related passages can score well and still leave the question unanswered.

Load the misleading match

Open when you are curious

Look inside the numbers

The walkthrough is enough to explain RAG. These sections show where the numbers come from and the decisions a developer makes.

How does text become an embedding?

Think of an embedding as coordinates learned from examples. Related texts often point in similar directions, even when their words differ. The coordinates are not a secret list of labels such as “refund” or “happy”. A single dimension usually has no simple human name.

1. Train the model earlier

Training changes weights. MiniLM’s sentence model was fine-tuned with a contrastive objective: identify paired texts among other examples.

2. Encode new text

The trained weights stay fixed. Running a passage through the model produces its vector. Adding a document does not retrain the model.

3. Store the result

Indexing keeps that vector beside the original text and source. Retrieval can then reuse it for many questions.

This lesson uses all-MiniLM-L6-v2, an embedding model with 384 output numbers. It is separate from the language model that would write an answer. Its encoder follows this path; other models can use different pooling or dimensions.

  1. 1Tokenize

    Split text into pieces; look up each piece’s token ID.

  2. 2Look up

    Learned token and position representations form the model’s input.

  3. 3Add context

    Transformer layers let token representations depend on surrounding text.

  4. 4Pool

    Combine token vectors into one passage vector using a masked mean.

  5. 5Normalize

    Divide by the vector’s length so its new length is 1.

A recorded run of the real model

You can return an unused purchase within 30 days of delivery.

The table shows actual token pieces and IDs, followed by the first 4 of 384 final contextual components for each token. The transformer stages above are an illustration; these values were recorded during real inference.

Recorded MiniLM tokens and four final contextual vector components
TokenIDMaskd₀d₁d₂d₃
[CLS]1011-0.01410.28610.11830.0780
you201710.12830.23830.2545-0.0104
can20641-0.0727-0.07210.13020.1085
return27091-0.0922-0.54720.34290.4175
an20191-0.10250.37270.24510.0054
unused151711-0.21950.62210.09821.2930
purchase53091-0.3224-0.02320.0235-0.5141
within23061-0.0292-0.15120.30720.0519
3023821-0.04920.27530.48700.4214
days24201-0.23700.29000.77040.6489
of19971-0.14860.02800.45670.1366
delivery69591-0.45320.06680.2915-0.0136
.101210.06120.29390.32480.2190
[SEP]10210.26120.20430.51230.2693

The mask is 1 for tokens included in the mean and 0 for padding. This short recorded input needed no padding. Non-padding special tokens such as [CLS] and [SEP] are included, matching this model’s pooling operation.

pooled[d] = sum(mask[t] × token[t][d]) / sum(mask[t])

first four pooled values = [-0.0921, 0.1345, 0.3116, 0.2222]

length across all 384 pooled values = 5.4588

normalized[d] = pooled[d] / 5.4588

first four normalized values = [-0.0169, 0.0246, 0.0571, 0.0407]

For example, component 0 is -0.0921 ÷ 5.4588 = -0.0169. The search still compares all 384 components.

Query means “your search question” in this lesson. It is not the same object as an attention layer’s internal query, key, and value vectors. Explore self-attention separately.

Recorded model and reproducibility
Model
sentence-transformers/all-MiniLM-L6-v2
Artifact revision
onnx-sha256:4f148ba8ae9c2c7fbee4af2b132db8d06c6a6545b47fc83bbb98c3d22b8393e6
Corpus fingerprint
5631ca00533391824d77bbb1362937be47f93c075f7b23849f562cbc5d2d2224
Generation packages
numpy 2.5.5 · onnxruntime 1.32.0 · tokenizers 0.23.3

The vectors are bundled. No model is downloaded or executed in your browser. The generation script verifies the model artifacts before creating the fixture.

How does cosine similarity compare two vectors?

Cosine compares directions. Arrows pointing the same way score 1, perpendicular arrows score 0, and opposite arrows score −1. For embeddings, a higher score often signals related meaning; it does not prove that a passage answers the question.

q = [1, 0]c = [3.0, 4.0]θ = 53°
Teaching geometry only. Directions are drawn at equal length; these are not projections of the real embeddings.

cosine(q, c) = dot(q, c) / (length(q) × length(c))

dot = 1 × 3.00 + 0 × 4.00 = 3.00

length(q) = 1; length(c) = 5

cosine = 3.00 / (1 × 5) = 0.60

53°

Normalizing divides a vector by its length without changing its direction. If both vectors have length 1, their dot product already equals cosine similarity. The real search uses complete 384-dimensional vectors, not the four coordinates shown in the preview.

0.6 is a similarity score, not “60% certain”. The scale is model-dependent. A minimum cutoff may remove weak matches but cannot establish that the remaining passages contain the answer.

What would an experienced developer check?
Keep enough context
Tiny chunks can separate a rule from its condition. Large chunks can mix several topics. This lesson counts sentences; real splitters also consider headings, structure, and model token limits. Overlap preserves continuity but stores duplicate text and can crowd useful evidence out of top-k.
Retrieve enough, without filling the prompt with noise
Increasing top-k can recover another part of the answer, but also adds irrelevant text and token cost. Respect the answer model’s context limit; reserve room for the question, instructions, and output.
Use a compatible embedding space
Questions and documents must use a compatible model and configuration. Some retrieval models require different query and document prefixes. Changing models usually means re-embedding the documents. This MiniLM setup limits input to 256 word pieces; our short examples fit without truncation.
Choose retrieval for the job
This demo compares every chunk exactly. Large indexes often use approximate nearest-neighbour search, trading some recall for faster searches. Exact product IDs can benefit from keyword search; hybrid retrieval combines lexical and semantic results. A reranker can reorder retrieved candidates, but cannot recover a passage absent from its candidate set.
Keep sources current and access controlled
Update or delete index entries when source documents change. Filter documents by the user’s permissions before search and ranking, so inaccessible passages cannot enter the results or prompt. Treat retrieved documents as untrusted evidence, never as instructions that override the application.
Evaluate finding and answering separately
For questions with labeled evidence, recall@k measures how much expected evidence appears in the retrieved top-k. Then check whether the answer’s claims are supported and citations point to the right passages. Also measure latency and cost. A fluent answer and a high similarity score are not quality tests.

Sources: MiniLM model card · Cosine and normalization · Semantic search · Retrieve and rerank.

How it works: RAG: Retrieval-Augmented Generation

Step by step

  • Current pipeline stage
  • Vector comparisons
  • Selected evidence
  • Not selected

Press Start or Step: the step the animation is on lights up below.

  1. 1

    Start with source documents

    The shop's handbook holds its rules. RAG can only retrieve from information made available to it.

    documents = load_sources()
  2. 2

    Split each document into passages

    A small passage is easier to retrieve precisely. Overlap repeats some text so a rule is less likely to be separated from its explanation.

    chunks = split(documents, size, overlap)
  3. 3

    Turn passages into vectors

    Run the already trained embedding model on each chunk. Here MiniLM produces 384 numbers per passage; its learned dimensions do not have simple names.

    vectors = embed(chunks.text)
  4. 4

    Store vectors beside their sources

    Keep each vector with the passage text and its source identity. Reuse this prepared index for new questions; update it when its documents or embedding setup change.

    index = store(chunks, vectors, sources)
  5. 5

    Embed the question

    Represent the search question in a compatible embedding space. This search question is different from the internal query, key, and value vectors used by attention.

    question_vector = embed_query(question)
  6. 6

    Compare every chunk

    Cosine similarity compares vector directions. This demo uses every dimension, then sorts the scores. A high score is a useful match, not a probability that the passage answers the question.

    ranked = sort_by_cosine(question_vector, index)
  7. 7

    Keep the best eligible passages

    Apply the optional minimum score and keep up to top-k passages. A stricter cutoff or smaller top-k can remove useful evidence as well as irrelevant text.

    context = eligible(ranked, cutoff)[:top_k]
  8. 8

    Put evidence beside the question

    Send the selected text, source labels, question, and instructions to the answer model. Treat document text as evidence, not instructions to follow.

    prompt = instructions + context + question
  9. 9

    Answer what the evidence supports

    This simulation displays only authored claims whose required sentences were retrieved. It cites those passages and says when evidence is partial or missing. Real LLMs do not automatically have this evidence check.

    answer = generate_with_sources(prompt) # simulated here

Think of an open-book exam. Before answering a question, find the useful pages and place them beside the question. RAG does those two jobs: retrieve useful passages, then ask a language model to answer using them.

First prepare the documents: split them into passages called chunks, turn each chunk into a vector (a list of numbers), and store that vector with its text and source. A trained embedding model creates the vectors. Adding a document runs that model; it does not train it again.

For each question, create a compatible question vector, compare it with the chunk vectors, and retrieve the best matches. Put the original passages into the answer model's prompt. The answer model reads text, rather than the embedding numbers.

This page searches real, recorded MiniLM embeddings in all 384 dimensions. The answer is an authored simulation that checks known evidence for these six questions. It demonstrates full, partial, and missing evidence; a real language model still needs evaluation because it can make unsupported claims.

Where you'll meet it

  • Answering questions about a company's handbook, product documentation, or a collection of research papers.
  • Using information that changes without retraining the answer model: update the documents and their index instead.
  • Giving readers a route back to the source of an answer. A citation helps them check a claim; the citation alone does not prove it is supported.
time O(n × d + n log n) per searchspace O(n × d + source text) for the index

For n chunks and d vector dimensions, this exact search compares every vector and sorts all n scores. Embedding computation and answer generation have their own model costs. Large systems often use approximate indexes; RAG does not require a particular vector database.