CodeQuest101Advanced RAGChapter Studio
Advanced RAG

8 chapters · Learn by building

RAG basics

Chapter 02 / 08

Two ways to represent a passage.

Inspect dense coordinates, sparse term signals, and a separate graph-based shortcut for finding vectors.

The idea, in plain words

Meaning and term signals

A dense embedding spreads information across many learned coordinates. It can help match a paraphrase even when the wording differs.

A sparse representation has mostly zero values. BM25 is a classical lexical ranking method based on term statistics; learned sparse encoders can also represent semantic expansions. Sparse does not always mean ‘exact words only’.

1

Represent the question and passage in the dense model's compatible space.

2

Compare all dense dimensions, rather than a displayed preview.

3

Inspect which query terms contribute to the BM25 score.

4

Compare an exact product-code question with a paraphrase.

Decision inspector

Follow the evidence.

D1Eligibility
Dense rank 1cosine 0.598
Sparse rank 2BM25 6.267

Opened packaging is allowed, provided the item has not been used.

Opened packaging and returns · D1-s2

12 coordinates shown; all 384 used. A coordinate is not a named topic.

opened 3.133packaging 3.133

Representations emphasize different signals. Product identifiers can make lexical evidence valuable, while different wording can make dense retrieval useful.

Math & code

BM25 uses term frequency, document frequency, saturation, and length normalization; defaults here are k1 = 1.2 and b = 0.75.

dense = cosine(query_vector, passage_vector)
sparse = sum(BM25_contribution(term, passage))
Developer’s reasoning

A dense coordinate is not a named topic. A BM25 score and cosine score use different scales, and neither is an answer-confidence percentage.

Sources & model provenance

sentence-transformers/all-MiniLM-L6-v2 · 384D · mean pooling + L2 normalization.

cross-encoder/ms-marco-MiniLM-L6-v2 · recorded raw logits.

Rewrites, subquestions, HyDE text, and answer claims are authored teaching examples.

Encoder revision onnx-sha256:4f148ba8ae9c2c7fbee4af2b132db8d06c6a6545b47fc83bbb98c3d22b8393e6

Reranker revision 233902d25c440f23af6f7d6e94d2946bac0bee0a

Corpus SHA256 d1c21da297e88785f9a6ac434e0a58f3e10975052a48419ba511145d461f704e

Current sources only; archived documents D9 are excluded before retrieval.

Dense & sparse · Meaning and term signals