CodeQuest101Advanced RAGChapter Studio
Advanced RAG

8 chapters · Learn by building

RAG basics

Chapter 01 / 08

Keep the meaning. Lose the noise.

Clean source text, find useful boundaries, and recover the context a small passage needs.

The idea, in plain words

Clean without changing the facts

Copied documents often contain repeated footers, broken spacing, and navigation text. Those extra words can compete with the passage that answers a question.

Remove recognizable noise while preserving the facts: X200, prices, dates, and words such as ‘not’. Cleaning is not the same operation as an embedding model's own tokenization.

1

Read the original source, including its repeated boilerplate.

2

Normalize spacing and remove the known repeated footer.

3

Keep product identifiers, table values, negation, and source identity.

4

Compare the useful text with the original before indexing it.

Decision inspector

Follow the evidence.

D4Delivery fees

Orders of 50 dollars or less pay a 5-dollar standard delivery fee.

Delivery guide and fee table · D4-s4

Useful text gives retrieval a clearer signal, while source identity lets a reader check the answer.

Math & code

Cleanup changes the input text; it does not update the model's weights.

clean = normalize_spacing(remove_known_boilerplate(raw))
assert identifiers_and_conditions_survive(clean)
Developer’s reasoning

A rule such as ‘remove all numbers’ destroys prices, dates, and product codes. Blanket punctuation or stop-word removal can also alter meaning.

Sources & model provenance

sentence-transformers/all-MiniLM-L6-v2 · 384D · mean pooling + L2 normalization.

cross-encoder/ms-marco-MiniLM-L6-v2 · recorded raw logits.

Rewrites, subquestions, HyDE text, and answer claims are authored teaching examples.

Encoder revision onnx-sha256:4f148ba8ae9c2c7fbee4af2b132db8d06c6a6545b47fc83bbb98c3d22b8393e6

Reranker revision 233902d25c440f23af6f7d6e94d2946bac0bee0a

Corpus SHA256 d1c21da297e88785f9a6ac434e0a58f3e10975052a48419ba511145d461f704e

Current sources only; archived documents D9 are excluded before retrieval.

Prepare text · Clean without changing the facts