Chapter 08 · RAG RAG Track Local-first Project

RAG with the lid off: watch every chunk travel from PDF to answer.

A tiny end-to-end Retrieval-Augmented Generation demo over the VWO product requirements PDF. Chunking, embeddings, and the vector database all run locally; only the final answer call goes to Groq. Every retrieved chunk and its similarity score is visible.

01 What students build

The first project of the RAG track makes the whole pipeline inspectable. ingest.py slices the VWO PRD into ~800-character chunks (120 overlap, page metadata kept), embeds each with Nomic Embed through local Ollama (768-dim vectors), and stores everything in a persistent ChromaDB collection. app.py serves a one-page Flask UI: ask a question, see the top-K nearest chunks with distance and similarity scores, then read the Groq answer that cites the exact chunks it used.

Visible Pipeline

The UI always shows the diagram: Question, Embed, ChromaDB, Top-K, Groq, Answer. Each node lights amber as your query flows through it.

Chunk Inspector

A DB viewer lists every stored chunk with embedding previews, and ingest writes a standalone chunks_report.html. Chunks used for the last answer are highlighted.

Local-first

Embedding model and vector DB run on your machine for free. The only network call is the final Groq gpt-oss-120b completion, swappable via .env.

02 Architecture

VWO PRDpdf · data/
──▶
Chunker800 chars · 120 overlap
──▶
Nomic EmbedOllama · 768-dim
──▶
ChromaDBcosine · persistent
──▶
Top-Knearest chunks
──▶
Groq LLMgpt-oss-120b
──▶
Answercites chunk ids
collection vwo_product_requirements · everything local except the final groq call

03 Live demo (simulated with sample data)

This is a UI walkthrough with pre-baked sample data so you can feel retrieval without a backend. In the course you run the real thing locally: Ollama for embeddings, ChromaDB on disk, Flask at 127.0.0.1:5000, and a free Groq key.

basic-rag · 127.0.0.1:5000 demo · sample data

04 Inputs & outputs

What goes in

  • One source PDF dropped into data/ (the VWO PRD)
  • Natural-language questions typed into the Flask UI
  • .env config: GROQ_API_KEY, model names, Ollama URL
  • Tunables: CHUNK_SIZE, CHUNK_OVERLAP, TOP_K

What comes out

  • Top-K retrieved chunks with distance and similarity scores
  • A grounded Groq answer citing chunk numbers
  • DB viewer of every stored chunk, last-used ones highlighted
  • chunks_report.html: a standalone report of the whole ingest

05 How it's built

Pull the free local embedding model: ollama pull nomic-embed-text (137M params).
ingest.py reads the PDF page by page and slices it into ~800-char chunks with 120-char overlap, keeping page metadata.
Each chunk is embedded via the Ollama embeddings API and stored with its vector in a persistent ChromaDB collection.
app.py embeds the incoming question with the same model and runs collection.query() for the top-K nearest chunks by cosine distance.
The retrieved chunks plus the question go to Groq gpt-oss-120b with a "answer only from context, cite chunk numbers" prompt.
The Flask page renders the lit-up pipeline, the scored chunks, the cited answer, and the full DB contents for inspection.

06 Tech stack

Python · FlaskChromaDB (persistent)Ollama · nomic-embed-textGroq gpt-oss-120bPDF chunking (800/120)Cosine similaritydotenv config