Builder lab · Module 2 · 45 min

Grounding with Retrieval (RAG)

Attach a governed knowledge source to the request so answers cite something other than the model's memory — then evaluate retrieval and answer as two separate layers.

After this module you can

Assemble and inspect a simple retrieval-augmented generation pipeline that retrieves evidence from a small governed healthcare knowledge base, injects that evidence into an LLM request, and evaluates whether the final answer is actually grounded in the retrieved sources.

Distinguish parametric model knowledge from retrieved governed knowledge, and say what each can and cannot be held to.
Inspect chunks, metadata and ranked retrieval results rather than trusting a similarity score.
Choose top-k, a relevance threshold and metadata filters, and predict what each changes.
Identify a retrieval failure even when the generated answer reads fluently.
Build citations and provenance into the output contract so a claim resolves to a source, chunk and version.
Evaluate retrieval quality and answer quality as two separate layers.

Module 1 was model + instruction + schema + validation. Module 2 adds retrieval, evidence, citations and a second evaluation layer.

Step 1

Why retrieval exists

Same clinical domain as Module 1. Same model. The only difference between these two answers is what the model was shown.

Retrieval and generation are simulated. Retrieval and generation are both simulated locally with fixed logic. No vector database is queried, no model is contacted and no key is used. All documents are synthetic teaching material, not guidance.

Visual Guide 03 — LLM · RAG · Fine-tuning · Tools · Agents

Keep the terminology map next to you while you continue the technical track.

Open visual guide

Question

The clinician has diagnosed uncomplicated reflux. What follow-up and escalation advice should be included in the note?

A · Model-only (parametric knowledge)

answer — no retrieval

Advise the patient that symptoms usually settle with treatment and
lifestyle measures. Suggest a course of acid-suppressing medication
and arrange review if symptoms persist. Advise seeking help if the
patient feels unwell.

B · Grounded in retrieved evidence

answer — with retrieval

{
  "answer": "Complete the four-week once-daily PPI trial and review at four weeks whether or not symptoms settle [reflux-current#2]. Safety-net explicitly: new difficulty swallowing, persistent vomiting, unintentional weight loss, evidence of gastrointestinal bleeding or chest pain on exertion warrant same-day contact rather than waiting for review [reflux-alarm#1].",
  "citations": ["reflux-current#2", "reflux-alarm#1"]
}

What is missing from A? Select every genuine gap (3 or more, with no wrong picks).

Identify at least 3 genuine gaps, with no incorrect picks.

Lab progress

0 of 7 steps complete

Sources & evidence · 4 sources

This module cites public or consensus guidance, scholarly literature, vendor documentation.

Content reviewed: September 2026. Publication dates of the individual sources are shown in each citation.

Lewis et al. — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020)
The original formulation of retrieving passages and conditioning generation on them.
Open source
NIST AI Risk Management Framework — Generative AI Profile
Where provenance, information integrity and evaluation obligations sit in a risk framework.
Open source
OpenAI — Structured Outputs guide
How the citation array is enforced as part of the response contract.
Open source
Masanneck L, Meuth SG, Pawlitzki M. Evaluating base and retrieval augmented LLMs with document or online support for evidence based neurology. npj Digital Medicine 8, 137 (2025), 4 March 2025. DOI 10.1038/s41746-025-01536-y
Independent healthcare-specific evaluation using 13 recent neurology guidelines and 130 questions. Retrieval augmentation improved accuracy over base models, but potentially harmful answers still occurred and performance was weaker on case-based than on knowledge-based questions. It supports the need to evaluate retrieval and answer quality; it does not establish that RAG is clinically safe.
Open source