Builder lab · Module 5 · 50 min

Evals for Healthcare GenAI

Build a small evaluation set with explicit expectations, split automated from human checks, run a baseline and a variant, read the failure slices and turn the result into a release decision.

After this module you can

Build a small evaluation set with explicit expectations per case, split checks into automated and human-judged, run a baseline, compare a changed configuration against it, read the failure slices and turn the result into a release decision.

State what a golden case must contain beyond an input and a good answer.
Separate structural, grounding, completeness, abstention, tool-call and safety dimensions.
Decide which checks can be automated and which require a clinical reviewer.
Read a case × dimension matrix instead of a single headline score.
Run a regression comparison and spot a change that helps one metric while harming another.
Name the release blocker and choose ship, revise or escalate on the evidence.

Modules 1–4 built the machinery. Evaluation is what tells you whether the machinery does its job on cases you did not hand-pick — and, just as importantly, what it does on the cases where it should refuse.

Step 1

A demo is not an evaluation

A demo is a case you chose after seeing the output. An eval is a set you fixed before.

Everything here is simulated. The eval set, the model outputs and the pass/fail results are fixed fixtures. Nothing is scored by a model here, and passing this simulation is not evidence about any real system.

Select every true statement (3 or more, no wrong picks).

Task-specific eval sets

Fix the cases before you look at outputs, and build them for your task and your population — not for a leaderboard.

Ground truth has limits

Reference standards are made by people and systems that also err: coding gaps, disagreement between reviewers, outcomes recorded late. Record how truth was set, and its known weaknesses.

Clinician in the loop

Some qualities cannot be scored automatically. Have clinicians judge a defined sample against written criteria, and track agreement between them.

Edge cases and subgroups

Include multimorbidity, rare presentations, missing data, out-of-scope questions, and subgroups where performance could plausibly differ. Report slices, not one average.

System-level, not model benchmark

Evaluate the whole application — prompt, retrieval, tools, guardrails, fallback — as the learner will use it. A public model benchmark says little about your pipeline.

Best model ≠ best application

A smaller or less capable general model can sometimes outperform a stronger frontier model on a narrow task when retrieval, context and constraints are better designed. This is a possibility to test, not a rule to assume.

Select at least 3 true statements, with no incorrect picks.

Lab progress

0 of 7 steps complete

Sources & evidence · 6 sources

This module cites public or consensus guidance, scholarly literature, vendor documentation.

Content reviewed: September 2026. Publication dates of the individual sources are shown in each citation.

NIST AI Risk Management Framework — Generative AI Profile (NIST AI 600-1)
Measurement and risk framing for generative systems, including evaluation and human oversight.
Open source
OpenAI — Evals guide
Vendor tooling documentation for graded case sets and regression runs. Useful for mechanics only; it is not independent conceptual authority on evaluation design.
Open source
Anthropic — Create strong empirical evaluations
Vendor guidance on designing test cases and grading rubrics. A practical engineering perspective, not an independent or neutral evaluation standard.
Open source
A framework for human evaluation of large language models in healthcare derived from literature review. npj Digital Medicine, 28 September 2024
Independent review of 142 studies proposing the QUEST human-evaluation framework and documenting gaps in the reliability, generalizability and applicability of current human evaluation practice. Support for structuring human evaluation; not itself a validation standard.
Open source
Johri et al. An evaluation framework for clinical use of large language models in patient interaction tasks. Nature Medicine 31, 77–86 (2025), 2 January 2025
CRAFT-MD evaluates realistic clinical dialogue and identifies limitations that medical-exam-style benchmarks miss. A task-specific evaluation example, not a universal benchmark.
Open source
DECIDE-AI reporting guideline (BMJ 2022;377:e070904)
What early live clinical evaluation of an AI system involves — the layer beyond any offline eval.
Open source