Module 6 · 85 min
Evaluating Healthcare AI
From Vendor Claims to Evidence-Based Decisions. How to judge whether a healthcare AI claim is credible: which evidence to demand at each layer, what the headline metrics do and do not tell you, whether a result transfers to your setting, the red flags in an impressive-looking deck, the evidence ladder from validation to real-world impact, and the questions to ask a vendor or data-science team. Technical methodology is available as optional deep dives.
- Separate the three claims an AI proposal usually blends together: model performance, workflow performance, and patient or operational impact.
- Interpret sensitivity, specificity, PPV, NPV, AUC and calibration in plain language, using absolute counts and alert workload rather than formulas.
- State precisely what each headline metric cannot tell you, and convert that gap into a question for the vendor or data-science team.
- Judge whether evidence generated elsewhere is likely to hold for your patients, your case mix and your workflow, and treat transportability as an empirical question rather than a permanent badge.
- Recognise the recurring red flags in impressive-looking evidence packs, including subgroup claims resting on a handful of events.
- Place a piece of evidence on the ladder from retrospective validation to post-deployment monitoring, and say which claim that rung can and cannot support.
- Run a structured evidence review of a realistic package and produce a defensible go / limited pilot / more evidence first / no-go recommendation, naming the evidence that would change it.
The metrics a healthcare leader must recognise
You do not need to compute these. You need to hear one in a meeting, know what it is a statement about, and notice when it is being asked to carry a claim it cannot carry.
Throughout, imagine the same fictional example: an inpatient deterioration model, running on a 500-bed hospital's general wards, where roughly 8 in every 100 admissions experience the outcome the model is trying to anticipate.
Sensitivity
Among the people who truly have or develop the outcome, how many does the system flag?
Of 100 admissions that deteriorate, a sensitivity of 75% means about 75 were flagged in advance and about 25 were not.
In the meeting: High sensitivity is bought with a lower threshold, which means more alerts. Always ask what it costs in flagged patients.
Specificity
Among the people who do not have the outcome, how many does the system leave alone?
Of 100 admissions that do not deteriorate, a specificity of 80% means about 80 pass unflagged and about 20 are flagged anyway.
In the meeting: In a population where most patients are fine, even a small shortfall in specificity generates a large absolute number of false alerts.
PPV (positive predictive value)
When the system flags someone, how often is the outcome actually present or coming?
A PPV of 25% means that of every four patients flagged, roughly one deteriorates and three do not.
In the meeting: This is the number the responding service feels. It is also the number most often missing from the slide.
NPV (negative predictive value)
When the system does not flag someone, how often is the outcome genuinely absent?
An NPV of 97% means that of 100 unflagged admissions, about 97 do not deteriorate and about 3 still do.
In the meeting: NPV looks reassuring wherever the outcome is uncommon. It is not permission to stop clinical observation.
AUC (ROC-AUC, sometimes called the c-statistic)
How well the system ranks: given one patient who has the outcome and one who does not, how often does it score the first higher?
An AUC of 0.82 means the ranking is good in the evaluated cohort. It is not 82% correct, and it says nothing about any particular threshold.
In the meeting: AUC is not accuracy, not calibration and not clinical benefit. Treat a pack that leads with AUC alone as incomplete.
Calibration
When the system says 20%, do roughly 20 out of 100 comparable patients actually experience the outcome?
If everyone told '20% risk' turns out to have about 35% risk, the ranking may still be fine while every absolute number shown to a clinician is wrong.
In the meeting: Whenever a displayed percentage drives the decision — a threshold, a conversation, a referral — calibration is not optional.
Prevalence and case mix
How common the outcome is in the population the system will actually run on, and who those patients are.
The same sensitivity and specificity produce a far lower PPV in a population where the outcome is rare than in a high-risk unit.
In the meeting: Ask what the event rate was in the evaluation, and what it is in your service. If they differ materially, the alert experience will differ too.
Name the metric
Six things you might genuinely hear. Attempt all six to continue; correctness is not required.
1. The outreach nurse's complaint
“We visit four flagged patients for every one who actually deteriorates.” Which metric is she describing?
2. The missed deteriorations
An audit finds that of 160 patients who deteriorated last year, 40 were never flagged by the model. Which metric does that describe?
3. The vendor's headline
“Our model achieves 0.91 — it is right nine times out of ten.” Which metric is being misdescribed?
4. The risk conversation
Clinicians will read the displayed percentage aloud to patients and use it to decide on referral. Which property matters most?
5. Two hospitals, one model
The same model, unchanged, is deployed in a high-acuity unit and in a low-risk elective ward. Which figure will differ most between them?
6. The reassuring slide
“Excellent discrimination across 40,000 admissions.” What is still entirely unknown from that sentence?
Notice that every one of those judgements used counts and context, not formulas. That is the level of literacy this module asks for: enough to interrogate the number, not enough to recompute it.
Technical deep dive — the confusion matrix and where these metrics come fromOptional. Useful if you want to reconstruct the numbers yourself. Not required for the assessment and not needed to complete this step.
A model that outputs a probability does not by itself produce a positive or a negative. Somebody chooses an operating threshold. Crossing the truth against the output at that threshold gives four cells: true positives (event occurred and the system flagged), false positives (flagged, no event), false negatives (event occurred, no flag) and true negatives (neither).
Sensitivity is TP / (TP + FN): the flagged share of everyone who had the event. Specificity is TN / (TN + FP): the unflagged share of everyone who did not. Both are counted down the truth columns, which is why they do not depend directly on how common the event is.
PPV is TP / (TP + FP) and NPV is TN / (TN + FN). These read across the model's own rows, so they move with prevalence: with sensitivity and specificity held fixed, a lower event rate generally lowers PPV and raises NPV, though the magnitude of each shift depends on the actual numbers.
Every cell is threshold-dependent. Lower the threshold and the flagged column grows: true positives rise, false positives usually rise faster in a low-prevalence population, false negatives fall. A confusion matrix reported without its operating point is not interpretable.
Read the counts before the percentages. Four hundred false positives in a month is a staffing decision; 'specificity 80%' is a statistic.
Technical deep dive — prevalence playgroundOptional. Move prevalence while holding sensitivity and specificity still, and watch PPV and NPV separate. Expected counts, computed deterministically and rounded — illustrative values, not observed data from any study. Nothing here is assessed.
| Expected counts | Outcome occurred | Outcome did not occur |
|---|---|---|
| Model positive | 80 TP | 90 FP |
| Model negative | 20 FN | 810 TN |
PPV
47.1%
Of those flagged, the expected share who experience the outcome.
NPV
97.6%
Of those not flagged, the expected share who do not.
Total flagged
170
The workload the responding service absorbs.
Sources & evidence · 8 sources
This module cites scholarly literature.
Content reviewed: September 2026. Publication dates of the individual sources are shown in each citation.
Collins GS, Moons KGM, Dhiman P, et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ. 2024;385:e078378. doi:10.1136/bmj-2023-078378. Published 16 April 2024. (A correction was published on 18 April 2024: BMJ. 2024;385:q902, doi:10.1136/bmj.q902; the substantive guidance is unchanged.)
The reporting standard for prediction-model studies, including AI methods. It specifies what must be reported so a study can be appraised. It is not a quality or safety standard, and adherence says nothing about whether the model performs well.
Open sourceMoons KGM, Damen JAA, Kaul T, et al. PROBAST+AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods. BMJ. 2025;388:e082505. doi:10.1136/bmj-2024-082505
A structured tool for appraising risk of bias and applicability in prediction-model studies. It supports a judgement about a study; it does not certify a model or license deployment.
Open sourceVan Calster B, Collins GS, Vickers AJ, et al. Evaluation of performance measures in predictive artificial intelligence models to support medical decisions: overview and guidance. Lancet Digital Health. 2025;7(12):100916. doi:10.1016/j.landig.2025.100916. PMID 41391983.
A contemporary overview of which performance measures answer which question, and how they are misused. Methodological guidance from an expert group, not a regulation or a mandatory standard.
Open sourceVan Calster B, McLernon DJ, van Smeden M, Wynants L, Steyerberg EW. Calibration: the Achilles heel of predictive analytics. BMC Medicine. 2019;17:230. doi:10.1186/s12916-019-1466-7. PMID 31842878.
The standard reference for why calibration matters and how discrimination can look strong while probabilities are wrong. Background for the calibration deep dive; a methodological review rather than evidence about any particular system.
Open sourceVickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Medical Decision Making. 2006;26(6):565-574. doi:10.1177/0272989X06295361. PMID 17099194.
The original description of net benefit and decision curves, referenced by the optional threshold deep dive. It compares strategies on evaluation data; it does not demonstrate that deploying a model improves outcomes.
Open sourceRiley RD, Debray TPA, Collins GS, et al. Minimum sample size for external validation of a clinical prediction model with a binary outcome. Statistics in Medicine. 2021;40(19):4230-4251. doi:10.1002/sim.9025. PMID 34031906.
Sizing for external validation driven by the precision required, given the expected event proportion. Referenced by the optional validation deep dive; useful for retiring fixed event-count folklore.
Open sourceVasey B, Nagendran M, Campbell B, et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nature Medicine. 2022;28:924-933. doi:10.1038/s41591-022-01772-9. (A publisher correction has been issued; cite the current record.)
A reporting frame for early live evaluation, covering human factors, safety and implementation — the third rung of the evidence ladder. It structures how such a study is described; it is not evidence that a system is safe or effective.
Open sourceSounderajah V, Guni A, Liu X, et al. The STARD-AI reporting guideline for diagnostic accuracy studies using artificial intelligence. Nature Medicine. 2025;31(10):3283-3289. doi:10.1038/s41591-025-03953-8. Published online 15 September 2025. (An Author Correction, doi:10.1038/s41591-026-04570-9, corrected an author name; the methodological content is unchanged.)
The AI-specific extension of STARD, including reporting of setting, population, reference standard and operating point — the details that determine whether an accuracy estimate travels. A reporting guideline, not an evidence standard.
Open source
Applied practice: Model Evaluation Lab
Review a hypothetical vendor evidence pack as a decision-maker: read the headline metrics, find what is missing, choose the next evidence step and make a pilot recommendation.
Capstone Board Pack
Add what you just learned to your own strategy document while it is fresh.