Skip to main content
Healthcare AI Learning
Course overview

Module 4 · AI in Healthcare Essentials

How to judge an AI claim

After this module you can take any AI claim apart: what is claimed, on what evidence, for whom, and what it would actually change.

Concept

Claim → Evidence → Fit → Impact

Four questions, always in this order. Skipping to the last one is how organisations buy impressive tools that change nothing.

  1. 01

    Claim

    What exactly is asserted — for which task, compared with what, and for which patients?

  2. 02

    Evidence

    Where does the number come from: a demo, a retrospective dataset, or a prospective evaluation?

  3. 03

    Fit

    Does the study population, setting and pathway resemble yours?

  4. 04

    Impact

    What changes in the workflow, who benefits, and who absorbs the extra work or risk?

In healthcare

Most weak claims collapse at step one. 'Our AI is 95% accurate' does not say accurate at what, measured against whom, or compared with the clinician doing it today.

Sources & evidence · 3 sources

Consensus guidance from WHO, international reporting standards and peer-reviewed literature. Healthcare examples in this module are synthetic teaching cases.

Content reviewed: September 2026. Publication dates of the individual sources are shown in each citation.

  • Collins GS, et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ. 2024;385:e078378. doi:10.1136/bmj-2023-078378

    What a prediction-model report should contain, including how outcomes, predictors and validation are described. A reporting standard: following it makes evidence legible, it does not make a model safe or effective.

    Open source

  • Vasey B, et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nature Medicine. 2022;28:924–933. doi:10.1038/s41591-022-01772-9

    Reporting standard for the early clinical evaluation of AI decision support in live use — the stage where model output meets clinicians, workflow and human factors. It does not replace comparative effectiveness evidence.

    Open source

  • Lekadir K, et al. FUTURE-AI: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare. BMJ. 2025;388:e081554. doi:10.1136/bmj-2024-081554

    A consensus guideline covering fairness, universality, traceability, usability, robustness and explainability across the AI lifecycle. Consensus guidance, not evidence that any specific tool is trustworthy.

    Open source