Healthcare AI Learning
Course overview

Module 1 · 50 min

AI Foundations for Healthcare Leaders

Practitioner level, building on the Essentials course: target and label design, the prediction moment and which predictors are legitimately available, leakage from process-generated data, the operating point as clinical and operational policy, actionability under real capacity, and stating a proposal as a falsifiable value hypothesis.

Lesson progress
0 / 5 steps
Not started
Learning objectives
  • Design a supervised target: state what counts as the event, who decided it, and what a process-generated label makes the model learn.
  • Fix the prediction moment for a proposal and decide which predictors are legitimately available at it.
  • Spot leakage from response-generated variables before seeing any performance figure.
  • Treat the operating point as clinical and operational policy, and state its workload and missed-event consequences.
  • Test a proposal for actionability: the action, the owner, the shift, the capacity.
  • State a proposal as a value hypothesis — for whom, what changes, what value, how measured, what would falsify it.
  • Say what an external performance result does and does not license for your own population and setting.

Learn

Building on Essentials

AI in Healthcare Essentials established the vocabulary: how AI, machine learning, deep learning and generative AI relate; prediction versus generation; what a large language model does; deterministic versus probabilistic behaviour; training versus inference; and the first test of whether a problem needs AI at all. None of that is re-taught here. This module asks what you must decide once those words are settled.

  • A predictive model estimates a defined target; a generative model creates content. So the first design decision is what the target is — and who decided that it counts as the event.
  • Training estimates parameters; inference applies the fixed model in the workflow. So the question that matters is what was visible at the moment of inference, not what existed in the training records.
  • Most models return a score rather than a verdict. So somebody has to choose the operating point — and that choice is a clinical and operational policy, not a setting.
  • Value comes from a decision changing, not from the output existing. So a proposal has to name the action, the owner, the shift and the capacity before it is fundable.

Target and label design: the label is the project

  1. 01

    Define the target

    What exactly counts as the event, and who decided?

  2. 02

    Fix the prediction moment

    The instant the model must produce an output.

  3. 03

    Select predictors

    Only what is genuinely available at that moment.

  4. 04

    Learn the mapping

    Fit inputs to the known outcome across many examples.

The label and the prediction moment are decided before any model is trained.

A clinical prediction model learns from examples carrying a known target. Two decisions are made before any modelling: what exactly counts as the event, and at which instant the model must produce an output. Everything downstream — which predictors are admissible, what the score means, how it is evaluated — follows from those two.

Timing runs in two directions. The outcome may legitimately occur after the prediction moment; that is what makes it a prediction. Every predictor must already exist at that moment.

Process-generated labels and where leakage enters

Directly observed

Measured, not decided

  • Death, a laboratory value crossing a stated limit, a documented reading
  • Meaning is stable across sites and years
  • Rarely available for the outcome a service actually cares about

Process-generated

Produced by a care decision

  • ICU transfer, coded diagnosis, escalation, accepted referral
  • Encodes escalation practice, treatment limits and capacity
  • The model reproduces the decision behaviour it was fitted to

Most healthcare labels are produced by a care process rather than observed directly: ICU transfer, a coded diagnosis, an escalation, a referral accepted. Such a label records physiology together with escalation practice, treatment-limitation decisions and capacity — so a model fitted to it partly learns local decision behaviour, including any inequity in who was escalated.

The same process generates the leakage risk. Variables created by the response to an event — new oxygen therapy, urgent medication, a jump in monitoring frequency — can appear to predict the event they followed. The test is availability and meaning at the prediction moment, applied variable by variable; blanket rules either admit the response or discard legitimate pre-index information. Module 2 gives this a systematic method.

Four questions per candidate predictor

  1. Did it exist before the prediction moment?

    If it is undated within the episode, treat it as unavailable.

  2. Could it only appear once someone was worried?

    Monitoring frequency, urgent orders, new therapy — the response, not the state.

  3. Would it be present live, at that instant?

    Retrospective records are more complete than the production feed.

  4. Does keeping it remove the lead time?

    A model that fires when the team already knows adds no warning.

The operating point is a policy decision

A score is not yet an alert. The instant it triggers an action, sorts patients or fills a fixed number of programme places, someone has chosen an operating point — and where capacity is fixed, capacity chooses it.

Move the alert threshold

An illustrative cohort — not validated clinical data.

Lower — flag more40%Higher — flag fewer
flagged, event occurred flagged, no event event missed not flagged, no event
Patients flagged
27 of 100
Events caught
7 of 12
Events missed
5
False alerts
20
Alerts that are real
26%

The model has not changed — only the operating point has. Threshold choice sets workload, missed events and alert credibility, so it is a clinical and operational policy decision, not a data science setting.

Actionability, capacity and the value hypothesis

Four questions before any model

  1. What decision will change?

    If nothing changes, the output is alert volume.

  2. Who acts, with what capacity?

    Name the role and the shift before development.

  3. Is the outcome recorded reliably?

    A noisy label limits everything downstream.

  4. Would something simpler do?

    Compare with a rule or a workflow fix first.

These four questions screen proposals out; the value hypothesis is the positive counterpart — how a proposal that survives states what it is for, including what would falsify it. Later modules refine each part: Module 2 the data, Module 3 the clinical decision, Module 6 the evidence, Module 7 the controls, Module 8 the investment case.

The value hypothesis

Before any model is chosen, write the proposal as five short answers. It is a specification device, not a business case — the fifth answer is what keeps it honest.

  1. 1For whom? Which patients, which staff group, which service — named, not 'the organisation'.
  2. 2What changes in the workflow or decision? The specific step that is done differently, by whom, and at what moment.
  3. 3What value is expected? Which of the four value surfaces, stated as a direction of travel rather than a promised number.
  4. 4How will we measure it? One or two indicators you could actually collect, plus the baseline you compare with.
  5. 5What would falsify the hypothesis? The result that would make you stop or redesign — no change in the indicator, or benefit bought at an unacceptable cost elsewhere.

Worked example: imaging worklist triage

  • For whom? Patients having outpatient CT scans, and the reporting radiologists.
  • What changes? Studies with a suspected time-critical finding are moved to the front of the reporting queue instead of being read in arrival order.
  • Expected value? Clinical quality — shorter time from scan to actionable report for the subset that needs it most.
  • Measured how? Median and 90th-percentile time from scan to report for confirmed time-critical findings, against the pre-deployment baseline; total reporting throughput as a balancing measure.
  • Falsified by? No shift in reporting time for that subset, or a shift bought by delaying everyone else beyond an agreed limit.

Two patterns worth carrying forward

Transportability

The model that travelled badly

A recurring, well-documented pattern: a risk model performs strongly where it was developed, then degrades elsewhere because the population, coding practice and workflow differ.

External results are evidence about the approach, not proof about your setting.

Capability turned around

What a system can be made to do

The same generative capability enables convincing deepfakes, and deployed systems can be manipulated through jailbreaks or prompt injection hidden in the content they read.

In healthcare this is a security and governance question: what a system can be made to say or do, not only how well it performs.

Illustrative examples for discussion — not assessed content.

Sources & evidence · 4 sources

This module cites public or consensus guidance, scholarly literature.

Content reviewed: September 2026. Publication dates of the individual sources are shown in each citation.

  • Collins GS, Moons KGM, Dhiman P, et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ. 2024;385:e078378. doi:10.1136/bmj-2023-078378

    Sets out what a prediction-model report should contain, including how predictors and outcomes are defined and timed. It is a reporting standard: following it makes evidence legible, but does not itself establish that a model is safe or effective.

    Open source

  • Vasey B, Nagendran M, Campbell B, et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nature Medicine. 2022;28:924–933. doi:10.1038/s41591-022-01772-9

    Supports the distinction between model performance and clinical evaluation of a decision-support system in live use. It covers early-stage clinical evaluation and does not replace comparative effectiveness evidence.

    Open source

  • World Health Organization. Ethics and governance of artificial intelligence for health. WHO guidance. 2021. ISBN 978-92-4-002920-0

    Supports the governance and 'does this need AI at all' framing at policy level. It is guidance rather than binding regulation, and does not determine any specific jurisdiction's legal requirements.

    Open source

  • National Institute of Standards and Technology. Generative artificial intelligence. NIST Computer Security Resource Center Glossary; definition sourced by NIST to NIST SP 800-218A, Secure Software Development Practices for Generative AI and Dual-Use Foundation Models (July 2024). doi:10.6028/NIST.SP.800-218A. Entry status checked 25 August 2026.

    Used only for a stable, citable definition of generative AI. A glossary entry standardises terminology; it supports no claim about healthcare performance, safety or risk.

    Open source

Capstone Board Pack

Add what you just learned to your own strategy document while it is fresh.

Open capstone