Module 4 · 120 min
Generative AI & Healthcare Productivity
From Next-Token Prediction to Clinical Copilots. How large language models actually work, what changes when a system generates rather than predicts, prompting as specification, hallucination forensics, multimodality, and where generative AI creates value and risk across clinical, administrative, patient-facing and research work.
- Reason from mechanism to failure: what tokenisation, embeddings, attention and a finite context window each imply about where a generated output can go wrong.
- Separate the base model, the post-trained assistant and the product or workflow around them — and say which layer a given problem lives in.
- Explain mechanistically why fluency is decoupled from validity, and turn that into a verification design rather than a warning.
- Specify a prompt as task, context, constraints, examples, output format and success criteria, while treating prompting as specification rather than a safety control.
- Name and distinguish common failure modes: fabrication, unsupported inference, omission, contradiction, invented citation and acceptance of a false premise.
- Trace an answer to its source: model parameters, the current context, retrieved material or a tool result.
- Judge multimodal proposals across text, speech, images and structured data, and size the validation burden each one carries.
- Map generative AI use cases across clinical work, productivity, operations, patient experience and research, each with its value hypothesis, principal failure mode and required oversight.
- Evaluate an ambient documentation or copilot proposal on workflow, quality, review burden, subgroup and language performance, privacy, integration, accountability, monitoring and economics.
- Apply a reusable executive review checklist before piloting or buying a generative AI system.
Executive Generative AI Review Checklist
Eight GenAI-specific questions to take into any generative AI proposal, clinical or administrative. They are answerable by a non-technical executive and ordered so that the earlier answers determine whether the later evidence is even relevant. Accountability structures, data governance and ongoing safety monitoring are covered fully in Modules 7 and 9 and are only flagged here.
Educational governance checklist — not a validated clinical, procurement, legal or regulatory instrument. Use it to structure a conversation, not as evidence of compliance.
This is a review checklist, not a scored exercise. For each question, think what a strong answer would sound like, then reveal the guidance.
1. Who is the user, and what exact task is being generated — in one sentence, without the word 'AI'?
2. For any given answer, where does the information come from: parameters, supplied context, retrieval or a tool?
3. Which characteristic GenAI failure modes matter most here, how is each detected, and what error rate did you measure against a human baseline?
4. For anything quoted, retrieved or cited, is it accurate, current and correctly attributed to a real source?
5. Who reviews the output, with what time and what training — and how do you know review is real?
6. How does this change the actual work — and what new work does it create?
7. How does performance vary by subgroup — language, accent, specialty, complexity, setting?
8. What happens when the model changes, and how would you know performance had degraded?
Accountability for the final artefact, data handling and audit-trail design, and ongoing safety monitoring are governance questions in their own right rather than GenAI-specific ones. Module 7 defines what safety signals, owners and stop rules make monitoring operational; Module 9 covers the regulatory and data-protection obligations — lawful basis, retention, deployer duties — that apply once a system is in use.
Sources & evidence · 14 sources
This module cites primary legal, public or consensus guidance, scholarly literature.
Content reviewed: September 2026. Publication dates of the individual sources are shown in each citation.
Vaswani A, Shazeer N, Parmar N, et al. Attention Is All You Need. Advances in Neural Information Processing Systems 30 (NeurIPS 2017).
The original Transformer paper. Introduces self-attention, the architecture underlying current language and multimodal models.
Open sourceOuyang L, Wu J, Jiang X, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35 (NeurIPS 2022).
The InstructGPT paper: a historically important published example of instruction tuning plus reinforcement learning from human feedback. Contemporary systems use a family of related alignment methods and vendors rarely disclose their exact recipe, so do not read this as the single modern standard.
Open sourceTu T, Azizi S, Driess D, et al. Towards Generalist Biomedical AI. NEJM AI. 2024;1(3):AIoa2300138. doi:10.1056/AIoa2300138.
Med-PaLM M: a single generalist model handling text, imaging, genomics and other biomedical modalities, competitive across a benchmark suite. The authors state explicitly that substantial further development and real-world validation are required before clinical use — a proof of concept, not a deployment claim.
Open sourceOmar M, Sorin V, Collins JD, et al. Multi-model assurance analysis showing large language models are highly vulnerable to adversarial hallucination attacks during clinical decision support. Communications Medicine. 2025;5:330. doi:10.1038/s43856-025-01021-3.
Across 300 physician-validated simulated vignettes containing fabricated clinical details, six large language models produced 5,400 outputs; every model elaborated on fabricated details at least some of the time. A mitigation prompt reduced but did not eliminate the behaviour, and setting temperature to 0 did not significantly improve it. The core evidence for treating prompt-based mitigation as risk reduction rather than as a control. Results come from simulated vignettes, not live clinical care.
Open sourceDuggan MJ, Gervase J, Schoenbaum A, et al. Clinician Experiences With Ambient Scribe Technology to Assist With Documentation Burden and Efficiency. JAMA Network Open. 2025;8(2):e2460637. doi:10.1001/jamanetworkopen.2024.60637.
Prospective single-group pre-post quality-improvement evaluation in 46 clinicians across 17 specialties; associated with less time in notes and less after-hours work. Small, uncontrolled and single-system, so no causal claim.
Open sourceOlson KD, Meeker D, Troup M, et al. Use of Ambient AI Scribes to Reduce Administrative Burden and Professional Burnout. JAMA Network Open. 2025;8(10):e2534976. doi:10.1001/jamanetworkopen.2025.34976.
Multi-site pre/post quality-improvement study; 451 clinicians enrolled, 263 with direct ambulatory patient care in the analysed sample, voluntary participation and no control group. Reports associated improvements in administrative burden and burnout measures, not causal proof.
Open sourceHusa RA, Haggerty J, Nute AW, et al. Ambient Artificial Intelligence Use and Clinician Documentation Burden, Productivity, and Efficiency. JAMA Network Open. 2026;9(5):e2615762. doi:10.1001/jamanetworkopen.2026.15762.
Large real-world retrospective analysis of electronic health record metadata covering 1,547 active users, using an interrupted time-series design with generalized estimating equations; modest associations with documentation burden, productivity and efficiency. Not a randomised causal estimate, and note quality and omission are not measured, so read the effect sizes as associations.
Open sourceSharma M, Tong M, Korbak T, et al. Towards Understanding Sycophancy in Language Models. International Conference on Learning Representations (ICLR) 2024. arXiv:2310.13548.
Evidence that several preference-trained assistants tend to agree with a user's stated view, and that human preference data can reward this. A documented tendency in the systems studied, not a universal property of every model or version.
Open sourceLiu NF, Lin K, Hewitt J, et al. Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics. 2024;12:157-173. doi:10.1162/tacl_a_00638.
Performance on the tasks tested was highest when relevant information sat near the start or end of the input and degraded when it was placed in the middle of a long context. Evidence about the models and tasks studied; it does not license a universal rule, but it is a reason not to assume long prompts are used uniformly.
Open sourceKoenecke A, Nam A, Lake E, et al. Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences. 2020;117(14):7684-7689. doi:10.1073/pnas.1915768117.
Substantially higher word error rates for Black than for white speakers across five commercial speech-recognition systems in the US. Evidence from those systems, speakers and recording conditions — a reason to measure capture performance by group locally, not a claim about every ambient scribe.
Open sourceGoddard K, Roudsari A, Wyatt JC. Automation bias: a systematic review of frequency, effect mediators, and mitigators. Journal of the American Medical Informatics Association. 2012;19(1):121-127. doi:10.1136/amiajnl-2011-000089.
Systematic review of the tendency to accept computer-generated advice uncritically, including its mediators and mitigations. The evidence base for treating 'a clinician always reviews it' as a claim to be measured rather than assumed.
Open sourceRegulation (EU) 2017/745 on medical devices (Medical Device Regulation, MDR).
The EU framework under which software with a medical intended purpose — including software providing information used for diagnostic or therapeutic decisions — may qualify as a medical device. Classification depends on intended purpose and risk.
Open sourceRegulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (AI Act).
EU obligations that may apply in addition to the MDR where the Act's criteria are met, with duties differing for providers and deployers. Consult current official guidance on timing and applicability. Consolidated text as amended by Regulation (EU) 2026/1744 (in force 27 July 2026).
Open sourceUS Food and Drug Administration. Clinical Decision Support Software: Guidance for Industry and Food and Drug Administration Staff. Final guidance issued 6 January 2026, superseding the September 2022 version.
The current FDA position on which software functions are device software functions and which meet the Non-Device CDS criteria, based on intended use, the information presented and the clinician's ability to independently review the basis for a recommendation. Check the FDA guidance database for the current version before relying on it; guidance is not law and applies only in the US.
Open source
Capstone Board Pack
Add what you just learned to your own strategy document while it is fresh.