Doctors' AI scribes get names of drugs and diagnoses wrong, NHS watchdog warns
Healthwatch England warns that AI scribes used in NHS consultations are making serious errors, including misidentifying drug names and fabricating diagnoses such as "demyelination" (a condition linked to multiple sclerosis) Patients are frequently catching errors that doctors miss, raising concerns about patient safety and the potential for incorrect information to persist in medical records The Medicines and Healthcare products Regulatory Agency has declined to classify AI scribes as medical de
Analysis
TL;DR
- Healthwatch England warns that AI scribes used in NHS consultations are making serious errors, including misidentifying drug names and fabricating diagnoses such as "demyelination" (a condition linked to multiple sclerosis)
- Patients are frequently catching errors that doctors miss, raising concerns about patient safety and the potential for incorrect information to persist in medical records
- The Medicines and Healthcare products Regulatory Agency has declined to classify AI scribes as medical devices, meaning there is no England-wide regulatory oversight for their safety and effectiveness
- NHS policy expects AI scribes to reduce administrative burden and free up clinical time, but doctors report that reviewing transcripts for errors may negate any time savings
- Experts warn that error rates increase with multi-person consultations, complex medical histories, and patients whose first language is not English, though some GPs still believe AI records are more accurate than handwritten ones
Why It Matters
This article highlights a critical tension in healthcare AI deployment: the push for efficiency through automation versus the imperative of patient safety. For AI practitioners and health tech developers, it underscores that accuracy in clinical settings—particularly with ambient voice technology and medical terminology—remains insufficient, and that hallucinations and transcription errors can have serious real-world consequences. The lack of regulatory classification as medical devices also signals a governance gap that could expose healthcare systems to legal liability.
Technical Details
- AI scribe technology in the NHS operates as ambient voice transcription tools that listen to doctor-patient consultations and generate summary notes, with 27 different AI scribe products currently in use across GP and hospital settings in England
- Documented failure modes include hallucinations (inventing drug names and diagnoses not mentioned in the consultation), misrecognition of similar-sounding drug names, omission of critical instructions (e.g., repeat prescriptions), and misinterpretation of medical terminology such as "null demyelination" versus "demyelination"
- Error rates are reported to be higher in consultations involving multiple participants, patients with complex medical histories, and non-native English speakers, suggesting limitations in the speech recognition and natural language understanding capabilities of current systems
- A survey of 1,003 UK GPs found that more than half believed their AI-generated records were more accurate than their own, indicating a perception gap between clinician confidence and documented error rates
- The regulatory framework currently does not classify AI scribes as medical devices under the Medicines and Healthcare products Regulatory Agency, meaning no mandatory safety or efficacy standards apply at the national level
Industry Insight
- AI developers in the healthcare space must prioritize clinical-grade accuracy, particularly in medical terminology and drug name recognition, and invest in robust error-detection and correction mechanisms before large-scale deployment
- Healthcare organizations adopting AI scribes should establish clear patient reporting pathways for errors and implement mandatory human review protocols, as relying on patient vigilance alone is neither sustainable nor ethical
- Regulators and policymakers should address the current oversight gap by defining clear classification criteria for AI scribing tools, potentially requiring medical device certification to ensure patient safety standards are met before widespread NHS adoption
Disclaimer: The above content is generated by AI and is for reference only.