Are doctors' AI scribes a threat to patient safety?
What's the story
Artificial intelligence (AI) technology used to transcribe patient-doctor consultations has been flagged for potential risks by a National Health Service (NHS) watchdog. The concern stems from the system's tendency to incorrectly identify drug names and illnesses, which could jeopardize patient safety. Healthwatch England has highlighted these errors as a major concern in the rapid rollout of this technology within the NHS.
Patient impact
Woman falsely diagnosed with serious illness
One case highlighted by Healthwatch England involved a woman who was wrongly diagnosed with demyelination, a serious nerve damage condition that can lead to multiple sclerosis.
The error was only corrected after the patient, an NHS health professional herself, questioned the AI tool's record of her MRI scan result.
"This was eventually corrected but was a very traumatizing experience to be given an incorrect diagnosis because of AI and then be told it's a typo," said the woman, who asked not to be named.
Safety concerns
Potential risks associated with AI in healthcare settings
Healthwatch England has warned that undetected errors by AI scribes could find their way into patients' medical records, potentially affecting their care.
In another case, an AI scribe confused a prescribed drug with one of a similar name, a mistake identified by the patient rather than the doctor.
These incidents highlight the potential risks associated with this technology in healthcare settings.
Reporting issues
Urgent need for clarity over how patients can report
Healthwatch England has received "multiple stories from patients who have noticed these errors when a health professional hasn't."
The organization stressed that these inaccuracies could remain in records if not caught by the patient.
A spokesperson for Healthwatch said, "Healthcare has never been error-free. But our findings show the urgent need for clarity over how patients can report and get corrected any mistakes made by AI scribing tools or the professionals that use them."
Trust issues
Trust and confidence depend on good communication
Rachel Power, Chief Executive of the Patients Association, said, "Trust and confidence in this technology depend on good communication and genuine partnership with patients and right now both are missing."
Healthwatch also expressed concern over the Medicines and Healthcare products Regulatory Agency's decision not to classify AI scribes as medical devices, leaving no England-wide oversight for their safety and effectiveness.