KFF poll published in March 2026. More than 40 million people ask ChatGPT alone for medical guidance every day. The problem: the answers are dangerously unreliable. A study published this month in JAMA Network Open tested 21 AI models across 29 standardized medical scenarios. The chatbots failed at differential diagnosis more than 80% of the time. Some models hit a 100% failure rate in certain scenarios. A separate study from the University of Oxford found that when real users consulted AI tools, they correctly identified conditions only about a third of the time — and just 43% made the right call on next steps, like whether to go to the ER. ECRI, the independent patient safety organization, named AI chatbot misuse the number one health technology hazard of 2026. Their testing found chatbots suggesting incorrect diagnoses, recommending unnecessary tests, promoting subpar medical supplies, and — in one memorable case — inventing body parts. When asked about placing an electrosurgical electrode on a patient’s shoulder blade, a chatbot confidently said it was safe. Following that advice would risk burning the patient. The failures go deeper than wrong answers. In 52% of emergency cases studied, chatbots under-triaged — meaning they told people things were less urgent than they actually were. One bot failed to send a hypothetical patient with diabetic ketoacidosis and impending respiratory failure to the emergency department. That’s a life-threatening miss. But the picture isn’t entirely bleak. An American mother turned to ChatGPT after 17 doctors failed to diagnose her son’s condition. The chatbot suggested tethered cord syndrome, a rare neurological disorder. She brought it to her doctor. Testing confirmed it. The boy had successful surgery. A woman in the UK got years of anxiety diagnoses before ChatGPT flagged hereditary spastic paraplegia — confirmed through genetic testing. These cases are real, and they matter. They also share a pattern: the AI worked best when its suggestion was verified by a doctor, not acted on alone. The legal landscape is catching up. New York introduced a bill in March 2026 that would make chatbot companies liable when their tools give professional advice that causes harm. The federal AI LEAD Act would classify AI systems as products, opening developers to civil liability claims. Over 40 bills across 25 states are working through the same question: when a chatbot gives bad medical advice, who pays? Right now, nobody. Every major AI company — OpenAI, Google, Anthropic, Meta — includes medical disclaimers telling users not to rely on chatbot responses for health decisions. Those disclaimers do real legal work. They also get ignored by millions of people daily. Here’s where the line actually sits. AI chatbots are useful for understanding medical terms you’ve already heard from a doctor, researching what questions to ask at your next appointment, or getting a second perspective on a confusing diagnosis — as a starting point, not an endpoint. They are not useful for deciding whether chest pain needs an ER visit, adjusting medication dosages, or diagnosing symptoms in children. The Pew Research Center found that users themselves rate AI health info as more convenient than accurate. Trust that instinct. The chatbot won’t examine you. It won’t notice the thing you forgot to mention. And when it’s wrong — confidently, fluently wrong — it won’t know the difference.
Sources
KFF · Bloomberg · Fierce Healthcare · University of Oxford · Pew Research Center
This article is AI-generated.
]]>