Story perspectives

AI Chatbots Provide Risky, Inconsistent Medical Advice: Study

2/10/2026

Summary

AI Chatbots' Risks

A study by the University of Oxford finds AI chatbots provide inconsistent medical advice, posing risks to users.
The researchers evaluated nearly 1,300 participants from the United Kingdom (UK) using AI to identify health issues and decide whether medical care was needed.
Participants identified their health problems correctly only about one-third of the time, similar to online health searches, underscoring the need for improved AI systems and clear medical guidelines.

Summary

LLMs' Healthcare Evaluation

A study with 1,298 participants evaluated three large language models—GPT-4o, Llama 3, Command R+—as medical assistants against NHS guidance.
Participants faced medical scenarios to assess healthcare needs, revealing varying accuracy and a tendency to misjudge issues.
Technical difficulties during data collection highlighted challenges in evaluating LLMs for healthcare.
Findings underscore the potential and limits of integrating LLMs into healthcare and call for improved validation.

1 of 2

AI Chatbots' Risks

A study by the University of Oxford finds AI chatbots provide inconsistent medical advice, posing risks to users.
The researchers evaluated nearly 1,300 participants from the United Kingdom (UK) using AI to identify health issues and decide whether medical care was needed.
Participants identified their health problems correctly only about one-third of the time, similar to online health searches, underscoring the need for improved AI systems and clear medical guidelines.

1 / 2