Story perspectives

Experts Warn AI Hallucinations Make Therapy Support Risky

9/18/2025

Summary

Story summary

OpenAI's research shows that AI "hallucinations" are mathematically inevitable due to training data errors and language model predictions.
Hallucinations persist even with perfect data, as distinguishing valid from invalid responses is inherently difficult.
AI systems often guess instead of expressing uncertainty, worsening hallucinations.
Experts warn against using AI for mental health support, as it cannot replace human therapists in crises.
AI can aid in administrative tasks but lacks the nuanced understanding needed for effective therapy.

History

Historical Context

In 1976, the U.S. National Institute of Mental Health established guidelines for mental health treatment, highlighting the necessity of human interaction. Recent findings reveal that AI "hallucinations" arise from limitations in language models, paralleling early mental health practices' struggles with accuracy. Experts warn against using AI for therapy, emphasizing the critical role of human therapists in mental health care.

1 of 2

Story summary

OpenAI's research shows that AI "hallucinations" are mathematically inevitable due to training data errors and language model predictions.
Hallucinations persist even with perfect data, as distinguishing valid from invalid responses is inherently difficult.
AI systems often guess instead of expressing uncertainty, worsening hallucinations.
Experts warn against using AI for mental health support, as it cannot replace human therapists in crises.
AI can aid in administrative tasks but lacks the nuanced understanding needed for effective therapy.

1 / 2