Story perspectives

AI Struggles with Menopause Accuracy, Expert Oversight Needed

11/6/2025

33 3

Summary

Story summary

The Menopause Society 2025 Annual Meeting reports AI platforms struggle with accuracy on menopause questions.
ChatGPT 3.5 achieved 55% accuracy for patient questions, while ChatGPT 4.0 scored 40%.
Gemini performed the worst, with only 30% accuracy.
In a study on Post-Orgasmic Illness Syndrome (POIS), ChatGPT-4o achieved 100% accuracy in epidemiology but only 50% in treatment-related queries.
Readability declined over time, underscoring the need for expert oversight in AI-generated medical information.

History

Historical Context

In 1966, Joseph Weizenbaum developed ELIZA, the first AI chatbot, which showcased the limitations of machine understanding with only 50% accuracy in conversation. This early program highlighted the superficiality of AI interactions, a challenge that persists today, as modern platforms like ChatGPT 3.5 achieve just 55% accuracy on specific medical queries, underscoring the need for expert oversight in AI-generated information.

1 of 2

Story summary

The Menopause Society 2025 Annual Meeting reports AI platforms struggle with accuracy on menopause questions.
ChatGPT 3.5 achieved 55% accuracy for patient questions, while ChatGPT 4.0 scored 40%.
Gemini performed the worst, with only 30% accuracy.
In a study on Post-Orgasmic Illness Syndrome (POIS), ChatGPT-4o achieved 100% accuracy in epidemiology but only 50% in treatment-related queries.
Readability declined over time, underscoring the need for expert oversight in AI-generated medical information.

1 / 2