Back to today’s briefing

Story perspectives

Study Reveals AI Summaries Are Wrong 73% of Time

5/14/2025

50 9

Summary

Story summary

A revealing study by Uwe Peters and Benjamin Chin-Yee uncovers a startling truth: large language models, such as ChatGPT, generate inaccurate summaries in a staggering 73% of instances. By scrutinizing nearly 5,000 summaries from prestigious journals like Nature and Science, the researchers shed light on the significant shortcomings of these AI tools in accurately conveying scientific information.

1 of 1

Story summary

A revealing study by Uwe Peters and Benjamin Chin-Yee uncovers a startling truth: large language models, such as ChatGPT, generate inaccurate summaries in a staggering 73% of instances. By scrutinizing nearly 5,000 summaries from prestigious journals like Nature and Science, the researchers shed light on the significant shortcomings of these AI tools in accurately conveying scientific information.