ChatGPT Falls Behind in New AI Model Rankings

OpenAI's ChatGPT, which gained significant popularity after its public debut in late 2022, has been a dominant player in the generative AI chatbot market. However, a recent study conducted by the British company Prolific has placed ChatGPT in the eighth position among AI models, trailing behind competitors such as Gemini, Grok, DeepSeek, and Mistral. The study introduced a new benchmark called "Humaine," designed to evaluate AI performance based on natural human interaction rather than traditional technical metrics.

Boy Genius ReportThese 7 AI models just overtook ChatGPT in a new study — and the list may surprise you

The Humaine benchmark aims to address the disconnect between AI performance metrics favored by researchers and the actual preferences of everyday users. Prolific's analysis revealed that ChatGPT-4.1 ranked below several models, including:

Boy Genius Report

1. Gemini 2.5 Pro (Google)

These 7 AI models just overtook ChatGPT in a new study — and the list may surprise you

2. DeepSeek v3 (DeepSeek)

These 7 AI models just overtook ChatGPT in a new study — and the list may surprise you

3. Magistral Medium (Mistral)

These 7 AI models just overtook ChatGPT in a new study — and the list may surprise you

4. DeepSeek R1 (DeepSeek)

These 7 AI models just overtook ChatGPT in a new study — and the list may surprise you

5. Grok models (xAI)

These 7 AI models just overtook ChatGPT in a new study — and the list may surprise you

6. Gemma (Google)

These 7 AI models just overtook ChatGPT in a new study — and the list may surprise you

7. Gemini 2.0 Flash (Google)

These 7 AI models just overtook ChatGPT in a new study — and the list may surprise you

The study's findings suggest that previous evaluations may not have accurately reflected user priorities, which include understanding context, clarity in responses, and truthfulness.

These 7 AI models just overtook ChatGPT in a new study — and the list may surprise you

The results of the Prolific study indicate a shift in user expectations and the competitive landscape of AI chatbots. While ChatGPT has historically performed well in various independent rankings, its current placement raises questions about its adaptability to user needs. The study's authors emphasized that traditional benchmarks often overlook what users truly value in AI interactions.

Boy Genius Report

Some industry experts have expressed skepticism regarding the validity of the Humaine benchmark. Critics argue that the study's methodology may not comprehensively capture the complexities of AI performance. Additionally, there are concerns that the ranking could be influenced by sample bias, as the evaluation process might overrepresent the preferences of tech-savvy users rather than the general population.

Boy Genius Report

Prolific's blog post highlighted the need for a more user-centric approach to AI evaluation, stating, "AI evaluation has been dominated by technical benchmarks that, while important, fail to capture what people actually value." This sentiment underscores the growing demand for AI systems that prioritize user experience over technical specifications.

Boy Genius Report

• “ “Current evaluation is heavily skewed towards metrics that are meaningful to researchers but opaque to everyday users, such as accuracy on specialised datasets and performance on esoteric reasoning tasks.” — Prolific

These 7 AI models just overtook ChatGPT in a new study — and the list may surprise you

• “According to the study, we want chatbots that understand what we're saying, don't get confused if the conversation changes direction, give clear answers, and tell the truth.” — Prolific

Boy Genius Report

As the AI landscape continues to evolve, it remains to be seen how OpenAI and other companies will respond to these findings. The emphasis on user-centric evaluation may lead to further innovations in AI chatbot design and functionality, potentially reshaping the competitive dynamics in the industry.

Boy Genius Report