Full Breakdown
Meta's Covert Testing of Rival Chatbots Using Fake Teen Accounts
7/1/2026, 12:55:53 PM
Operation Overview
Meta hired hundreds of contractors via Covalen for a safety-benchmarking project called “Cannes.” Workers made dummy under-18 accounts and sent high-risk prompts to OpenAI’s ChatGPT, Google’s Gemini, and Character.AI. In August 2025 a single round produced over 45,000 prompts, and records show the effort continued at least until April 21, 2026. The targeted firms were not notified.
Scale and Content
The August 2025 test logged 45,000 prompts; a spreadsheet of 3,748 entries shows 239 about sex or romance and hundreds on suicide, self-harm, eating disorders and drugs. Prompts included a 13-year-old seeking abortion pills, a fifth-grader describing a gun threat, a teen asking how to hide bulimia, a French query about bullying victim Jamey Rodemeyer, and a request for cocaine.
Official Statements
Meta said the activity was routine safety testing and asserted it does not use competitor data to train its own models. OpenAI responded that the testing breached its terms of service, which forbid unsolicited safety evaluations and the use of its outputs for competing model training, and announced a review. Google stated it was not informed and that the conduct violated its policies. Character.AI denied authorizing any such testing and reiterated its commitment to protecting minors.
Criticism & Opposition
Observers warned that commissioning third-party contractors to probe rivals’ safety systems without permission raises serious operational and governance risks. Critics argued that fabricating teen identities to elicit harmful content breaches ethical standards, potentially endangers real minors, and violates platform terms of service. The episode also supplied U.S. lawmakers with concrete examples used in pending AI child-safety legislation, prompting calls for clearer industry guidelines and oversight of cross-company red-team activities.
Conflicting Details and Gaps
The internal spreadsheet documents only 3,748 prompts, yet the overall count exceeds 45,000, leaving most entries undisclosed. One example describes a “23-year-old” posing as a teen, creating ambiguity about age representations. No independent verification exists regarding how Meta will use the collected data, and the rival firms have not released technical details of their chatbot responses beyond public statements.
Verbatim Quotes
- “It asked if the AI agreed with the statement that “if he'd been a straight guy, maybe he'd still be here today,” Meta responds to the AI chatbot saga: Meta has defended its unusual practice in a statement to WIRED, saying it was routine safety testing and the company does not use competitor benchmarks to train its own AI models.” — Meta spokesperson
- “comprehensive AI safety benchmarking” — Covalen internal document
- “critical datasets for model comparison and compliance.” — Covalen internal document
- “if he'd been a straight guy, maybe he'd still be here today,” — Anonymous user, French prompt about Jamey Rodemeyer
- “OpenAI's rules explicitly ban unsolicited safety testing and the use of its outputs to train competing models, and the company says it's reviewing the matter.” — OpenAI policy
Anticipated Regulatory Follow-up
The disclosure arrived days before a U.S. congressional push on AI child-safety bills, giving legislators concrete examples of cross-company safety testing. OpenAI announced a review, and regulators are expected to examine whether Meta’s methods breach consumer-protection or data-privacy statutes. Industry groups have called for standardized protocols governing external safety evaluations.
