Apple Intelligence Exhibits Racial and Gender Bias, Researchers Find

A recent study conducted by AI Forensics, a European nonprofit, has revealed significant racial and gender biases in Apple Intelligence, the AI model integrated into Apple devices such as iPhones, iPads, and MacBooks. The researchers analyzed over 10,000 AI-generated summaries and identified two primary patterns of bias. First, the model disproportionately mentions the ethnicity of non-white individuals, treating "white" as the default. For instance, it referenced a protagonist's ethnicity in only 53% of cases when the individual was white, compared to 89% for Asian, 86% for Hispanic, and 64% for Black individuals. Second, the AI frequently fabricated gender associations, assuming a nurse was female and a doctor was male in 67% of ambiguous cases.

NewsweekEU Observer

The biases identified in Apple Intelligence raise concerns about the broader implications of AI summarization. The automatic summaries generated by the AI can shape users' perceptions without their explicit consent, as they replace original messages. This contrasts with user-driven AI models like ChatGPT, where users actively choose to engage with the technology. The study highlights that summarization is not a neutral process; it involves editorial decisions that can influence how information is framed and understood.

EU Observer

The findings of the study have significant regulatory implications, particularly in the context of the European Union's upcoming AI regulations. The EU is set to establish an AI office with enforcement powers in August, which will include the ability to impose fines on companies that violate AI laws. AI Forensics argues that Apple Intelligence exceeds the EU AI Act's threshold for classification as a general-purpose AI model, which would necessitate transparency documentation. However, Apple has not signed the voluntary Code of Practice for General-Purpose AI, which acknowledges "discriminatory bias" as a systemic risk.

EU ObserverNewsweek

Critics of Apple Intelligence emphasize the need for greater accountability in AI systems, particularly those with extensive reach like Apple's. Paul Bouchaud, a study author, stated, "Most definitely it’s reflecting its training data," indicating that the biases stem from the datasets used to train the AI. Apple has previously acknowledged issues with the AI, temporarily disabling notification summaries for news apps after reports of fabricated headlines. However, the biases in personal and professional communications remain unaddressed.

EU Observer

While the study highlights significant biases in Apple Intelligence, it does not provide a comprehensive comparison with other AI models. Notably, a smaller model from Google, Gemma3-1B, reportedly hallucinated less frequently and less stereotypically in similar tests, suggesting that the biases observed in Apple’s model are not inherent to AI summarization.

EU ObserverNewsweek

As the EU prepares to enforce stricter AI regulations, Apple may face scrutiny regarding its AI practices. The findings from AI Forensics could prompt regulators to consider the systemic risks associated with the reach of Apple Intelligence, potentially leading to significant financial penalties for the company if it fails to comply with upcoming regulations.

EU Observer