Surge of AI-Generated Disinformation Amidst Iran-U.S. Conflict

In the context of the ongoing conflict in West Asia, a significant increase in AI-generated disinformation has emerged on Elon Musk's platform, X. These deepfake videos depict various fabricated scenarios, including American soldiers captured by Iran and U.S. embassies in flames. Researchers indicate that the volume of such content has surpassed that seen in previous conflicts, complicating the ability of social media users to discern fact from fiction. In response to this surge, X announced a new policy aimed at curbing the spread of misleading AI-generated war content, which includes suspending creators from its revenue-sharing program for 90 days if they fail to disclose the artificial nature of their posts.

The Hindu

X's head of product, Nikita Bier, emphasized that subsequent violations of this policy could lead to permanent suspension from the platform. This policy shift has garnered support from U.S. State Department officials, such as Sarah Rogers, who praised it as a beneficial complement to X's Community Notes, a crowd-sourced verification system. However, disinformation researchers express skepticism regarding the effectiveness of these measures. Joe Bodnar from the Institute for Strategic Dialogue noted that the feeds he monitors remain inundated with AI-generated content, suggesting that the new policy has not deterred creators from disseminating misleading visuals.

The Hindu

Despite the introduction of the demonetization policy, researchers highlight that many users spreading AI content are not part of the revenue-sharing program and thus remain unaffected by these restrictions. A report from the Tech Transparency Project raised concerns that X may be profiting from premium accounts linked to Iranian government officials and state-controlled media, potentially violating U.S. sanctions. Although X has removed blue checkmarks from some of these accounts, the platform's overall approach to managing disinformation continues to face scrutiny.

The Hindu

The challenges of effectively moderating AI-generated content are compounded by X's own AI chatbot, Grok, which has reportedly misinformed users by affirming the authenticity of several AI visuals related to the conflict. Additionally, the effectiveness of the Community Notes system has been questioned, with a study indicating that over 90% of notes are never published. Alexios Mantzarlis, director of the Security, Trust, and Safety Initiative at Cornell Tech, remarked that while X's policy could theoretically reduce incentives for spreading disinformation, its practical implementation remains uncertain.

The Hindu

• “The feeds I monitor are still flooded with AI-generated content about the war,” — Joe Bodnar, Institute for Strategic Dialogue

The Hindu

• “X's policy is a reasonable countermeasure to viral disinformation about the war.” — Alexios Mantzarlis, Cornell Tech

The Hindu

The ongoing conflict in West Asia has highlighted the urgent need for effective measures against disinformation, particularly as AI technology continues to evolve and complicate the information landscape.

The Hindu