The Rise of Military Deepfakes: Implications for Propaganda and Society

The emergence of military deepfakes—AI-generated images and videos depicting fabricated individuals in military contexts—has raised significant concerns among researchers about their potential use in propaganda and misinformation. These digital creations, which can be monetized and serve political agendas, blur the lines between reality and fiction, influencing public perception and belief systems.

The Guardian

Advancements in generative AI technology have made it increasingly easy to create realistic deepfakes. Since early 2025, the Governance and Responsible AI Lab (Grail) has documented over 1,000 instances of political deepfakes on social media, a stark increase compared to just 1,344 incidents recorded in the previous eight years. This surge is attributed to the enhanced capabilities of AI tools, allowing users to generate convincing content rapidly.

The Guardian

Daniel Schiff, an assistant professor at Purdue University, highlights the growing normalization of deepfakes in political discourse, stating, “We are blending the lines between political cartoons and reality.” Sam Gregory, executive director of Witness, emphasizes the potential dangers of these technologies, warning that they can reinforce existing beliefs rather than challenge them. Additionally, Valerie Wirtschafter from the Brookings Institution notes that deepfakes contribute to a cycle of belief reinforcement among viewers.

The Guardian

During the ongoing conflict in Iran, social media has seen a surge of videos featuring AI-generated female soldiers, despite the country's prohibition on women serving in combat roles. One notable instance involved an AI-generated account of Jessica Foster, a fictional military figure, which amassed over 1 million followers on Instagram before being removed. This account linked to an OnlyFans page, showcasing the monetization aspect of such deepfakes. Furthermore, political figures like Donald Trump and California Governor Gavin Newsom have utilized deepfakes to further their narratives, with Trump sharing AI-generated images of Taylor Swift fans supporting him.

The Guardian

Critics argue that deepfakes can mislead audiences, even when viewers are aware of their artificial nature. Gregory points out the absurdity of some deepfake scenarios, such as Foster's unrealistic portrayal alongside Trump and Nicolás Maduro. The potential for deepfakes to create “AI swarms” that manipulate public opinion is a growing concern among researchers, likening it to a troll farm operating without human intervention.

The Guardian

In response to the challenges posed by deepfakes, the Coalition for Content Provenance and Authenticity has developed technical standards aimed at ensuring the authenticity of digital content. However, a recent investigation revealed that major platforms like Instagram and YouTube have inconsistently implemented these labeling standards, with only a fraction of AI-generated content being properly marked. Gregory attributes this inconsistency to a lack of political will among tech company leadership, urging for immediate action to improve content discernibility.

The Guardian

As the technology continues to evolve, researchers emphasize the need for robust measures to combat the malicious use of deepfakes. The development of effective labeling systems and the commitment of tech companies to uphold these standards will be crucial in mitigating the risks associated with AI-generated content.

The Guardian