The Impact of AI Scraping on Online Media

In recent weeks, openDemocracy's website has faced significant disruptions due to automated bots deployed by AI companies. These bots, often sophisticated and difficult to detect, scrape data from various websites, undermining their traffic and operational capabilities. This phenomenon has been described as an "arms race" against public-interest projects, which are increasingly targeted by industrial-scale data extraction efforts.

openDemocracy

Grant Slater, a core developer at OpenStreetMap, highlighted a "dramatic shift" in the prevalence of scrapers over the past two years. He noted that these scrapers often utilize residential proxy networks, making it challenging to differentiate between legitimate users and automated data collectors. This ongoing battle forces organizations to divert resources to defend against these incursions, which can lead to a "war of attrition" for those trying to maintain their online presence.

openDemocracy

Audrey Hingle, a researcher, pointed out that the costs of large-scale crawling are primarily borne by publishers, who may need to restrict access to their content to survive. Tony Curzon Price, former editor-in-chief of openDemocracy, initiated the First International Data Union to empower individuals in the age of AI, advocating for collective control over personal data used by AI operators.

openDemocracy

Critics express concern that the rise of AI-driven scraping could lead to a more gated internet, where access to information is increasingly restricted. Hingle warns that this trend could accelerate the enclosure of online content, making it harder for users to engage with diverse sources of information. Furthermore, the commercial interests driving AI development raise alarms about the potential for misinformation and the erosion of trust in online news.

openDemocracy

The rapid integration of AI into various sectors has prompted calls for oversight and accountability. Stakeholders emphasize the need for meaningful counterweights to the profit-driven data extraction practices of AI labs. The urgency of this issue is underscored by the potential negative consequences of unchecked AI development, including environmental impacts and the risk of misuse in harmful contexts.

openDemocracy

• “We assume they are scraping data to fuel AI LLMs or start-ups set up to supply the AI companies with training data.” — Grant Slater, Core Developer at OpenStreetMap

openDemocracy

• “The actors extracting the most value from large-scale crawling are largely insulated from the costs it creates, which are absorbed by publishers.” — Audrey Hingle, Researcher

openDemocracy

• “Our data, used both for our personal benefit and for collective benefit, is the one chokepoint that ordinary citizens have over platform and BigAI power.” — Tony Curzon Price, Former Editor-in-Chief of openDemocracy

openDemocracy

• “The rapid AI-fication of the web makes it all the more urgent that we create meaningful counterweights to planetary-scale profit-driven data extraction.” — Anonymous Expert

openDemocracy

As the landscape of online media continues to evolve under the influence of AI, stakeholders are urged to advocate for policies that protect public interests and ensure equitable access to information. The ongoing dialogue surrounding AI's role in society will likely shape the future of digital engagement and information dissemination.

openDemocracy