AI Companies Navigate a Perfect Storm of Profit, Control, and Protest
Artificial intelligence (AI) companies are currently navigating a challenging landscape, described by experts as a “perfect storm.” The sector faces the dual pressures of controlling the use and output of its powerful platforms while simultaneously striving to maintain profitability and contend with growing calls for boycotts. This complex situation is amplified by concerns over potential misuse of AI for autonomous warfare and mass surveillance, leading to significant public and governmental scrutiny.
One prominent example of this friction is the “QuitGPT” protest movement, which has emerged as a direct challenge to OpenAI’s popular ChatGPT. This grassroots campaign urges individuals to cease using and cancel subscriptions for OpenAI’s flagship platform. The movement claims a significant following, reportedly attracting over 4 million participants, though OpenAI states it serves more than 900 million users weekly, with 50 million subscribers.
The QuitGPT movement gained significant traction in January, coinciding with a period of heightened political tension in the US. The campaign’s origins are closely tied to concerns about the potential for AI companies to inadvertently or intentionally contribute to the rise of authoritarianism. This anxiety was particularly stoked by a deal struck between the Trump administration and OpenAI, allowing the deployment of OpenAI’s technology into classified government networks.
OpenAI, in its agreement with the Pentagon, reportedly set out three “main red lines”: no use of the technology for mass domestic surveillance, to direct autonomous weapons systems, or to make “high-stakes automated decisions.” However, this accord intensified the QuitGPT movement, whose participants identify as “democracy activists” deeply concerned about the implications of AI development.
The Rise of QuitGPT and its Motivations
The QuitGPT movement’s website outlines its core mission: to encourage people to stop using and cancel subscriptions for ChatGPT. Beyond the political concerns surrounding government use, the campaign also cites several other reasons for boycotting AI technology:
- Emotional Dependence on AI: The movement highlights the risk of users developing an unhealthy emotional reliance on AI platforms, potentially leading to what they term “AI psychosis” after prolonged interactions with chatbots.
- Environmental Impact: QuitGPT draws attention to the significant environmental footprint of AI data centres, a growing concern in the broader conversation around technological sustainability.
- Influx of Advertising: The increasing presence of advertising within AI platforms is also cited as a reason for dissatisfaction among some users.
OpenAI has responded to these criticisms. A spokesperson for the company clarified that individual political donations made by OpenAI president Greg Brockman were made in a personal capacity and not on behalf of OpenAI. Regarding concerns about emotional dependence, the spokesperson stated that OpenAI has collaborated with over 170 mental health experts to address these issues. The company maintains that the QuitGPT campaign presents misleading claims and that its focus remains on developing AI that is safe, useful, and broadly beneficial. They emphasize their commitment to accessibility, robust safety investments, and global collaboration with governments to ensure responsible AI deployment.
Legal Battles and Government Contracts: The Anthropic Saga
The complexities facing AI companies are further illustrated by the ongoing legal dispute between rival AI firm Anthropic and the Trump administration. Anthropic, the developer of the AI platform Claude, has found itself at odds with the Pentagon over the use of its technology.
While the Pentagon asserted its right to use Claude as long as it complied with US law, Anthropic drew a firm line on two critical issues: it refused to permit Claude’s use for lethal autonomous warfare without human oversight and would not allow it for mass surveillance of American citizens. In response, the Pentagon designated Anthropic as a supply-chain risk, effectively barring government contractors from utilising Claude in US military operations. This move has been noted as potentially the first instance of the US government using such a designation against a domestic company.
Defense Secretary Pete Hegseth has insisted that Anthropic and its CEO, Dario Amodei, must accept “all lawful” uses of Claude, threatening further repercussions. The Pentagon’s interest in Claude was reportedly for analysing intelligence and aiding operational planning, particularly in support of US military actions, including those in Iran.
On March 10, Anthropic initiated legal action against the White House, labelling the Pentagon’s designation an “unlawful campaign of retaliation.” The company’s court filings argued that the Constitution does not permit the government to wield its power to punish a company for protected speech and that no federal statute authorises the actions taken.
Adding another layer to this intricate situation, just a day after Anthropic’s lawsuit, OpenAI announced its own agreement with the Pentagon to integrate its technology into the Department of Defense’s classified network, with the stipulated “red lines” for its use.
The Guardrail Dilemma: Safety vs. Functionality
Experts like Jeannie Paterson from the University of Melbourne’s Centre for Artificial Intelligence and Digital Ethics observe that Anthropic is attempting to maintain a positive stance on generative AI use. She points to the company’s established “constitution” that articulates its commitment to human benefit. However, she also acknowledges that, as a corporation, Anthropic must prioritise profit.
Professor Paterson explains that all providers of large language models implement “guardrails” to prevent certain harmful outputs, such as the generation of child sexual abuse material or instructions for creating weapons. While these guardrails are partially effective, she notes that they can be circumvented. The substantial cost of developing these tools raises questions about how companies sustain their operations. Government contracts, she suggests, offer a lucrative avenue for financial support.
Accountability in the Age of AI
Kimberlee Weatherall, co-director of the University of Sydney’s Centre for AI, Trust and Governance, acknowledges OpenAI’s relative success in making ChatGPT safe for general use. However, she posits that when governments employ these technologies, the lines of responsibility for misuse become blurred.
“The Anthropics and OpenAIs of the world are in a little bit of a bind,” Professor Weatherall states. “They are promoting the massive capacity of their technology and how it’s going to completely change the world. That puts them in a difficult position of saying, ‘Look, we’re super powerful,’ which makes governments around the world more inclined to say, ‘OK, if you’re super powerful, you need to take responsibility.’ But then you also have governments saying, ‘You say this stuff is super powerful, we’d like to use it, and we’re the government, so you just have to trust us to use it lawfully or appropriately, and we’re the ones that get to decide what that means.'”
Recent data from mobile consumer intelligence analyst Apptopia indicates a decrease in the US market share for both Anthropic’s Claude and OpenAI’s ChatGPT in the 12 months leading up to January this year. ChatGPT’s share of overall AI platform users dropped from 69.1 per cent to 45.3 per cent, while Google’s Gemini saw a significant increase.
Professor Paterson highlights a fundamental tension: increasing safety guardrails can diminish functionality, making platforms less appealing to users. “The problem with these technologies is there is no right use,” she explains. “They’re kind of general-purpose AI, which means absolutely they’re being designed to be used for all sorts of things. Unfortunately, that means they can be used to hurt people as well as benefit them. There’s a double-edged sword there because you want the tool to be safe enough that people will use it and that it’s used widely. We don’t want it too safe, because if it’s too safe it ceases to be fun or adaptive to use.”
Nicolas Suzor from the QUT School of Law points out that regulators have struggled to keep pace with AI’s rapid growth, including establishing clear rules on accountability for harm caused by its use. “We’re not used to regulating AI infrastructure when it can be used for unknown purposes,” Professor Suzor comments.
Will the Boycott Make a Difference?
Opinions among experts are divided on the potential impact of boycotts like QuitGPT. Professor Suzor views the movement as a symptom of a broader consumer revolt against AI, suggesting it is far from over. He believes it reflects a growing discomfort with the decision-making power wielded by AI developers and a distrust of government policies.
Professor Weatherall concurs, anticipating that protests will escalate. She notes pushback on environmental concerns, data centre locations, and the broader information environment. The increasing number of individuals concerned about various risks, coupled with governments’ dual desires to protect citizens and leverage AI, creates a complex regulatory landscape. The fear and concern surrounding AI, she suggests, are brought to a head by real-world events like military conflicts, making technology a potential flashpoint for large corporations in turbulent times.
Conversely, Professor Paterson expresses scepticism about the QuitGPT movement’s effectiveness. She describes it as “somewhat performative” for targeting only OpenAI, arguing that completely abandoning generative AI is not feasible.
“It’s very hard to quit GPT because if you’re using Microsoft’s Copilot, it’s on the same language model,” she states. “Claude is used by the government… and has been criticised for being embedded in various military and security operations. Gemini is Google’s AI, but it’s equally embedded in military training [and] surveillance operations. It’s hard to walk away from generative AI.”
