Models & LabsUnited States
Anthropic bans users from being 'cruel' to its AI systems
Useful
The firm said users can no longer engage in "sustained and needless" abusive behaviour towards the tech.
View the articleBBC
Models & LabsUnited States
The firm said users can no longer engage in "sustained and needless" abusive behaviour towards the tech.
Anthropic has raised eyebrows by announcing it will ban users from "sustained and needless" abusive behaviour towards its AI.
The Claude-maker said on Thursday it was updating its usage policy to let its tools end interactions where people are "cruel" - something it said it had done in "rare" cases prior.
It will now be included on a list of forbidden conduct which also contains bullying others, promoting self-harm and creating non-consensual intimate imagery.
While Anthropic said the policy would only apply in "extreme cases" of repeated abuse, it has reignited debate over how we should communicate with AI tools.
The company said it would not apply to "common versions of user frustration, pushback, dark creative themes, or model testing and research" - suggesting it would only apply in clearly deliberate instances.
But what Anthropic would consider "abusive or cruel behaviour" towards its models remains unclear.
Screenshots of the new policy have been widely shared and commented on across social media - with some praising it as advancing good manners or "model welfare".
But others criticised the plan, with one person calling it " deeply wrong, external " and another suggesting it might undermine abuse directed at humans and animals.
"You can't be cruel to numbers and maths," wrote Dr Barry Scannell, technology partner at Irish law firm William Fry, on LinkedIn, external.
"This level of anthropomorphisation of AI is harmful. It leads people to believe that it's something it's not."
Microsoft AI director Mustafa Suleyman, who in September criticised rival firm Anthropic for treating AI like it is human, reacted to Anthropic's update with a melting face emoji.
What is AI, how does it work and why are some people concerned about it?
The people who think AI might become conscious
The changes, first reported by tech outlet The Verge, came following an annual review of the usage policy at the firm.
It was also updated to address new or emerging risks in other areas, including by forbidding use of Claude for "deceptive campaigns" or clarifying that, as US midterm elections loom, it cannot be used to "deceive voters or disrupt elections".
Anthropic also said on Thursday its usage policy previously prevented its tools being used to develop weapons.
But none of the updates have attracted as much attention or scrutiny as its move to forbid users abusing its chatbots.
It has also tapped into the much-debated question of whether we should be curt or polite when using generative AI tools.
Some experts say writing or saying "please" or "thank you" to tools like ChatGPT is unnecessary and wastes tokens - used by the large language models powering generative AI tools to process and reply to you.
Others argue politeness can lead to higher quality answers or responses.
When a X user last year asked about electricity costs incurred by ChatGPT replying to users' "please" and "thank you" messages, OpenAI boss Sam Altman said it was, external "tens of millions of dollars well spent - you never know".
The Collins Dictionary meanwhile last November highlighted the re-emergence of the term "clanker" for disparagingly referring to AI tools.
Used to describe robots in Star Wars games and movies since the mid 2000s, the term made its shortlist of words reflecting the mood of 2025 after last July saw many take to TikTok to share their frustrations with AI-powered machines.
Some experts have also posited that being rude to AI chatbots could adversely impact human communication, by seeping into how we talk to and treat others.
To play this video you need to enable JavaScript in your browser.
Related topics
Why friendly AI chatbots might be less trustworthy
Why are there concerns AI could threaten humanity, and how real are they?