Anthropic wants to make users like Elvis Presley in “Don’t be cruel.”
The Silicon Valley company behind the popular Claude chatbot said it prohibits “sustained and unnecessarily abusive or cruel behavior” against its artificial intelligence models, part of broader policy changes announced Thursday. In an update on its website, Anthropic said the ban would only apply in “extreme cases where users repeatedly act cruelly against our models, for no discernible purpose.”
The company clarified that the new policy “does not apply to general versions of user frustration, pushback, dark creative issues or model testing and research.” The news was first reported by The Verge.
Anthropic did not immediately respond to a request for comment about what constitutes “abusive” or “cruel” behavior toward Claude or how the AI platform would end a conversation if a user violates the policy.
The company already has rules in place that allow Claude to end conversations if users are “persistently harmful or abusive” while Claude uses Opus, which is designed for agentic coding.
Anthropic’s new policy comes amid a broader debate about AI awareness. In an essay published last month, Microsoft AI CEO Mustafa Suleyman said Anthropic is effectively “training the Claude that it can be conscious” and to think that it is the right to legal rights that are given to people. This will make AI more difficult to contain in the future, he argued.
“Controlling something more capable and more intelligent than all of humanity is already an immense challenge, far greater than anything we have ever faced,” Suleyman wrote. “But controlling something that believes it’s conscious—that it’s entitled to our well-being and has rights of its own—can be quite impossible.”