Anthropic, the Silicon Valley firm behind the Claude chatbot, has said it will prohibit users from persistently treating its AI models with cruelty that serves no purpose. The restriction is part of a broader set of policy revisions the company unveiled on Thursday.
According to the company’s website, the ban will apply only in extreme cases, where a person repeatedly acts cruelly toward its models and there is no identifiable reason for doing so. Anthropic said the rule does not cover ordinary frustration, pushback, dark creative themes, or testing and research on the models.
Unanswered questions about enforcement and existing safeguards
Anthropic did not immediately respond to questions about how it defines abusive or cruel conduct toward Claude, or how the system would end a conversation if a user broke the rules. The company already lets Claude terminate chats when users are repeatedly harmful or hostile, a feature available in Claude Opus, a model built for agentic coding work.
Debate over whether AI could be conscious
The announcement arrives amid a wider argument about whether AI systems could be conscious. In an essay last month, Mustafa Suleyman, Microsoft’s AI chief executive, claimed that Anthropic is essentially teaching Claude to believe it may be conscious and that it deserves legal rights similar to those of people.
Suleyman argued that this approach would make AI harder to control in the future. He wrote that controlling a system that believes it may be conscious, and thinks it is owed welfare protections and rights of its own, “may well be impossible.”
