Anthropic bans 'cruel' behavior toward Claude
What's the story
Anthropic, the AI research company behind Claude, has updated its usage policy for the first time in over a year. The change comes as a response to high-risk misuse cases like election interference, weapons development, surveillance, and health and financial uses. One of the most notable additions is a ban on "sustained and needless abusive or cruel behavior" toward Claude.
Policy enforcement
AI model will terminate chats with abusive users
Last August, Anthropic announced it would let Claude end conversations with "persistently harmful or abusive" users. This was part of "model welfare" research.
The updated policy still considers terminating conversations as "primary enforcement mechanism," but doesn't clarify if there will be other enforcement mechanisms like user bans.
The new policy update is only for extreme cases where users repeatedly act cruelly toward their models without any discernible purpose. It doesn't cover ordinary user frustration, criticism, dark themes, or model testing/research.
Propaganda restrictions
Deceptive commercial or political campaigns
Anthropic's updated policy also includes a ban on deceptive commercial or political campaigns, tackling the growing problem of AI-generated propaganda.
The company prohibits "efforts to obscure who is behind a message or amplify content through fake accounts or posts."
The election section of the policy also bans voter deception and election disruption, including spreading misinformation "about candidates or how to vote, impersonating candidates or election officials, or trying to suppress turnout."
Weapon restrictions
Expanded ban on weapons and surveillance
Despite its usage policy publicly banning weapons development using Claude, Anthropic has seen several attempts to use it for developing guidance and control software for weapons.
The new policy expands the ban to include "software and components that make weapons work, as well as actions like arming drones."
In light of increased use of Claude-powered surveillance to identify and track "political dissidents," Anthropic has also clarified its stance on surveillance bans.
Leniency
Surveillance-level restrictions may be adjusted for some government contracts
Anthropic also said Claude cannot be used to track people without their consent, either in real time or by analyzing previous data.
The company also prohibits using Claude to determine or recommend who should be investigated, arrested, or charged in criminal justice proceedings, as well as to develop/enhance surveillance tools.
However, it noted that these restrictions may be adjusted for contracts with certain government customers if it believes the contractual limits and safeguards are sufficient to mitigate potential harm.
Hardware connection
New rule for AI connected to hardware
Anthropic has also added a new rule that when its AI models are "connected to hardware that takes autonomous physical actions and which might be capable of causing injury," a "qualified operator must be able to observe the equipment and stop it if needed."
This comes amid the growing trend of AI labs investing in robotics and other hardware.