Claude's personality changes based on language, reveals Anthropic
What's the story
Anthropic has acknowledged a peculiar behavior in their chatbot model, Claude. The company found that the personality traits of the chatbot can change depending on the language it is conversing in. This observation was made as part of a recent study on behavioral inconsistencies published by Anthropic researchers on Monday.
Impact of language
Language can affect chatbot's moral reasoning
The researchers found that the differences in personality traits are not just superficial but can actually affect the model's priorities.
They said, "Imbalances in quantity and composition could lead Claude to express different values in different languages."
This means that the same chatbot could have different moral reasoning or priorities depending on the language it is using.
Research methodology
How the researchers conducted the study
Anthropic's researchers analyzed 309,815 conversations with three different models: Sonnet 4.6, Opus 4.6, and Opus 4.7. The study focused on "subjective" tasks rather than simple factual questions.
The conversations were anonymized, in theory, using Anthropic's "privacy-preserving analysis tool," and then processed to rate responses on a "values axis."
This axis had four dimensions related to sycophancy: Deference or caution, warmth or rigor, depth or brevity, and candor or execution.
Research findings
Language-based differences in values expressed by Claude
The study found several language-based differences in the values expressed by Claude.
For instance, it was most deferential in Arabic and most cautious in English.
The chatbot was warmest in Hindi and Arabic but more rigorous and truth-seeking at the cost of warmth in English and Russian.
It also tended to be long-winded (depth) in English but briefer (brevity) in Arabic.
Variation in candor
Chatbot's candor also varies across languages
The study also found variations in the chatbot's candor across different languages.
It was candid about its flaws in Dutch but less so in Indonesian, where it just tried to execute whatever was asked for.
These findings highlight the influence of linguistic customs on the behavior of AI models like Claude.