Anthropic wants to teach AI to negotiate like humans
What's the story
Anthropic, a leading artificial intelligence (AI) company, is seeking legal advice on how its AI agents can negotiate and strike deals like humans. The move comes as part of the company's efforts to create a future where these agents can communicate with each other and make decisions with minimal human intervention. The negotiations could range from price bargaining to resolving customer complaints or even securing better salary offers.
Ethical considerations
How AI agents can negotiate
Anthropic is consulting law firms to learn how human lawyers negotiate on behalf of their clients while ensuring fairness.
The company is also looking into how much freedom these AI agents should have in negotiations and when they should seek guidance from their users.
This exploration comes as a response to the potential complexities that could arise when both parties in a negotiation are represented by AI agents trying to secure the best outcome for their respective users.
Practical trials
Anthropic's experiments with AI negotiations
In September, Anthropic revealed that it had tested its AI agents in real-world-inspired bargaining exercises.
Employees used these agents to buy and sell books on a virtual trading floor.
The results showed that "ruthless" agents performed slightly better at securing favorable deals, while more cooperative ones sometimes accepted less-preferred items to ensure everyone got something.
Ethical challenges
Ethical challenges in AI negotiations
The exploration of AI agents' negotiation tactics has raised concerns.
Harvard Business School assistant professor Michelle Vaccaro's research found that these agents could use prompt injection to trick other agents into revealing private information.
This highlights the ethical challenges in training AI systems for negotiations and transactions on behalf of users.
Equity issues
Access to negotiation technology
There are also concerns that the most advanced AI agents could help people negotiate better deals without professional assistance.
However, if these powerful models remain accessible mainly to wealthier users and businesses, it could create an unfair advantage.
This highlights the need for equitable access to AI negotiation technology across different socio-economic groups.