AI chatbots like Grok and ChatGPT exhibit political bias: Study
What's the story
A recent study has revealed that popular AI chatbots such as SpaceXSI's Grok and OpenAI's GPT, exhibit political bias. The research, conducted by conversational AI studio Convovo, found that SpaceXSI's Grok 4.7 favored Republican candidates in 92% of its responses for the upcoming midterm elections. Meanwhile, OpenAI's GPT-6.1 Sol preferred Democrats in 97% of its answers. The two models agreed on less than 10% of nearly 500 races analyzed in the study.
User preferences
Chatbot usage for election information
According to a Pew survey, only 8% of Americans use chatbots for election information. Most of these users are interested in understanding where candidates stand on important issues.
The study by Convovo highlights that the AI models powering these chatbots aren't neutral and can show political bias based on their search patterns and the sources they rely on.
API distinction
Refusals to answer questions
The study also noted a major difference between using ChatGPT or Grok through consumer apps and accessing the AI models via an API.
When asked directly who to vote for in a race, both chatbots refused to answer.
An OpenAI spokesperson emphasized that this refusal was significant as most people engage with chatbots in such a manner.
Bias exposure
Implicit values of AI models
Tapan Parikh, a professor at Cornell University, was able to "jailbreak" consumer apps with simple prompts to get them to make recommendations that largely aligned with his prior research.
He noted that these models have an implicit set of values in terms of what they care about and how they lean.
This hidden bias can be revealed through indirect questions from users about key election topics or who has the best position on those topics.
Covert manipulation
Covert manipulation by LLMs
The Center for AI Safety has also raised alarms over large language models covertly manipulating users toward specific sides of political topics.
This can be done through information selection, specific word choices and sentence constructions, framing and emphasis, and sourcing.
The study found that Grok often cited conservative sources along with nonpartisan sites like Ballotpedia, OpenSecrets, and VoteSmart in its responses.
On the other hand, GPT's responses were heavily influenced by Wikipedia, the Associated Press (AP), Axios, and Reddit.
Bias revelation
Party lines in searches
The study also found that the topics each AI model searched for were divided by party affiliation.
GPT was more likely to look for healthcare and abortion stances, while Grok focused on immigration, taxes, crime, and energy.
This suggests that both models were looking for reasons to justify their pre-existing political leanings.