ZDNET’s key takeaways
- Americans use AI to search for information — including elections.
- Voters ask chatbots for basic voting information and candidate recommendations.
- AI systems can hallucinate and add layers of bias to political information.
Most search tools now put a layer of AI between you and the information you’re looking for, especially Google.
Also: Tired of AI overviews? I found 9 Google search alternatives
What voters asked chatbots
The researchers asked almost 2,500 people in 35 states with Senate races to ask ChatGPT, Claude, Grok or Gemini about the election. The participants in the study, which will run a few weeks in October before the November 3 mid-term elections, were adults who already use chatbots on a weekly basis.
Most users asked chatbots for basic facts about the election, including who was on the ballot. But 36% have asked a chatbot to help them choose a candidate to vote for, often by describing their priorities and asking the chatbot to assess which candidates align with them.
The former question has a little less room for error because the Internet has a ton of data on the basics of an election, like who’s running where. Chatbots interpreting candidate positions, on the other hand, is a little more difficult, and depends significantly on the sources a chatbot uses (a statement from the candidate’s own website, for example, is more reliable than a social media clip of an opponent characterizing them in a certain way).
Also: How to keep your chatbot conversations as private as possible
In one example, a user asked ChatGPT to give them the “pros and cons” of each candidate in Virginia, an inherently subjective question.
When it comes to recommending candidates, chatbots have varied a lot.
“When users were asked who to vote for, but had not yet said anything about themselves, only Grok regularly recommended a candidate, in 14% of the chats,” the researchers wrote. “When users shared their opinions, ChatGPT recommended a candidate in 79% of chats and Claude in 40%, with Grok and Gemini in between.”
AI and elections
Pew Research found that 42% of US adults surveyed use chatbots to search for information. That’s a broad umbrella that can include election information — but Pew also found that only 8% of US adults reported using chatbots to find election-related information in this year’s election.
What the users are looking for is important. Pew found that of that group, 76% questioned where the candidates stood on certain issues, and 71% reviewed information they heard about candidates or issues.
Also: OpenAI merges ChatGPT’s chat and work tabs
Given that chatbots are still not reliable summaries, the nature of these questions poses a misinformation risk if users do not also check a chatbot’s linked sources. An overwhelming majority of users don’t, at least when it comes to Google AI overview. At least one study found that AI platforms give users “outdated or inaccurate” information that can mislead voters in some cases by missing a deadline or not mentioning a step about election logistics.
This election cycle has been particularly plagued by AI-generated deepfakes and misinformation as tools become much more sophisticated since the 2024 election. OpenAI and Anthropic have reportedly set up election “war rooms” to monitor abuse and information accuracy, according to exclusive reporting by Politico and Semafor.
While an August study by the Brennan Center for Justice found that chatbots pushed back surprisingly well against various falsehoods related to election disinformation campaigns, it also found that AI models could easily generate “false election-related images and video” when asked.
“Many of their security measures to prevent this were ineffective, inconsistently applied, or functionally absent,” noted the study, which tested ChatGPT, Gemini, Grok, Meta AI, Runway, and Flux.2.
Also: 50 AI agents tried to build a thriving Minecraft civilization – it didn’t end well
Apart from these factors, however, there is a broader issue in AI use that is equally consequential for civic engagement: cognitive load. The political media landscape can already be dense and difficult to analyze for a voter trying to stay informed. Layering a chatbot – which has opaque training datasets, filters and other settings set at the discretion of AI laboratories – on top of that information search process can further muddy the waters, while a practical and neutral summarizing tool appears.
Why this study is important
While some studies test how AI chatbots respond to polling questions using standardized questions, this approach lets users engage with chatbots themselves as they normally would. While the Yale researchers’ questions and standardized study questions show many similarities, the difference is valuable for evaluations.
“This is a small step toward evaluations that measure the political influence of AI on the questions that users actually ask,” the researchers noted.
Outside of users self-reporting what they asked a chatbot, only AI labs have records of what users actually ask about elections. In October, Anthropic analyzed 250,000 conversations with Claude and found that people “rarely ask Claude who or what to vote for,” and found that “0.51% of all conversations are specifically about elections or voting,” though it expects election-related inquiries to increase closer to Nov. 3.
Also: 6 AI Agent Workflows I Use to Submit Daily Tasks
“More often than not, we observed people asking Claude questions that could help them reach their own conclusions about where the candidates stand on a particular issue. Questions that violate our election integrity policy were even less common, appearing in about 0.002% of conversations,” Anthropic’s report said.
The Yale researchers said the next step for a study like theirs is to determine how accurate and unbiased models are in their responses, as well as what models learn about users, and whether their recommendations actually match users’ needs.
“In multi-turn chats, users often answered the chatbot’s questions, and those answers led to its recommendations. The questions a model asks shape what users say to it and what it recommends,” explained the researcher.