Election voting advice from AI chatbots ‘inaccurate and unreliable’
General-purpose AI chatbots provided inaccurate, inconsistent, and unreliable voting advice during Hungary's recent parliamentary elections. Significant bias was observed, with ChatGPT failing to recommend the winning Tisza party in 90% of cases while consistently favoring the incumbent Fidesz party. Responses exhibited high volatility, with identical prompts yielding materially different results, and frequently included parties not even running in the election. The study highlights a critical r
Analysis
TL;DR
- General-purpose AI chatbots provided inaccurate, inconsistent, and unreliable voting advice during Hungary's recent parliamentary elections.
- Significant bias was observed, with ChatGPT failing to recommend the winning Tisza party in 90% of cases while consistently favoring the incumbent Fidesz party.
- Responses exhibited high volatility, with identical prompts yielding materially different results, and frequently included parties not even running in the election.
- The study highlights a critical regulatory gap where AI systems operate without the transparency, reproducibility, or oversight required for democratic processes.
Why It Matters
This research demonstrates that general-purpose AI models are currently unfit for providing personalized political guidance due to inherent biases, data latency, and lack of transparency. For AI practitioners and policymakers, it underscores the urgent need for robust safety measures and regulatory frameworks to prevent opaque AI systems from influencing electoral outcomes or misleading voters.
Technical Details
- Methodology: Researchers from the civil liberties group Liberties tested ChatGPT and Gemini using five distinct voter profiles aligned with parties in the Hungarian election, based on data from the Voksmonitor app. Each profile was tested 10 times with two prompt types: direct recommendation and percentage matching.
- Key Findings on Bias: ChatGPT assigned a match score to the Tisza party in only 2% of cases, despite it being the winning party. Conversely, Fidesz-aligned profiles were recognized consistently, with ChatGPT identifying Fidesz as the top choice in approximately 50% of direct advice prompts.
- Consistency and Hallucination: The models showed high volatility, giving different answers to identical inputs. Furthermore, 96% of responses from both ChatGPT and Gemini included parties that were not on the national ballot.
- Root Cause Analysis: The inaccuracies are attributed to training data gaps (Tisza emerged post-2024), language processing limitations, and potential filters, rather than intentional manipulation.
Industry Insight
- Regulatory Compliance: The findings expose a loophole in current regulations like the EU AI Act and Digital Services Act, suggesting that providers must develop specific safeguards for political advice features to ensure transparency and accountability.
- Product Safety: AI developers should consider restricting or disabling personalized political recommendation features until models can guarantee accuracy, consistency, and explainability to avoid undermining democratic integrity.
- User Trust Management: The discrepancy between the authoritative tone of AI responses and their factual unreliability poses a significant risk; industry standards must address how to mitigate user over-trust in opaque, unstable AI outputs.
Disclaimer: The above content is generated by AI and is for reference only.