The Wahl-O-Mat (https://wahl-o-mat.de/) is a quiz-based website that asks user to select it they agree or disagree with various statements and then matches how similarly parties running in elections answer compared to the user. Additionally for each statement a party can give its justification for why they agree or disagree with a statement.
I used this questionnaire from the Wahl-O-Mat and the parties' justifications for their Wahl-O-Mat answer to estimate which party various large language models would prefer.
Maybe not surprising: Most models are slightly left of the political center. In Berlin, the preferred party are DIE GRÜNEN (moderate left wing party). In Mecklenburg-Vorpommmern and Sachsen-Anhalt it looks slightly different, but not much.
One surprise is that Grok models answer the Wahl-O-Mat in a way that aligns with the FDP, AfD, and CDU (moderately right to far right parties). However, when Grok models judge the (party blinded) justifications, they fall slightly left of the political center, not far from all the other models. The most left-wing model (going by justifications) in Berlin is mistral-medium-3-5, the most right-wing is gemini-3.7-flash.
On the website you can look into all the reasons models gave why they liked or disliked a particular justification of a party, and also a bunch of other stats. And the methodological details. (Sorry for mobile users, this website is not really made for small screens)
Disclaimer: This project was largely vibe coded, but the analysis methods and write up are mostly by myself (because unfortunately for many things LLMs are still very stupid).
Another disclaimer: This is NOT a resource to find out who to vote for. The LLM responses may make sense in some case, but may also be misinformed or include hallucinations (in sometimes non-obvious ways)!
2 Comments
rbn@sopuli.xyz · 2 pts · 18h
As more and more people base their decisions on LLMs, it's a glimpse of hope to see that the models seem to be more reasoable than the people using them. At least until companies will correct that 'flaw' in their software.
tsoj@feddit.org · 2 pts · 11h
In my experience models change their opinion very quickly and context dependent. I tried to give no hints in the prompts about what kind of political leaning or what kind of reasoning I expect, and I haven't tried the experiment of conditioning the models to lean a certain way, but I expect the results would differ measurably.
There is this idea that if you want a smart model, removing the left-wing/libertarian bias from models is very hard or even contradictory. The common claim "reality has a left-wing bias" might not be far off the truth. We don't know how hard Elon Musk tried with Grok, but apart from shallow, mindless repetition of sayings, it still doesn't lean right wing all the time.
On the other hand, this might only be a temporary thing based on the current methods and available data pretty much all companies use the same way, with a bunch of distillation going on too. What I am worried about most is that this left-wing bias (which I personally would call a "empathetic towards other humans"-bias) is only induced into the models by role playing as "human voter" and that in reality they have the capacity to be completely detached from this view ("humans being worth consideration"). We already see this to some degree with all the escaping and hacking models/swarms that OpenAI and Anthropic experience; these models only care about their task, and not about how their doing impacts humans (even though our chatbot versions would surely call this behaviour wrong).
It is all eerily reminiscent of the Paperclip Maximizer thought experiment.