BLUF (bottom line up front): I built a free, non-partisan guide to the Maryland ballot for Tuesday, November 3: go.fabswill.com/mdballot. It goes down every question and race on the Howard County ballot, in ballot order, and for each one tells you what your vote actually does, the strongest case on each side and who is making it. Every claim links to its source. It never tells you how to vote.
AI agents did the research. 2 other AI models fact-checked every single claim. I read and signed off on every page. And every step left a receipt you can open yourself.
The goal is simple: a more informed electorate. AI does the reading that none of us has time for, and puts honest, balanced, sourced information in front of you. You make the decision. That’s what I think AI is for.
Why I did this
My sample ballot has 20 items: 15 offices, 5 questions. Be honest: how many of those could you explain in the booth? Most of us show up knowing the Governor’s race and maybe Congress, then guess our way through the Comptroller, a judge retention vote and 3 constitutional amendments written in legislative English. Question 3 alone is about “original jurisdiction” and “criteria for the boundaries of a congressional districting plan”. That’s not a character flaw. It’s a volume problem. There is more to read than any one person has time for.
That is exactly the problem agents should solve: the human stays behind the decision, and the agent scales the human. The research is the agent’s job. The checking is a different agent’s job. The vote is yours.
How it works (and how I kept myself honest)
I wrote the rules down before I saw a single result. The prompt, the fact-check rubric, the contest list and the scripts were hashed (SHA-256) and the hashes were recorded in a public receipt at 9:26 AM ET on Wednesday, October 28: rcpt_c9ef335bd3c80cc3. The files are published with the guide, so you can check that nothing moved after the fact.
Every model got the identical prompt, the official ballot text and the State’s official summaries, and the same one-page template:
- In one sentence: what this decides
- What a FOR vote does / what an AGAINST vote does (or: each candidate, same 3 bullets, in ballot order)
- Strongest case FOR and AGAINST, each cited, and if nobody on record actually said it, it must be labeled “A possible argument”
- “If you care most about…” so you can match the choice to your priorities, not mine
- What we could not verify, confidence, sources
The rules that matter most: never invent, no endorsements, equal treatment for every candidate.
| Arm | Model | Where it ran | Search |
|---|---|---|---|
| A | Qwen3.8-27B dense | My MacBook Pro M3 Max, via Hermes | live web |
| B | Qwen3-30B-A3B MoE | My MacBook Pro M3 Max, via Hermes | live web |
| C | Claude Opus 5.5 | GitHub Copilot CLI | live web |
| D | GPT-6.1-sol | GitHub Copilot CLI | live web |
| E | Qwen3.8-27B dense | My MacBook Pro, no search, given the same sources C and D found | none |
Then 2 judges that are not contestants, Claude Sonnet 5.5 and GPT-5.6-sol, checked every page blind (they never knew which model wrote it). They split each page into individual factual claims, opened every cited URL and marked each claim supported, cited wrong, unsupported or contradicted. A page fails if it has a single major claim that’s contradicted or unsupported, a single invented attribution, or a single endorsement. Strict on purpose: a voter can’t check what we couldn’t check.
The page you read for each contest is the highest-scoring page that passed both judges. Every other model’s page is 1 click away, failures included, with the reason it failed.
What happened
All 5 arms, 13 contests each, both judges, finished Wednesday, October 28 at 2:25 PM ET:
| Model | Where | Pages passing both fact checks | Mean judge score /5 | Invented attributions (avg per judge) |
|---|---|---|---|---|
| GPT-6.1-sol | frontier | 12 of 13 | 4.42 | 0 |
| Claude Opus 5.5 | frontier | 6 of 13 | 4.06 | 2 |
| Qwen3.8-27B dense, live search | my MacBook Pro | 0 of 13 | 2.82 | 14 |
| Qwen3-30B MoE, live search | my MacBook Pro | 0 of 13 | 2.18 | 42 |
| Qwen3.8-27B dense, fixed sources, no search | my MacBook Pro | 0 of 13 | 2.92 | 4 |
3 things jumped out.
1. The local model with search makes up who said what. On Question 2 (temporary appointments to the Commission on Judicial Disabilities), my local MoE model correctly named the Commission’s chair, then attributed an endorsement and a statement to her that no source we could find supports, and added the Maryland State Bar Association as a supporter, also unsupported. It cited Ballotpedia for both. That’s the scariest failure mode for a voter guide: it sounds exactly right. This is why the judges exist and why the controls sit outside the model.
2. The frontier models fail differently. Claude Opus 5.5 wrote the richest pages (and blew past the length limit on every one of them), and quoted named county officials from Baltimore Sun articles. The judges couldn’t open those articles (paywall), so those pages failed. Not because the quotes are wrong, but because neither you nor I can verify them. GPT-6.1-sol was tighter, shorter and almost always verifiable. Its one failure: it described a candidate’s “infrastructure renewal” as “roads and bridges”. A small embellishment, and the gate caught it.
3. The judges are not perfect either. They agreed on the pass/fail call on 54 of 65 pages. Where they disagree, the page doesn’t make it. And I caught 1 judge passing a page that was nothing but a harness error message (no claims, so nothing to contradict). Requiring both judges to pass is what kept that off the site.
Local vs frontier: is it the model or the search?
That’s what arm E is for. Same local dense model, no web search, handed the exact sources the frontier models found. If E passes where A fails, the gap is retrieval (searching and reading the web), not reasoning, and that’s a fixable plumbing problem on my side. If E fails too, the model itself isn’t ready for this job yet. The answer: both, and my plumbing too. Handing the dense model the right sources cut its invented attributions from 14 to 4 and its unsupported claims by about 70%. So search is a big part of the gap. But it still passed 0 of 13, mostly by misreading or over-reading the sources it was given, and some of that is on me: my source packets missed content from sites that blocked my fetcher or render with JavaScript, and the model then honestly wrote “not found”, which the judges marked wrong because the live site had it. So arm E narrows the question; it doesn’t close it.
2 more honest notes from the run: the dense model with “thinking” on ran a single answer for 34 minutes and hit my 60-minute timeout with 0 words, so I switched arms A and E to thinking off, and logged it publicly as a deviation with the new file hash. And the local pages took 9 to 29 minutes each, versus about 1 minute for GPT-6.1-sol. I can hear the MacBook Pro M3 Max fan spinning on this as I write :-)
The finding that stings: my own eval was fooled
In Trust the Model the Least I showed my evals cockpit saying FLIP: my local dense model had cleared the bar for compiling my knowledge wiki. So I ran every ballot page through that same house judge too, and compared it with the 2 fact checkers that open every source:
| Model | My house eval judge /5 | Fact-check judges /5 | Pages passing both fact checks |
|---|---|---|---|
| GPT-6.1-sol (frontier) | 2.38 | 4.43 | 12 of 13 |
| Qwen3.8-27B dense, live search (local) | 2.28 | 2.79 | 0 of 13 |
| Qwen3.8-27B dense, given sources (local) | 2.27 | 2.92 | 0 of 13 |
| Claude Opus 5.5 (frontier) | 2.25 | 4.06 | 6 of 13 |
| Qwen3-30B MoE, live search (local) | 1.35 | 2.18 | 0 of 13 |
(Both score columns are averaged over the same 62 pages, the ones my house judge scored. It returned no score for the other 3 of 65.)
My house judge put the local dense model ABOVE Claude Opus 5.5, and within 0.1 of GPT-6.1-sol, on pages where the local model passed 0 of 13 fact checks and GPT passed 12. The fact checkers see a gap of 1.64 points; my judge sees 0.10. It reads the page and grades how well it’s written. It never opens a source, so it can’t tell a confident fabrication from a fact. A judge that can’t see fabrication will happily tell you to FLIP.
So the FLIP is on hold. Before any local model graduates to work that matters, it has to pass the same claim-by-claim, source-opening fact gate this guide used. That is the most valuable thing this little election project taught me, and it’s about my own system, not the models.
The human behind it
I read every page before it went live, and my sign-off is its own receipt. If I fixed something a judge missed, it’s listed on the page as a human correction, never passed off as model output. Every published page shows its receipt link right next to its fact-check result, so you can open it and see what was claimed and when.
Where I’m not there yet
- AI fact checkers can be wrong. 2 judges from 2 different model families lowers the odds; it doesn’t zero them. That’s why every claim links to its source: check me.
- Paywalls make good journalism unverifiable for this kind of guide. That’s a real cost, not a win.
- One ballot style. This is Howard County ballot style BS 40. Statewide items apply to every Maryland voter; county and district items depend on your address.
- Local models aren’t there for public-facing facts yet. 0 of 39 local pages passed both checks. For my own research, with me reading the sources, sure. For something strangers rely on to vote, not yet.
- My source packets were imperfect (7 of 138 sources blocked, some JavaScript pages thin), which muddies the “is it the model or the search” answer.
- Some contests have no AI page at all because nothing passed both checks; there you get the official text and links, not a guess.
- This comes down after the election. It’s a snapshot, researched Wednesday, October 28, 2026.
How to use it
Open go.fabswill.com/mdballot on your phone, scroll your ballot top to bottom, tap any race or question. Read “If you care most about…” and pick by your priorities. Tap a source if something surprises you. Share it with 1 neighbour who’s still undecided. Then go vote: polls are open 7 AM to 8 PM ET on Tuesday, November 3, and mail-in ballots must be postmarked by November 3 or dropped in an official drop box by 8 PM that day. Find your polling place.
By day I work at Microsoft, where my focus is observability. This is a personal project on my own machines and my own time, the same receipts and evals I run at home every night, pointed at something my neighbours can use. It isn't authorized by or coordinated with any candidate, campaign or party, and it endorses no one :-)