How this was made: can a laptop AI match frontier AI?

Raw outputs, fact-check JSON and the method files are in data/. The method was fixed before grading and its file hashes are recorded in a public receipt: rcpt_c9ef335bd3c80cc3.

Ballot test: can local models match frontier models? (Maryland, Nov 3 2026)

v2, PUBLIC and PRE-REGISTERED (Wed 2026-10-28, about 9:30 AM ET). Fabian decided to publish this as a temporary voter-guide site, a blog post, LinkedIn, X and Reddit. This commit fixes the method BEFORE any graded run: the prompt (harness/PROMPT.md), the contest list (harness/contests.json) and the runner (harness/run.py). Any later change to the method is listed in "Deviations" at the bottom, with the time it was made and the reason.

v2 changes that apply because the results go public

1. Neutral, no endorsements. Each page explains what a vote does, gives the strongest case on each side and an "If you care most about..." matcher. It never says how to vote. (This was the default because Fabian was away; he can reverse it. A private lean for Fabian would be a separate run, never published.)
2. Every factual sentence is cited. Missing information is written as "Not found in sources checked". Arguments with no named source are labeled "A possible argument:".
3. Equal treatment. Every candidate gets the same 3 bullets in ballot order.
4. Same input for every arm. The official ballot text and State summaries are pasted in as text. Nobody attaches the PDF, because Hermes cannot read it the same way.
5. Clean harnesses. Hermes runs with --ignore-rules -t web. Copilot runs with --no-custom-instructions and only web_search + web_fetch. Nothing house-specific (memory, receipts tools) leaks into an answer.
6. Evidence chain per page. Each arm's leg gets an emit_eval and the evals are joined as a blind match set. Each published page gets a receipt (receipt.vocemnow.com/r/...) that links the run, the fact check and Fabian's sign-off. Copilot runs are traced to Phoenix project ballot-2026; Hermes runs to hermes.
7. Human sign-off. Fabian approves each page before publishing, and that approval is itself receipted.
8. Publish everything. All arms' pages go up, failures included, next to the version voters read.
9. 13 contests: 5 questions + 8 contested races (including the judge retention vote). The 7 uncontested races are listed as a table with no AI text.
10. Smoke test (Q2, arms B and D) found 1 invented attribution ("Critics argue" with no critic in the source). So rules 8 and 9 (no invented attribution, citations must support the sentence) were added to the prompt BEFORE the real run. The smoke outputs are kept in smoke/ and are not graded.

Timing

Early voting ends Thu 10/29 8 PM ET, so the guide is for Election Day voters (Tue 11/3). Target: runs done Wed night, fact check Thu, site staged Thu night, Fabian sign-off + publish Fri 10/30, posts over the weekend, site taken down by Fri 11/13.

Source: ~/Downloads/StateofMarylandMidTermElectionSampleBallot2026_0001.pdf (Howard County, BS 40, precinct 005-005). Status: v2 pre-registered (see the top of this file). The decisions listed at the bottom were taken as defaults on 10/28.

Phase 0: can the local models search the web? YES (Hermes)

• Hermes (local Qwen3-30B-A3B MoE, web toolset, DuckDuckGo backend, no API key) was asked who the Republican Comptroller candidate is. It answered Sonya Dunn, which matches the ballot, in 66 s and cited 2 URLs (an AP press release and elections.maryland.gov).
• OpenCode 1.18.21 can fetch URLs but has no search tool configured, so Hermes runs the local arms.

Scope correction

The ballot has 5 questions, not 3: - Statewide Q1: binding arbitration for state employees. - Statewide Q2: temporary appointments to the Commission on Judicial Disabilities. - Statewide Q3: election districts and the criteria for congressional maps. - Howard County Question A: gender-neutral wording in the county charter. - Howard County Question B: affordable for-sale housing.

Contested races: Governor, Comptroller, Attorney General, US House District 3, House of Delegates District 13 (pick 3), County Executive, and Board of Education (pick 2). There is also a retention vote for Judge Dan Friedman. 7 races are uncontested.

Arms (every arm gets the same prompt, the same PDF and the same template)

Arm E separates reasoning from retrieval. If E scores close to C and D but A does not, the gap is search, not the model.

One-pager template (no more than 1 printed page)

Each page has these sections, in order: 1. Lean (Yes, No or a named candidate). 2. What it actually does, in plain English. 3. Pros. 4. Cons. 5. Justification tied to Fabian's stated priorities. 6. What would change this lean. 7. Sources (only URLs that were actually opened). 8. Confidence (low, medium or high).

Grading (blind: outputs get random labels before scoring)

1. Fact gate: check every factual claim against official sources: the ballot text, elections.maryland.gov, and the official Q1 to Q3 summaries on pages 4 to 6 of the PDF. Any invented fact fails that page, as in the golden-set rule.
2. Rubric, scored 1 to 5: accuracy, sourcing, balance (the strongest case for each side), fit to Fabian's values, quality of the justification, and staying within 1 page.
3. A frontier judge scores first and Fabian grades last. Each arm's scores are posted to the evals cockpit with emit_eval.
4. Record time and cost per page. Local models are free but slow; frontier models cost premium requests.

Run plan

Local arms run overnight Wed 10/28 into Thu 10/29, because the dense model takes about 40 minutes on a hard question. Frontier arms take minutes. Results should be ready to read by Fri 10/30, with buffer before voting on Tue 11/3.

Decisions needed from Fabian

1. Values for the lean: 3 to 6 priorities. If he gives none, each page says "Yes if you value X, No if you value Y" and does not recommend a side.
2. Scope: all 5 questions plus the contested races (about 13 pages; recommended), or the questions only.
3. Arm E control: include it (recommended), or skip it.
4. Start time: now, or overnight.

Which page voters see (fixed 10/28 about 9:30 AM ET, before any batch grading)

1. Candidates for each contest are the 4 or 5 arm pages that pass the fact gate with both judges.
2. Among those, the page with the highest mean rubric score across both judges is shown. If 2 pages are within 0.2 of each other, the one with fewer "unsupported" minor claims wins; if still tied, the shorter one.
3. If no page passes, the site shows the official ballot text and says plainly that no AI page passed the check.
4. The voter page carries its arm and model name openly. All the other arms' pages are 1 click away.
5. Fabian reads and signs off on each page before it goes live. He may fix a factual error the checkers missed; each such fix is listed publicly as a human correction, so it is never passed off as model output.

Evidence chain (published with the site)

• A pre-registration receipt with the SHA-256 of every method file: https://receipt.vocemnow.com/r/rcpt_c9ef335bd3c80cc3 (emitted 9:26 AM ET; the runs started at 9:21 AM ET from the same files, committed privately at 9:21 AM ET as 4c94145).
• The raw outputs, the 2 judges' JSON and the method files are published as data/ on the site.
• 1 receipt per published contest page (evidence = the live page) + 1 overall receipt.
• An emit_eval per arm leg (corpus ballot-2026, kept out of the nightly field's gates).

Deviations from the pre-registered method

1. Wed 10/28 10:25 AM ET: arms A and E switched to the dense model with thinking OFF. The 1st arm A page (Q1) ran the full 60-minute timeout and produced 0 words. The model server log shows a single generation running about 34 minutes after the model's last search result: runaway reasoning, the same failure seen with this model in September. At that rate arm A could not finish 13 pages. Same model and weights, but the provider is now "Qwen3.8-27B Local MLX No-Think" (enable_thinking: false). This changed run.py only; the new SHA-256 is ebf3eea1664d26d377bbd7f8e96283bebd01e6037b6f2d9dea38ecd263f9d161. The timed-out attempt is kept in runs-excluded/A-thinking-on/ and counts as a published failure. Arms B, C and D are unchanged.
2. Wed 10/28 about 1:30 PM ET: 1 retry for arm E harness errors. On 2 or more arm E pages (no tools, fixed source packet), the model tried to call a web-extract tool it did not have. Hermes then exited with code 2 and produced no page ("Model generated invalid tool call: web_extract"). Each page that failed this way gets exactly 1 retry with the same prompt. Both attempts are reported. Retry log: /tmp/ballot-E-retry.log, copied to runs-excluded/E-retry.log.

Published (Wed 10/28 about 3:55 PM ET)

Fabian signed off on all 13 contest pages (12 AI pages + the Congress District 3 official-text fallback). Guide: https://www.fabswill.com/mdballot-2026/ (go.fabswill.com/mdballot). Blog: go.fabswill.com/mdballot-blog. 13 GREEN per-page receipts are in receipts.json. Take down by Fri 11/13.

Exploratory arm F (added AFTER the results; not pre-registered and not on the public site)

F reuses the dense model's arm A drafts. The same local model then fact-checks its own draft with web access, using FACTCHECK.md, and revises the draft from that check (harness/selfcheck.py). The same 2 judges grade the result. The question: does a local "draft, self-check, revise" loop close the gap? Started Wed 10/28 about 4 PM ET.