case study
Caduceus
Agents cast against measured gaps, not generic roles — and the strongest dissent ships intact.
01 the problem
Multi-agent councils fail in a predictable direction. Seats drawn from one model family agree with each other, generic role labels produce generic disagreement, and a coordinating model quietly promotes its own preference to house consensus. The operator's own profile carries a measured version of the same problem: a PrinciplesYou assessment returned a Shaper archetype, least like Implementer — Composed 99 and Creative 94 sitting directly over Organized 26 and Detail-Oriented 34. A spread that wide is not a flaw to fix; it is a staffing plan.
02 the solution
Built a five-seat council on Hermes Agent in stdlib-only Python, cast across four model families so seats cannot agree by shared training. The casting rule is mechanical: strengths are amplified, never duplicated, and every measured gap becomes a seat's job description — three seats cover gaps, two exist to argue with strengths. The orchestrator is deterministic Python rather than a model, so no seat's judgment can become the house answer. There is no majority vote. Each final answer is weighted as believability times confidence: confidence is the seat's own stated 0–1 for this answer, and believability is its settled track record in the question's domain — a recency-decayed win rate with a 45-day half-life, passed through a Laplace prior of (wins + 1) / (outcomes + 2), so every seat starts at exactly 0.500 and has to earn its way from there in either direction. Cross-examination is anonymized, and the strongest dissent ships verbatim as a Minority Report beside the recommendation.
03decisions & tradeoffs
Four model families over one provider
Seats from a single family agree too easily, which makes a council an expensive way to buy one model's answer twice. Heterogeneity costs latency and adds provider-specific failure modes, and that is the price of dissent that means anything.
Weighted dissent over majority vote
A majority rewards whichever position is easiest to agree with, which is the exact failure a council is supposed to remove. Weight is believability in the domain multiplied by confidence, and the strongest dissent is published rather than averaged away.
Guarded seats, unguarded operator
The destructive-command guard is fail-closed with no override at CRITICAL severity, and it will occasionally block legitimate work. Agents get the guard because they act without accountability; the operator stays unguarded because a human who bypasses it is at least the one answerable for the result.
A humility prior over a raw win rate
One lucky win must not mint an oracle. Believability is (wins + 1) / (outcomes + 2) with a 45-day half-life on every outcome, so a seat starts at 0.500, a single real win moves it to roughly 0.67 rather than 1.0, and credibility decays unless re-earned. The cost is that the ledger differentiates slowly — which is why every weight still sits at the prior today.
Kill criterion over an open-ended build
A system that cannot lose an argument with reality is a hobby. The criterion was set on day one — beat bare Hermes on three real tasks within a week or park the project — and it is still pending, because the ledger has no settled outcomes to score it with.
04 tech stack
FORGE — the missing Implementer
Cast against the archetype gap itself: the profile reads Shaper, least like Implementer. FORGE turns approved direction into sequenced, tested, runnable output — the council's hands, because the operator measurably is not.
FINISHER — the loose-thread gate
Cast against the two lowest scores on the sheet, Organized 26 and Detail-Oriented 34. Nothing is assumed done: every 'handled' claim gets re-verified before anything ships, sends, or merges.
RECON — the source-grinder
Cast against a measured tension rather than a gap: Curious 95 generates ten research questions an hour, and Detail-Oriented 34 will not grind the sources. RECON reads whole documents and returns findings graded CONFIRMED / LIKELY / UNVERIFIED.
SPARRING — the counterweight
The one seat cast against strengths — Creative 94, Original 95, Feisty 91. Another idea generator would be redundant, so SPARRING attacks the weakest load-bearing assumption of a plan with a concrete failure scenario instead.
ENVOY — the tone pass
Cast against Direct 94 crossed with Empathetic 41: often right, delivered hard enough to cost the room. ENVOY rewrites outbound text to keep 100% of the substance at about 80% of the edge.
Believability ledger
A SQLite table and honest arithmetic: every settled call is a win, mixed, or loss, and believability is the recency-decayed, Laplace-smoothed win rate per seat per domain. No Kendall's W, no reinforcement learning.
Cross-family seat routing
Seats run on four separate model families because same-family agents converge too easily and manufacture agreement that reads as validation.
Deterministic orchestrator
Plain Python owns sequencing, weighting, and the final memo, and cross-examination is anonymized so seats argue positions rather than reputations.
Destructive-command guard
Twenty-three fail-closed rules run as a pre-tool hook on all five seat profiles, and CRITICAL severity carries no override path.
05 results
- Full five-seat council measured at 167 seconds and $0.147 total cost on 2026-08-19, with zero seat errors
- Three-seat configuration measured at 143 seconds and $0.096 on the same date
- 62 passing tests, plus a 23-rule destructive-command guard wired fail-closed into all five seat profiles
- The first live run exposed the failure mode directly: two seats returned provider billing errors and a raw 402 nearly became the memo's headline recommendation, which is why provider errors are now filtered out of weighting
- The outcome ledger currently holds zero settled outcomes, so every believability weight still sits at the 0.500 prior
- Private repository, single operator, no external users; no advantage over a single strong model is claimed yet