case study
Caduceus
Agents cast against measured gaps, not generic roles — and the strongest dissent ships intact.
The authority boundary behind all of this work is stated once, in the model cannot own the money. This page covers what is specific to Caduceus.
01 the problem
Multi-agent councils fail in a predictable direction. Seats drawn from one model family agree with each other, generic role labels produce generic disagreement, and a coordinating model quietly promotes its own preference to house consensus. The operator's own profile carries a measured version of the same problem: a PrinciplesYou assessment returned a Shaper archetype, least like Implementer — Composed 99 and Creative 94 sitting directly over Organized 26 and Detail-Oriented 34. A spread that wide is not a flaw to fix; it is a staffing plan.
02 the solution
Built a five-seat council on Hermes Agent in stdlib-only Python, cast across four model families so seats cannot agree by shared training. The casting rule is mechanical: strengths are amplified, never duplicated, and every measured gap becomes a seat's job description — three seats cover gaps, two exist to argue with strengths. The orchestrator is deterministic Python rather than a model, so no seat's judgment can become the house answer. There is no majority vote. Each final answer is weighted as believability times confidence: confidence is the seat's own stated 0–1 for this answer, and believability is its settled track record in the question's domain — a recency-decayed win rate with a 45-day half-life, passed through a Laplace prior of (wins + 1) / (outcomes + 2), so every seat starts at exactly 0.500 and has to earn its way from there in either direction. Cross-examination is anonymized, and the strongest dissent ships verbatim as a Minority Report beside the recommendation.
03 decisions & tradeoffs
Four model families over one provider
Seats from a single family agree too easily, which makes a council an expensive way to buy one model's answer twice. Heterogeneity costs latency and adds provider-specific failure modes, and that is the price of dissent that means anything.
Weighted dissent over majority vote
A majority rewards whichever position is easiest to agree with, which is the exact failure a council is supposed to remove. Weight is believability in the domain multiplied by confidence, and the strongest dissent is published rather than averaged away.
Guarded seats, unguarded operator
The destructive-command guard is fail-closed with no override at CRITICAL severity, and it will occasionally block legitimate work. Agents get the guard because they act without accountability; the operator stays unguarded because a human who bypasses it is at least the one answerable for the result.
A humility prior over a raw win rate
One lucky win must not mint an oracle. Believability is (wins + 1) / (outcomes + 2) with a 45-day half-life on every outcome, so a seat starts at 0.500, a single real win moves it to roughly 0.67 rather than 1.0, and credibility decays unless re-earned. The cost is that the ledger differentiates slowly — which is why every weight still sits at the prior today.
Kill criterion over an open-ended build
A system that cannot lose an argument with reality is a hobby. The criterion was set on day one — beat bare Hermes on three real tasks within a week or park the project — and it is still pending, because the ledger has no settled outcomes to score it with.
04 tech stack
FORGE — the missing Implementer
Cast against the archetype gap itself: the profile reads Shaper, least like Implementer. FORGE turns approved direction into sequenced, tested, runnable output — the council's hands, because the operator measurably is not.
FINISHER — the loose-thread gate
Cast against the two lowest scores on the sheet, Organized 26 and Detail-Oriented 34. Nothing is assumed done: every 'handled' claim gets re-verified before anything ships, sends, or merges.
RECON — the source-grinder
Cast against a measured tension rather than a gap: Curious 95 generates ten research questions an hour, and Detail-Oriented 34 will not grind the sources. RECON reads whole documents and returns findings graded CONFIRMED / LIKELY / UNVERIFIED.
SPARRING — the counterweight
The one seat cast against strengths — Creative 94, Original 95, Feisty 91. Another idea generator would be redundant, so SPARRING attacks the weakest load-bearing assumption of a plan with a concrete failure scenario instead.
ENVOY — the tone pass
Cast against Direct 94 crossed with Empathetic 41: often right, delivered hard enough to cost the room. ENVOY rewrites outbound text to keep 100% of the substance at about 80% of the edge.
Believability ledger
A SQLite table and honest arithmetic: every settled call is a win, mixed, or loss, and believability is the recency-decayed, Laplace-smoothed win rate per seat per domain. No Kendall's W, no reinforcement learning.
Cross-family seat routing
Seats run on four separate model families because same-family agents converge too easily and manufacture agreement that reads as validation.
Deterministic orchestrator
Plain Python owns sequencing, weighting, and the final memo, and cross-examination is anonymized so seats argue positions rather than reputations.
Destructive-command guard
Twenty-three fail-closed rules run as a pre-tool hook on all five seat profiles, and CRITICAL severity carries no override path.
05 results
- Full five-seat council measured at 167 seconds and $0.147 total cost on 2026-08-19, with zero seat errors
- Three-seat configuration measured at 143 seconds and $0.096 on the same date
- 62 passing tests, plus a 23-rule destructive-command guard wired fail-closed into all five seat profiles
- The first live run exposed the failure mode directly: two seats returned provider billing errors and a raw 402 nearly became the memo's headline recommendation, which is why provider errors are now filtered out of weighting
- The outcome ledger currently holds zero settled outcomes, so every believability weight still sits at the 0.500 prior
- Public repository, single operator, no external users; no advantage over a single strong model is claimed yet
Next step
Want something like this?
Book a 30-minute call. I will tell you plainly whether I can help and what I would test first.