case study
Caliber
A team of agents, or a team of agents fitted to you. Concept stage, not a product.
01 the problem
Agent teams have converged on one shape: a planner, a researcher, a critic, the same cast handed to every operator. That team is a commodity — it can be reproduced in an afternoon, and it is calibrated to nobody in particular. It also carries a quieter failure, because agents drawn from a single model family agree with each other faster than they surface the mistake a specific person is most likely to make.
02 the solution
Caliber is in design. A user takes a public-domain IPIP Big Five assessment, and the measured scores set the calibration dials on three agents — a doer, a challenger, and a finisher — weighted toward the traits the assessment scores lowest rather than toward the operator's existing defaults. The three models stay fixed for every user; only the profile layer moves, which holds inference cost flat and leaves one behavior set to regression-test instead of one per customer. The mechanism runs today as a single-operator reference implementation in a private repository: a five-seat council cast across four model families against the operator's own assessment.
03decisions & tradeoffs
Fixed models, per-person calibration
Swapping model families per user would personalize harder, and it would also multiply both the inference bill and the surface that has to be evaluated. Keeping the roster fixed pushes personalization into the profile layer, which is cheap to change and cheap to test.
Public-domain IPIP over a proprietary instrument
PrinciplesYou is the better-known assessment, it is not open source, and its owner already sells a B2B version, so wrapping it is neither licensable nor safe to build on. IPIP is public domain and free for commercial use, which is the only version of this that can ship.
Deterministic orchestrator over a lead agent
A model appointed to arbitrate becomes the answer, and the other agents become decoration. Weighting and assembly stay in plain code so a minority report survives all the way to the reader instead of being summarized away.
Provider errors excluded from weighting
On the reference implementation's first live run an inherited configuration produced two provider billing errors, and a raw 402 response came close to becoming the memo's headline recommendation. An agent that failed to answer is not a dissent, so error drafts are now filtered out before anything is weighted.
The ledger as the moat, not the quiz
The assessment is cheap to clone and will be cloned. The only part that compounds is a per-user record of which agent was right in which domain, and that record does not exist until real users settle real outcomes.
04 tech stack
Public-domain IPIP assessment
The International Personality Item Pool is public domain and free for commercial use, which makes it the only instrument here that can be shipped without a license.
Complement casting
Measured scores set each agent's calibration dials to cover the traits scored lowest, so the team is fitted to the gaps rather than to the preferences.
Fixed three-model roster
Doer, challenger, and finisher run on the same models for every user; personalization lives entirely in the profile and soul layer.
Deterministic orchestrator
Weighting, anonymized cross-examination, and the final memo are plain code, so no agent's judgment becomes the house answer and the strongest dissent ships verbatim.
Per-user outcome ledger
Records which agent was right, per domain, decaying by recency — the component intended to compound, and the one that does not exist yet.
05 results
- Concept and design stage: no product, no users, and no revenue
- The casting mechanism runs only as a single-operator reference implementation in a private repository
- Reference implementation measured on 2026-08-19 at 167 seconds and $0.147 for a full five-seat run, with 62 passing tests
- Outcome ledger holds zero settled outcomes, so every believability weight still sits at its 0.500 prior
- Kill criterion set on day one: beat a single strong model on three real tasks inside a week, or park it