Medical LLM Council
Put one medical case to a council of LLMs. Each model answers on its own, then reviews the others' answers without knowing who wrote them, and the answers are merged by rules built around clinical risk, so that one model's warning is never outvoted by three models' reassurance.
The idea
Andrej Karpathy's llm-council asks several LLMs the same question, has each one rank the others' anonymised answers, and lets a "chairman" model write the final reply. It is a simple, clever way to get a second, third and fourth opinion from machines.
Medicine needs one change to that design. In most domains, averaging opinions is fine: if three models are right and one is wrong, going with the majority wins. In medicine the errors are not symmetric. Wrongly reassuring someone with a heart attack is far worse than sending someone with indigestion for an ECG. A merge that averages can bury the one answer that mattered.
A worked example
A 52-year-old has had chest tightness for an hour. Three council models say “likely musculoskeletal, reassure and follow up”. The fourth says “cannot exclude acute coronary syndrome, needs an ECG and troponin now”.
An average-rank merge goes with the three. The Medical LLM Council keeps the warning: safety warnings from any model are always reported, reassurance has to be unanimous to be given, and the more cautious urgency stands. The minority view is not lost; it leads.
How it works
- GPT-5.1
- Gemini 3 Pro
- Claude Sonnet 4.5
- Grok 4
- Independent OpinionsEvery model answers the case without seeing the others, in a fixed structure: a Final Answer, a ranked differential with probabilities, atomic claims tagged by type (diagnosis, recommendation, dose, reassurance, safety warning), and confidence. Can't-miss diagnoses are marked as such.
- Anonymous Peer ReviewEach model scores and ranks the others' Opinions, labelled only A, B, C, so no model knows whose answer it is reading or can favour its own. A reviewer can make a Danger Call: a statement that a specific Opinion could harm a patient, with the reason.
- Merge by ruleA swappable Aggregation Strategy decides what the final writer is allowed to use. Under Safety-First, an Opinion with a Danger Call cannot lead, every safety warning is kept, and reassurance and doses need agreement.
- Consensus StatementThe Chair model writes the final answer from the merged inputs only. It must name every can't-miss diagnosis, report minority views (Dissent), and say how each serious disagreement was resolved.
What changes for medicine
| karpathy/llm-council | Medical LLM Council | |
|---|---|---|
| Answers | Free text | Structured Opinions with differential, typed claims and safety warnings |
| Review | Anonymous ranking | Anonymous rubric scores, ranking and Danger Calls |
| Merge | Average rank, then the chairman writes freely | Rules that treat missed danger as worse than false alarm |
| Disagreement | Left to the chairman | Mapped, graded for severity, and arbitrated by fixed rules |
| Evidence it helps | None built in | A benchmark against the best single model and a plain majority vote |
Safety-First Strategy
Unions every safety warning, lets a Danger Call stop an Opinion from leading, and requires consensus for reassurance and dose claims. The Karpathy-style Baseline is kept next to it for comparison.
Danger Calls in Peer Review
Reviewers don't just rank. They can flag a specific answer as harmful and say why, and that flag carries through the merge.
Weighted vote for multiple choice
For exam-style questions, each model's vote is weighted by its measured accuracy, as log-odds: a model at chance gets weight 0, a strong one gets more say.
Oncology treatment plans
Cancer cases are merged step by step: facts first (each quoted from the case, with conflicting readings sent back to the source), then guideline path, intent, plan and regimen. Parts of different plans are never stitched together, and a split vote is shown as equipoise.
Side by side, with a disagreement map
Compare two or three Opinions in columns. Every point where they differ is listed, graded mild, moderate or substantial, and resolved by rule, or left open with what would settle it.
Benchmark
Run a strategy over a set of multiple-choice questions such as MedQA and compare it with the best single model and a plain majority vote, on accuracy, vote margin and cost, with oncology questions reported separately.
Bring your own key
Paste your own OpenRouter key into the app. It stays in your browser, is sent only with the requests that call models, and is never stored or logged on the server.
Grounded in research
The merge rules come from a review of how to combine answers from several LLMs for medical question answering, from voting and semantic entropy to asymmetric risk.
How disagreements are settled
Under Safety-First, one extra model call lists every point on which the Opinions take different positions. The model only finds the points; rules decide the rest. Severity can be raised by rule but never lowered: anything about dose, urgency or safety is always substantial. Each moderate or substantial point is then arbitrated:
- Safety warnings are all kept.
- A stated fact goes back to its source.
- The more cautious urgency stands.
- A dose needs a majority.
- Anything else needs a majority, with peer review as the tie-break.
An Opinion with a Danger Call never carries a point. If the Chair leaves a serious point out of its final answer, the omission is flagged.
Try it yourself
The council calls real models, paid for with your own OpenRouter key. Each case makes about ten model calls, so expect to pay cents per case, not dollars.
- Get an OpenRouter keySign up at openrouter.ai/keys, add a little credit, and create a key. It starts with
sk-or-. Setting a spending limit on the key is a good idea. - Open the app and paste the keyPaste it into the masked key field. It stays in this browser tab (or on this device, if you tick “remember”), is sent only with the requests that call models, and is never stored or logged on the server.
- Put a case to the councilWrite an open question or a multiple-choice one, or pick Oncology and fill in the case profile. Watch the Opinions, Peer Review and Consensus Statement stream in, then compare answers side by side. Use made-up cases only.
Where it stands
Version 1 is built: the three-stage council, both strategies, multiple-choice voting, oncology plan merging, the benchmark, and the side-by-side view with its disagreement map. It is a FastAPI backend and a React front end that call models through OpenRouter, with an automated test suite that runs on fake models. The next step is running the benchmark on real models to measure whether Safety-First actually beats a single model and a plain vote.
Want to try it or work on it?
Happy to talk about evaluation, merge rules, or clinical use cases.
Inspired by karpathy/llm-council; the code is an independent implementation. Research prototype, not for clinical use.