Caliper Lab
AI capability · Medical coding

Can AI code a hospital chart?

Not without a coder. The best model gets the main diagnosis right on most charts, but misses more than half of the other diagnoses.

LiveEdition 1, 19 Oct 2026Sectors: Healthcare
84%

of charts where the best model chose the same main diagnosis as a certified coder.

38 to 44%

of the coder's other diagnoses that each model found.

1 of 55

charts that the best model coded with no corrections needed.

No corrections neededMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis right, other codes needed fixingMain diagnosis wrongMain diagnosis wrongMain diagnosis wrongMain diagnosis wrongMain diagnosis wrongMain diagnosis wrongMain diagnosis wrongMain diagnosis wrongMain diagnosis wrong
Each dot is one of the 55 test charts, coded by the best model
  • 1 No corrections needed
  • 45 Main diagnosis right, other codes needed fixing
  • 9 Main diagnosis wrong

Why it matters. Codes decide what a hospital is paid. Revenue cycle firms are selling AI coding on the promise of fewer coders, so the gap between the pitch and the result is a diligence question.

Claude Opus 5.5 scores highest, at 49 out of 100Coding score: how closely each model's codes match the coder's, 0 to 100
0255075100Claude Opus 5.5Claude Opus 5.5: 4949Kimi K3Kimi K3: 4444GPT-6.1-solGPT-6.1-sol: 4343

Source: Caliper Lab medical coding benchmark, Edition 1, 19 Oct 2026.

Show the data
ItemValue
Claude Opus 5.549
Kimi K344
GPT-6.1-sol43

What we did

Three frontier AI models coded 55 synthetic inpatient charts in infectious disease, endocrinology and gastroenterology. A certified coder's codes were the reference: 55 main diagnoses, 269 other diagnoses and 18 procedures.

When all three models agree on a code, the coder agrees 83% of the timeShare of proposed codes the coder also assigned, by how many models proposed them
0%25%50%75%100%All three modelsAll three models: 83%83%Two of threeTwo of three: 55%55%One model onlyOne model only: 31%31%

Source: Caliper Lab medical coding benchmark, Edition 1, 19 Oct 2026.

Show the data
GroupValue
All three models83%
Two of three55%
One model only31%

A practical review rule follows: accept codes all three models agree on, and send the rest to a coder. Even then, about 1 in 6 agreed codes would still be wrong.

Full results

ModelCoding scoreMain diagnosis rightOther diagnoses foundCharts with no correctionsCost per chartTime per chart
Claude Opus 5.54984%43.9%1 of 55$0.3565 s
Kimi K34478%37.5%1 of 55$0.12100 s
GPT-6.1-sol4369%38.3%0 of 55$0.25350 s

How we measured it

  • Each model received the full chart and the same instructions, with no examples and no retrieval.
  • Coding score balances codes found against codes wrongly added (an F1 score), across all 342 codes the coder assigned, out of 100.
  • Cost and time are per chart, at list API prices on the run date.

Limits

  • One coder set the reference; codes were not adjudicated by a second coder.
  • Charts are synthetic, written to match real inpatient cases.
  • Grouping to a payment category (DRG) is not yet scored; Edition 2 adds it.
Cite this study. Caliper Lab, "Medical coding benchmark", The Bench, 19 Oct 2026. https://thecaliperlab.com/study-medical-coding.html

See everything we cover in healthcare.

AI in healthcare