Accuracy figures in vendor demos are measured on clean documentation and favorable specialties. The questions that decide whether a tool works for you are what it does with low-confidence cases, whether it undercodes to stay safe, and who carries the compliance exposure for what it submits.
The honest answer is that it depends on your specialty, your documentation and what you do with the cases the model is unsure about. That is less satisfying than a percentage and considerably more useful.
What the quoted accuracy actually measures
Usually agreement with a reference coder on a curated sample, in a specialty where the model performs well, on documentation that was complete. Your encounters will not all look like that.
Ask what the sample was, which specialties it covered, and what proportion of real-world cases the system declines to code at all.
The direction of error matters
Overcoding creates compliance exposure. Undercoding creates silent revenue loss with no denial to alert anyone. Vendors optimize away from the first, which means the second is the more likely failure in practice.
A tool with excellent accuracy that quietly selects lower levels is expensive in a way no dashboard reports.
Confidence routing is the real feature
The useful question is not what the model does when it is confident. It is what happens when it is not — does it route to a human, does it guess, and can you set that threshold.
A system that codes everything is worse than one that codes seventy percent and hands you the rest cleanly.
Responsibility does not move
The practice submits the claim and signs for it. Automation changes who produced the code and nothing about who answers for it.
That makes review sampling a permanent requirement rather than a pilot-phase activity.
How to run a pilot honestly
Blind. Have the tool and your coders work the same encounters without seeing each other’s output, then compare, splitting disagreements by direction and by dollar impact.
Measure the review rate too. A tool that requires human review on half its output has a different economic case from one that requires it on five percent.
Documentation quality is the ceiling
These systems read notes. Where documentation is thin, the model has the same problem your coders do and cannot resolve it by inference.
Practices with documentation problems will not automate their way out of them; they will automate the consequences faster.
Where it genuinely helps today
High-volume, low-variation encounters. Charge capture prompts. Flagging missing elements before a note is signed. Those are real gains and they are less dramatic than full autonomy.
The narrower use is usually the one that pays.
Common questions
- Is AI medical coding accurate?
- It can be, on straightforward encounters with good documentation. Performance varies enormously by specialty and by the quality of the notes it reads.
- Who is responsible if the AI codes wrong?
- The practice. You submit the claim, so the position is yours regardless of what produced the code. That does not change with automation.
- Does AI coding undercode?
- Frequently, because a conservative model is safer for the vendor. Systematic undercoding is a real revenue loss and it produces no denials to alert you.
- What should I measure in a pilot?
- Agreement with your own coders on a blind sample, split by overcoding and undercoding, plus how many cases it routes for human review.
- Does it work for every specialty?
- No. Straightforward high-volume encounters automate well; surgical, oncology and behavioral health coding depend on nuance that models handle less reliably.
Denials Piling Up?
We handle the revenue cycle end to end — coding by certified coders, claim submission, denial management and appeals, and A/R follow-up, with six reported numbers every month.