I have been thinking about a question that nobody in enterprise software seems to want to sit with: why can the most advanced AI models in the world solve Olympiad-level mathematics but fail to reliably extract a total from an invoice?
This is not an academic exercise for me. I have been building automation software for twenty years. My company has processed billions of documents for some of the largest enterprises in the world. Yes, I have a stake in this answer. But twenty years of watching models work on real enterprise data, not benchmarks, gives you a different view than turning a model in a lab. And when those real-world models cannot get the simple stuff right, I notice.
The conventional answer to my question goes something like this: math is a reasoning problem and AI is good at reasoning now. Invoices are a perception problem—messy layouts, bad scans—and we just need better models. Give it another generation.
I think this is wrong.
The math
Let me start with math, because I think people misunderstand what is actually happening when an LLM solves an olympiad problem. It looks like reasoning. But competitive mathematics has maybe a few hundred proof techniques that appear over and over. A “novel” problem is really a novel combination of familiar building blocks. The model has trained on tens of thousands of proofs. It has learned to remix those blocks very well. Call it composable pattern matching.
Chess is the opposite. Every serious middlegame position is genuinely new in the way that matters. You can know every pattern, every tactical idea, and still be completely wrong about whether a particular sacrifice works. The only way to know is to calculate the concrete lines. Chess engines solved this—by building a system around the neural network, not by making the neural network bigger. That distinction matters more than people realize.
Where the risk lives
Most clerical work looks like the math problem, not the chess problem. Claims processing, compliance checks, loan document review. You are applying known rules to new instances. An LLM can handle 85 to 95% of the volume—and that is a real win. But the remaining 5 to 15% is where the risk lives. These are the cases where the pattern does not match. And the dangerous thing is that the model does not know it is stuck. It gives you a confident answer anyway.
We have spent years testing AI models on document extraction. Not edge cases—invoices. The simplest version of the task: read a value, put it in the right field. No reasoning. No judgment. Just read the number. Even the best models cannot do it at 100% accuracy. A less experienced human will.
I remember when we first saw this clearly. I assumed it was our pipeline. It was not. We tested multiple models. Same result. And it stuck with me, because you do not need to reach the hard part of the process, the judgment calls, the exceptions, to find the failure. The failure is in the reading.
The human knows what an invoice “is.” They know a total should be bigger than the line items. They know that “Montant TTC” means the same thing as “Total incl. VAT.” The model is matching patterns from training data. When the layout shifts, the match breaks. Not because the task is hard. Because the model was never actually reading the invoice.
A more powerful model that still does not understand what an invoice is becomes a more confident model, not a more reliable one. And here is what people miss: every generation of models makes the problem look more solved, which means you trust it more, which means you route more volume through it, which means the damage from the remaining failures gets bigger, not smaller. A wrong number on an invoice that feeds into a payment that feeds into a regulatory filing is a different kind of 2% error than a wrong number on a dashboard.
A specific argument
I am not making an argument against AI. I am making an argument against a specific idea: that a powerful enough model, deployed on its own, can be trusted with enterprise operations.
The model is not the thing that matters. The system around it is—the part that knows when the model cannot be trusted. Validation rules. Cross-field checks. Confidence scoring. Escalation to a human when something does not look right. When you are pushing 90% of your volume through a system that can fail without telling you it failed, governance is not a nice-to-have. It is the product.
Every enterprise AI vendor right now is selling you the composable pattern matching. That part is real. But the hard problem is knowing when pattern matching is not enough—knowing when you have hit a chess position, not a math problem, and you need to stop interpolating and start checking.
The companies that figure that out will build something that lasts. The ones that pretend the problem does not exist will spend the next ten years explaining to customers why the AI got the invoice wrong.
