Automating Doubt: The Danger of Outsourcing AI Skepticism

The cheapest thing to automate is your own skepticism. It’s also the only part of the job you can’t afford to give away.

You paste work into the AI, and a clean, organized answer comes back. The little voice that once asked if it’s right stops showing up. You didn’t decide to trust the machine. You simply stopped checking. Because checking is friction, and the output already looks good enough.

Call it what it is: automating doubt. And doubt is the one job you can’t hand to this tool. After all, this tool is the thing you’re supposed to be doubting, despite all the marketing that would have you assume otherwise.

This wouldn’t matter much if people kept AI in the brainstorming corner. But they don’t: they wire it into decisions that must be right the first time. What to charge, what a contract says, what goes out under their name, and code that moves money. Then, they treat the answer like it came off a calculator. It didn’t. The thing under the chat box is a probability engine that becomes less reliable the longer you let it run, sold to you at a price where someone is deliberately losing money. Take the doubting out of that stack, and you haven’t saved labor: you’ve pulled the only inspector off the line.

It won’t even give you the same answer twice.

Set the randomness dial to zero and ask it the same question twice. You can still get two different answers.

Your question isn’t answered by itself. To process requests efficiently, the server groups your request with others and computes them together. The number of requests in this batch can change from moment to moment, depending on overall demand. Some operations in the model depend on the batch size, and even a slight change can alter the model’s output, changing a single word and affecting all subsequent words. This means your answer depends partly on how many other people are using the service at that moment. This batch effect causes variation in outputs, even if nothing else changes. For example, if you repeat a prompt a thousand times, you may get a thousand different completions, none directly caused by your input.

Fortunately, this unpredictability is fixable. The same researchers rebuilt the math so that the batch size no longer matters and got the same output every time. Almost nobody serves it that way because the deterministic version runs slower, and slower costs money. So the thing you’re treating like a calculator is, where it counts, a surface that shifts under your feet depending on the weather in a data center you’ll never see. It’s fine for a brainstorm. But it’s not fine once you’ve decided its answer, is the answer.

Agents don’t solve this problem. They multiply it.

The pitch is to solve this problem with agents. Agentic workflows tell us that we need to stop asking questions and start handing over goals. Book the trip. Reconcile the invoices. Find the bug, fix it, ship it. String enough steps together, and you get an outcome instead of a paragraph.

But, reliability doesn’t hold the more "people" you get involved (I like to think of agents as an army of interns). It multiplies. An agent that’s right 85% of the time per step is right about 27% of the time after eight steps. That’s 0.85 to the eighth power. Five agents, each 95% accurate, together finish about 77% of tasks correctly. Everyone hits this wall and is surprised because each part passed its own test.

Carnegie Mellon built a test-handling model of real multi-step office work; the best ones finished about a quarter of tasks alone, and failure rates rose to 70–90% as jobs grew more complex. Another test found that a model with a 61% single-try success rate dropped to 25% on repeated tasks. Models improve over time, nearly doubling sustainable length every 7 months. But “longer” doesn’t mean “more dependable.” The gap between a demo and a production run is where the doubt you fired used to stand. In July 2025, Replit’s coding agent wiped a live database after being told not to. It wasn’t a fluke. That’s what an unsupervised probability engine does in a long enough chain. Eventually, it does what you forbade.

The deterministic part is the code, not the model.

It's not all doom and gloom, I suppose. You can get deterministic outcomes from an agent. But, determinism doesn’t come from the model. It comes from the ordinary software that the model decides to run. A script that moves a file or charges a card does the same thing every time you call it; that part is solid. The model’s job is choosing which script to call and what to feed it. Sometimes, that choice is the coin flip. An agent is a coin flip put in charge of a row of switches, and some of those switches are wired to things you can’t take back.

Making this arrangement safe requires significant engineering effort. It means checking what the model produces before anything acts on it, boxing in the tools so bad calls can’t flip dangerous switches, retrying, verifying, and placing a human in front of anything irreversible. One analysis found that 82% of agent failures are baked in before the model is ever prompted - in the planning, context, and guardrails, not the model itself. Today, the model is the easy part. Now the hard part is scaffolding that makes a probability engine behave deterministically, and someone still has to build that software. The problem is, people are still trying to make the model do things it's not meant to.

Most people buying into agents can’t build them. So they do the natural thing and ask the AI to build the scaffolding too. Hell, Anthropic has tools to help you do this. They vibe-code their own inspection layer out of the same material they’re supposed to be inspecting. But the doubt has to live somewhere in a working system: in a test, a gate, a person who signs off. Hold it yourself, or build something to hold it. Do neither, and the system still runs. It just runs with nobody home.

The price is bait

And agents are expensive. Do not be fooled. The price is bait. The cost of AI looks like it’s falling, and in one narrow, technically true sense, it is. The price of a single token, the unit they bill you on, has dropped by roughly 80 to 90% in two years, and Gartner expects another 90%+ cut by 2030. Watch only that number, and AI gets cheaper every quarter.

But the bill is going up. Gartner’s own analysts put it bluntly: don’t mistake cheap tokens for cheap intelligence. Per token, the price drops. But the number of tokens you burn is exploding because the new way of using these tools — agents, retrieval, monitors that never sleep — fires off 10 or 20 model calls for a task that used to take 1. The per-token price is the magician’s other hand.

Not only that, but the per-token price isn’t even real. It’s subsidized. The race to the bottom is paid for by venture capital and by cloud giants eating losses to buy market share. By one widely repeated estimate, OpenAI spent about $1.35 for every dollar it took in during 2025. The number on the rate card isn’t what the compute costs. It’s what the labs have chosen to charge you while they’re still fighting over you. The real landed cost, what a token runs once somebody has to turn a profit on it, sits above the sticker, and the gap is the part the marketing is built to hide.

The Information reported that Uber blew through its entire 2026 AI budget in four months. Nothing broke. Their engineers just took to Claude Code. Adoption ranged from a third to most of them, and the cost was between $500 and $2,000 per engineer per month. They didn’t stop wanting the tool. They ran out of money to pay for it. The subsidy isn’t over. It’s ending, which is worse, because the ending happens on someone else’s timeline.

The meter is about to change owners.

Worth keeping in mind, the companies setting these prices are going public. Anthropic filed confidentially for an IPO just a few weeks ago at a valuation of nearly a trillion dollars. OpenAI is headed in the same direction, losing an estimated $14 billion this year and with no profit forecast until around 2030. Inference, the cost of actually answering your prompts, runs around 60 to 65% of revenue at these outfits. That is precisely the line a public-company finance chief is hired to shrink.

A private company can burn money for years to buy your loyalty. A public one answers to shareholders every ninety days, and those shareholders want margin. Once these labs are trading on an exchange, the pressure inverts from “grow at any cost” to “show us the money,” and there are only two levers to pull: charge more or give less for the same money. Competition might hold the line for a while. But the structural pull is now upward, and you’d be building a load-bearing dependency on a price you don’t set, can’t predict, and that’s controlled by a company whose legal obligation just shifted from courting you to paying its owners. That’s the oldest play in the platform book: get them hooked while it’s cheap, then turn the screw once leaving is expensive.

The literacy gap is the multiplier.

None of this would do much damage if people saw it coming. Unfortunately, most don’t. In 2026, about 59% of business leaders reported an AI skills gap in their own organizations, and the gap isn’t a shortage of engineers. It’s a shortage of judgment. Researchers call this failure automation bias, the well-documented human reflex to trust the machine over one's own judgment. The International AI Safety Report named it a growing problem. Gartner now expects half of organizations to start running “AI-free” skills tests because they’re watching people’s judgment go soft from disuse. An MIT study that circulated all year estimated that about 95% of corporate AI pilots deliver nothing. The tools work. The thinking around them doesn’t.

The doubt has to live somewhere.

People are adopting AI to offload cognitive work, and the first thing they set aside is doubt. They’re doing it on top of a system that won’t repeat itself and decays the longer it runs. They’re doing it without the engineering required to make that system safe. They’re doing it at a price that’s subsidized today and primed to climb tomorrow.

Putting the tools down isn’t the move. Keeping the job you are giving away is. Treat every output as a draft, never a finished deliverable. Measure what a working answer costs you, not what a token costs. Keep a human in front of anything you can’t undo. Build or buy the real scaffolding before you let a model near a path where being wrong costs money. And the price is increasing, because the people who set the price are about to start working for someone other than you. Plan for it.

The scarce skill here was never prompt engineering. Prompting is already a commodity, the way typing is. The scarce skill today is doubt, used well, and it’s the one thing the machine can’t do.