Inference is nearly free. Judgment, trust, and accountability are not.
Two clocks are running in every workplace right now. One belongs to the model. The other belongs to the institution.
The model's clock got more than 280 times cheaper in eighteen months. The institution's clock still runs on meetings, approvals, and the time it takes for people to trust a result enough to act on it.
We did not make decisions cheaper. We made the draft before the decision almost free.
The most important AI number of the last two years is not a benchmark score. It is a price.
For output quality roughly equivalent to a 2022-era frontier model, the cost of inference fell from $20.00 per million tokens in November 2022 to $0.07 by October 2024. That is a decline of more than 280-fold in about eighteen months.
When the marginal cost of a plausible first draft approaches zero, the economics of work change. The scarce resource is no longer thinking of something to say. It is deciding what is worth saying, what is true, and what the organization is willing to stand behind.
Two caveats matter. The 280-fold figure is for a fixed quality level; expectations rise as models improve. And a falling unit price does not guarantee a falling total bill, because volume often expands faster than price falls. Cheap tokens can still mean an expensive habit.
In 1967, Gene Amdahl made an observation about parallel computers that now reads like a management insight. If only part of a system can be sped up, the overall improvement is capped by the portion that cannot.
Organizations have a serial remainder. A model can draft in seconds, but someone still has to check the facts, decide the trade-offs, accept the accountability, and coordinate with the people affected. Making the parallel part almost infinitely fast does not make the serial part disappear. It makes it more visible.
This is why AI can feel simultaneously miraculous at the task and disappointing at the company level. The model is not the bottleneck. The bottleneck is the decision the model is waiting for.
The bottleneck moved. It used to be knowledge: could you get the information, the analysis, the words? Now it is permission: who is allowed to act, how much evidence is enough, and who signs their name to the result.
The task-level evidence is strong. In customer support, agents with a generative assistant resolved about 15% more issues per hour. In professional writing, people completed assignments about 40% faster, with higher measured quality.
Both studies are also reminders to be careful. They measure specific tasks under controlled conditions. They do not say that every job, team, or firm becomes 15% or 40% more productive. They say the technology already works where the work is bounded and the output can be checked.
The firm-level picture is more modest. By early May 2026, about 19.8% of U.S. employer firms reported using AI in at least one business function. Information led at 39.7%, finance and insurance at 33.9%, retail near 14%.
Both things are true. AI already works at the level of the task. It still has trouble becoming the operating system of an institution.
A draft is cheap because it is reversible and nobody has to defend it. A decision is expensive because it allocates commitment.
Four things make it so:
AI reduces the cost of options. It does not reduce the cost of choosing.
A model can propose a new price list in a minute. Changing the price list still touches customers, contracts, finance, and support. The proposal is cheap; the commitment is not.
A draft is a suggestion. A decision is a promise with a name attached.
For most of the last century, the unit of white-collar work was the document: the memo, the model, the deck. AI collapses the cost of producing one.
The scarce unit now is the decision: a commitment to act, with an owner and a consequence. Organizations that treat decisions as designed artifacts — with owners, evidence standards, and reversal costs — will compound AI's gains. Organizations that treat decisions as an afterthought will simply produce more documents faster.
If the serial remainder is the constraint, the work is not prompt engineering. It is decision engineering.
Map an end-to-end process and mark where work waits for a human, a meeting, or a policy. That queue, not the model, sets your ceiling.
Specify who decides, what evidence is required, what is out of bounds, and what happens on appeal. A clear decision path is worth more than a better prompt.
Model latency is an engineering metric. Institutional latency is the business metric. Measure the distance from question to committed answer.
Audit trails, source citations, and review tiers are not bureaucracy. They are the mechanism by which fast output becomes acceptable output.
Smaller approval batches, clearer ownership, and shorter review loops often beat another model upgrade.
Tokens got cheap. Decisions did not.
The model writes the first draft. The institution still writes the last one.
The organizations that win the next decade will not be the ones with the best models. They will be the ones that make deciding cheap too.
Thoughts and essays, published with Yokush. See more posts
Siddharth13 min
Siddharth20 min1
Siddharth
Comments 0