Scaling laws are the closest thing modern AI has to a law of physics, and, like most things that get called laws in a young field, they are really a well-fit empirical curve with a contested domain of validity. Understanding exactly what they claim — and, more importantly, what they do not — is the difference between reading frontier news as inevitability and reading it as a set of bets.
What a scaling law actually is
A neural scaling law is an empirical observation that a model's loss falls as a smooth power law in three quantities: the number of parameters, the amount of training data, and the compute spent training. Plotted on log-log axes, test loss traces a near-straight line across many orders of magnitude. The striking part is not that more helps — that is obvious — but that the relationship is regular enough to extrapolate. You can train a series of small models, fit the curve, and predict the loss of a model you have not yet paid to train. That predictability is what turned frontier training from a research gamble into a capital-allocation exercise: you can forecast, roughly, what a nine-figure training run will buy before you authorize it.
The crucial caveat lives in that word loss. Scaling laws are laws about next-token prediction error, not about capability, usefulness, or truth. The map from loss to anything a person cares about is real but loose, and most of the field's disagreements hide in that gap.
Kaplan to Chinchilla, precisely
OpenAI's 2020 scaling-laws paper, often called Kaplan after its lead author, fit these curves and drew a specific operational conclusion: given a fixed compute budget, you should spend most of it on making the model bigger and comparatively little on training it longer. Under Kaplan's recipe, models grew fast and were fed relatively modest amounts of data.
Two years later, DeepMind's Chinchilla paper re-ran the analysis more carefully — varying data and size together, and correcting a subtle issue in how the learning-rate schedule was handled — and reached a materially different answer. For compute-optimal training, parameters and training tokens should scale in roughly equal proportion: about 20 tokens per parameter. The practical verdict was blunt: GPT-3 and its contemporaries were several times too large for how little data they had seen. A 70-billion-parameter Chinchilla trained on more data beat the 175-billion and 280-billion models of the day. The result reset the field's defaults almost overnight.
One distinction from Chinchilla turned out to matter enormously: compute-optimal is not the same as deployment-optimal. Compute-optimal minimizes the cost of reaching a target loss during training. But a model you will serve to millions is dominated by inference cost, and a smaller model trained well past its Chinchilla point is cheaper to run at the same quality. That is why production models are now routinely 'overtrained' by Chinchilla's standard — open models fed many trillions of tokens — pushing token-to-parameter ratios far past 20, by some estimates toward the hundreds. Chinchilla told you where the compute-optimal point sits; industry now deliberately trains to the right of it, spending more on training to save vastly more on serving.
What scaling laws predict — and what they don't
Scaling laws predict loss. They do not directly predict capability, and the translation between the two is where most of the disagreement lives. A smooth fall in next-token loss can surface as an ability that appears to switch on abruptly at some scale — arithmetic, say, or in-context learning — which fed the 'emergent abilities' narrative. A widely cited rebuttal argued that much of that abruptness is an artifact of harsh, all-or-nothing metrics; measured on smoother scoring, the same abilities improve gradually. Both things can be true: the underlying capability grows smoothly while the useful, thresholded version of it appears to arrive suddenly.
They also say nothing about generalization out of distribution, about whether a model's knowledge is calibrated, about alignment, or about whether the loss you are minimizing is the loss you actually want minimized. And crucially, the standard curves are for pretraining. They were never a promise about the ceiling of intelligence — only a description of how fast one particular loss falls as you spend more on one particular training regime.
The second axis: inference-time compute
For most of the field's history, scaling meant training scaling. The reasoning-model wave opened a second, largely independent axis: spend more compute at inference time, letting the model generate and check long chains of reasoning before committing to an answer. OpenAI's o1 and its successors showed a clean scaling curve here too — accuracy on hard reasoning tasks climbs smoothly with the reasoning-token budget, across orders of magnitude, with no plateau yet visible in published evaluations.
This matters for the road-to-AGI argument in two ways. First, it is a fresh source of gains at exactly the moment pretraining returns were being questioned — a second engine, not a replacement for the first. Second, it changes the economics: capability becomes something you can dial up per query by paying for more thinking, rather than something fixed in the weights at training time. The tradeoff is that inference-time scaling raises the marginal cost of every hard answer, where pretraining scaling paid once and served cheaply forever. Analysts now expect inference to dominate total AI compute as this axis matures.
The wall debate
Whether pretraining scaling has hit a wall became the field's defining argument of 2025, and it is worth representing the credible positions fairly rather than picking a side.
The skeptical case points to specifics. OpenAI's Orion, shipped as GPT-4.5 in early 2025, reportedly improved over GPT-4 by far less than GPT-4 had improved over GPT-3, despite more compute — and it was expensive enough that the lab visibly reoriented toward reasoning models afterward. The data-wall argument underlies this: high-quality human text is finite, the largest training runs are approaching the readable internet, and synthetic data risks a 'hall of mirrors' in which a model inherits and amplifies its predecessor's blind spots. On this view the smooth curves are still real, but the cheap, high-quality data that fed them is running out.
The dismissive case, associated with Sam Altman's flat 'there is no wall,' notes that scaling laws have been declared dead before nearly every generation and have not been. Epoch AI's analysts, while carefully documenting diminishing returns per dollar, expect the next pretraining generation to resume the trend once enough compute is assembled, and argue that the apparent stall reflects a pause between hardware build-outs more than a ceiling. A third position — arguably the current working consensus — is that both training and inference scaling still function, but that the industry has rationally shifted marginal investment toward inference, data quality, algorithmic efficiency, and post-training, because that is simply where the cheapest gains currently are.
The honest summary: nobody has yet observed the pretraining curve bend to flat. What is genuinely contested is whether the inputs that curve depends on — affordable compute and high-quality data — can keep growing fast enough to matter, and whether continued loss improvements still translate into capability the market is willing to pay for.
Where things stand, August 2026
As of August 2026 the frontier is defined by both axes at once. The leading systems pair very large pretrained bases — increasingly sparse mixture-of-experts architectures — with heavy reinforcement learning on reasoning and large inference-time budgets, and they sit close enough to one another that the ranking reshuffles every few weeks. Pretraining has not stopped; labs are still assembling larger training clusters. But the visible quarter-over-quarter gains lately come more from reasoning, tool use, and post-training than from raw base-model scale. The debate has quietly moved from 'is scaling over?' to 'which kind of compute buys the next capability most cheaply?'
A judgment, labeled as such: the wall framing was always too binary to be useful. Scaling laws were never a guarantee of AGI, and their softening at the pretraining frontier is not proof it is unreachable. The more informative question is the one the field has already backed into — how to convert each additional unit of compute, whether spent in training or at inference, into capability that generalizes. On present evidence that conversion is getting harder per dollar, and, so far, it has not stopped.