AI Efficiency: Why Bigger Models Are No Longer Enough
Scale built modern artificial intelligence. But history suggests that mature technologies advance by learning to do more with less.
For most of the past decade, the strategy behind artificial intelligence was simple: add more computing power.
Researchers trained larger neural networks on larger datasets, using increasingly expensive clusters of specialized chips. The method worked. More compute produced better language models, more realistic images, stronger scientific tools and systems capable of solving increasingly difficult problems.
That race is not over. Frontier language-model training compute has continued to grow by about five times per year since 2020. At the same time, the estimated cost of the largest training runs has risen sharply, from a few million dollars earlier in the decade to hundreds of millions for some frontier projects.
But a second race is now becoming just as important.
It is not the race to use the most computation. It is the race to extract the most intelligence from each chip, watt and dollar.
The Efficient Compute Frontier
For any fixed amount of computing power, there is a practical limit to the performance that current algorithms can achieve. Researchers can improve a model by adding processors, data or training time, but the gains usually become progressively smaller.
This is not the same as saying that artificial intelligence has reached a permanent ceiling. The frontier itself can move. A new architecture, training method or software optimization may allow the same hardware to produce a much better result.
The distinction is important.
Adding more compute means moving along the existing frontier. Inventing a better algorithm means shifting the frontier outward.
Modern scaling laws indicate that model performance generally improves according to power-law relationships. Scaling therefore continues to work, but doubling the computational budget does not normally double intelligence or accuracy. Larger improvements require increasingly large investments.
This is the technological version of diminishing marginal returns: more resources still help, but each additional unit tends to contribute less than the previous one.
The Cost Is Becoming Physical
The problem is no longer limited to the price of GPUs.
AI systems need electricity, cooling equipment, high-bandwidth memory, networking infrastructure and large amounts of land and capital. Thousands of processors must exchange data and remain synchronized during training. As clusters grow, communication delays and memory bottlenecks can prevent additional chips from producing proportional gains.
The International Energy Agency estimates that global data-centre electricity consumption could rise to about 945 terawatt-hours by 2030, roughly twice the 2024 level. Data-centre demand increased by approximately 17% in 2025, while electricity use by AI-focused data centres grew by around 50%.
This does not mean that AI will run out of electricity tomorrow. It means that efficiency is becoming an economic and infrastructure requirement rather than merely an academic preference.
The central question is changing from:
How large can we make the model?
to:
How much capability can we obtain from the resources already available?
Chinchilla: A Smaller Model Trained More Intelligently
One of the clearest examples came from Google DeepMind’s Chinchilla research.
Before Chinchilla, many developers assumed that the best use of a fixed training budget was to build the largest possible model. DeepMind researchers argued that several major language models had too many parameters and had been trained on too little data.
They trained Chinchilla with 70 billion parameters, one-quarter the size of the 280-billion-parameter Gopher model, but exposed it to four times more training data. Chinchilla upon its release in March 2022, used approximately the same total training compute as Gopher yet outperformed it across a broad range of evaluations. It also required less computation during deployment because the final model was smaller.
The lesson was not that small models are always superior. It was that the balance between model size, data and computation matters as much as the total budget.
A poorly allocated billion dollars can produce less intelligence than a well-allocated fraction of that amount.
Mixture of Experts: Not Every Neuron Must Work Every Time
Another strategy is to avoid activating the entire model for every word it processes.
Mixture-of-Experts systems divide a neural network into specialized sections. A routing mechanism decides which experts should handle each token. The model may contain a large total number of parameters while using only a fraction of them during any single calculation.
Mistral AI’s Mixtral 8x7B (December 11, 2023), for example, contains about 46.7 billion parameters, but activates only around 12.9 billion for each token. In the evaluations reported by its developers, it matched or exceeded the much denser Llama 2 70B (July 18, 202) while requiring the computational work of a substantially smaller active model.
DeepSeek-V3 (December 2024) applied a similar principle on a larger scale. The model contains 671 billion total parameters, but activates about 37 billion per token. Its developers reported 2.788 million Nvidia H800 GPU-hours for the official training process. Using an assumed rental price of two dollars per GPU-hour, they estimated that phase at approximately $5.6 million.
That figure requires an important qualification. It covers the reported final training process, not the full cost of developing the company, collecting and processing data, purchasing infrastructure, conducting earlier experiments or testing alternative architectures. The technical report explicitly excludes prior research and ablation experiments.
The value of the example is therefore not the claim that a frontier model can be created from nothing for $5.6 million. It is the demonstration that architectural and systems engineering can substantially reduce the computation required for the final training run.
FlashAttention: Sometimes the Mathematics Is Not the Problem
Efficiency does not always require changing what a neural network calculates. Sometimes the breakthrough comes from changing how the calculation is executed.
Transformer models repeatedly move information between different levels of GPU memory. Those transfers can become a major bottleneck, particularly when models process long documents.
FlashAttention (May 27, 2022) reorganized the standard attention calculation to reduce unnecessary movement between slower high-bandwidth memory and faster on-chip memory. It computes the same mathematical result rather than replacing attention with a lower-quality approximation.
The original research reported a threefold training speed improvement for GPT-2 (February 14, 2019) in one test. Later work described wall-clock gains of roughly two to four times and memory savings of up to ten to twenty times in some configurations.
No additional intelligence was obtained by adding more GPUs. The improvement came from understanding how the hardware actually moves data.
That kind of optimization may sound less dramatic than announcing a trillion-parameter model, but multiplied across millions of users and billions of queries, it can have an enormous economic impact.
The Price of Intelligence Is Already Falling
The strongest evidence for the efficiency revolution may be found in inference—the process of running a trained model to answer a question.
According to Stanford’s 2025 AI Index, the cost of using a model that achieved roughly GPT-3.5-level performance on the MMLU benchmark fell from about $20 per million tokens in November 2022 to $0.07 in October 2024. That was a reduction of more than 280 times in less than two years.
The models did not become cheaper because the world suddenly stopped valuing AI. Prices fell because of better chips, smaller capable models, quantization, improved serving software, competition and more efficient architectures.
More recent research estimates that the price of obtaining a fixed level of performance has continued to decline by approximately five to ten times per year across several categories, while purely algorithmic efficiency has improved at roughly three times per year.
This may ultimately matter more to the economy than another modest increase on a benchmark. A model that costs one-hundredth as much can be deployed in schools, hospitals, small businesses, personal devices and public services that could never afford the original system.
James Watt and the Reinvention of Steam
The same transition has appeared repeatedly in technological history.
Thomas Newcomen’s early eighteenth-century steam engine was useful but extremely inefficient. It repeatedly heated and cooled the same cylinder, wasting large amounts of fuel. The design remained viable mainly in coal mines, where low-grade coal was readily available and the machine could be used to pump out water.
James Watt’s crucial innovation was not simply to build a larger engine. He introduced a separate condenser, allowing the main cylinder to remain hot while the steam was condensed elsewhere.
That reduction in wasted heat transformed the economics of steam power. A machine previously practical mainly near cheap coal could be used across a much wider range of industries and locations.
The history is instructive. Newcomen demonstrated that steam power worked. Watt made it efficient enough to spread.
Technological revolutions often begin with raw capability, but they become economically transformative through efficiency.
Aviation: Better Aircraft, Not Merely Bigger Engines
Commercial aviation followed a similar path.
Early improvements came partly from greater engine power, but modern aircraft did not become successful by increasing thrust indefinitely. Engineers improved high-bypass turbofan engines, aerodynamics, wing design, navigation, structural weight and composite materials.
The International Civil Aviation Organization estimates that modern aircraft are approximately 80% more fuel-efficient per passenger-kilometre than aircraft from the 1960s. ICAO attributes those gains to improvements in propulsion, aerodynamics, materials and structural design.
A modern airliner can travel farther while carrying more passengers and using much less fuel for each passenger transported.
The mature aviation industry is therefore not defined by which manufacturer can attach the largest possible engine to an aircraft. It is defined by how effectively the entire system converts fuel into safe transportation.
AI is moving toward the same systems-level thinking.
From Clock Speed to Multicore Computing
Personal computing offers another example.
For years, processor manufacturers improved performance largely by shrinking transistors and raising clock speeds. Faster clocks allowed processors to execute more instructions each second.
Eventually, heat and power consumption became serious constraints. Increasing frequency further produced excessive energy use and made processors difficult to cool. The industry shifted toward multicore processors, parallel execution, specialized accelerators, improved memory hierarchies and architecture designed for specific workloads.
Computer-architecture researchers describe the weakening of Moore’s Law and Dennard scaling as a transition toward diminishing returns from the traditional approach. Continued progress increasingly depends on new architectures rather than simply raising the operating frequency of a general-purpose processor.
Modern AI accelerators are themselves products of that transition. GPUs became important not because each individual core was universally faster, but because many calculations could be performed in parallel with greater efficiency.
From Incandescent Bulbs to LEDs
Lighting provides perhaps the simplest example.
An incandescent bulb produces light by heating a filament until it glows. Much of the input energy is therefore lost as heat.
An LED produces light through a fundamentally different physical process. According to the US Department of Energy, modern LEDs can use up to 90% less electricity and last up to 25 times longer than traditional incandescent bulbs.
The LED did not win by becoming the hottest or most powerful incandescent lamp ever manufactured. It won by changing the mechanism through which electricity became light.
That is what a genuine shift in the efficiency frontier looks like.
Efficiency Does Not Mean the End of Scale
There is an important paradox.
More efficient AI may not reduce total computing demand. It may increase it.
When the cost of generating a response falls, companies create more applications, users make more requests and AI is incorporated into more products. Cheaper intelligence can create enough new demand to offset—or surpass—the savings from each individual operation.
This is why algorithmic efficiency and enormous data centres can grow at the same time. Frontier developers will probably continue building larger systems, while efficient models spread into phones, laptops, vehicles, robots and industrial equipment.
The future is unlikely to be “scale or efficiency.”
It will be scale multiplied by efficiency.
A company that improves its algorithms can use the savings to reduce prices—or to train an even more capable model with the same infrastructure.
The Next Breakthrough May Look Smaller
The next major AI advance may not be announced as the largest neural network ever built.
It may be a model that matches the performance of a much larger competitor while activating one-tenth of the parameters. It may process long documents using a fraction of the memory. It may run locally instead of sending every request to a data centre. It may learn from better-selected information rather than indiscriminately consuming more data.
Such improvements can appear less spectacular than another record-breaking training cluster. Yet they may determine which AI systems become economically sustainable and widely accessible.
The first phase of modern AI proved that scale works.
The next phase will determine how intelligently that scale is used.
The history of technology suggests that mature industries are rarely defined by who builds the biggest machines. Steam power spread when engines wasted less coal. Aviation advanced by carrying passengers farther with less fuel. Computing moved beyond clock speed through new architectures. Lighting was transformed not by a larger filament, but by the LED.
Artificial intelligence is now approaching the same turning point.
The winners may still own vast computing infrastructure. But hardware alone will not be enough. The decisive advantage will belong to those who can transform computation into capability with the least waste.
In the next era of AI, intelligence will not be measured only by what a model can do.
It will also be measured by how efficiently it does it.
References
Epoch AI. 2026. “Trends in Artificial Intelligence.” Updated February 5, 2026.
International Energy Agency. 2025. Energy and AI. Paris: IEA.
Hoffmann, Jordan, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, et al. 2022. “Training Compute-Optimal Large Language Models.” Advances in Neural Information Processing Systems 35: 30016–30030.
Mistral AI Team. 2023. “Mixtral of Experts.” December 11, 2023.
DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, et al. 2025. “DeepSeek-V3 Technical Report.” arXiv:2412.19437 [cs.CL].
Dao, Tri, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. 2022. “FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness.” Advances in Neural Information Processing Systems 35: 16344–16359.
Stanford Institute for Human-Centered Artificial Intelligence. 2025. The 2025 AI Index Report. Stanford, CA: Stanford University.
Gundlach, Hans, Jayson Lynch, Matthias Mertens, and Neil Thompson. 2025. “The Price of Progress: Price Performance and the Future of AI.” MIT FutureTech Working Paper.
National Museums Scotland. n.d. “The Newcomen Atmospheric Engine.” Accessed July 22, 2026.
International Civil Aviation Organization. 2010. ICAO Environmental Report 2010: Aviation and Climate Change—Chapter 2, Aircraft Technology Improvements. Montreal: ICAO.
Al Amin, Md. Tanvir. 2010. “Multi-core: Adding a New Dimension to Computing.” arXiv:1011.3382 [cs.OH].
U.S. Department of Energy. 2021. Consumer Guide to Energy-Efficient Lighting. DOE/EE-2465. Washington, D.C.: U.S. Department of Energy.
Editorial transparency note: This article, as with all articles published on this site, was conceived, directed, written, and reviewed by Prof. Maurício Veloso Brant Pinheiro. Artificial intelligence was used as an assistant for editorial refinement, formatting, image generation, SEO metadata, and publication workflow.
Copyright 2026 AI-Talks.org
