On the slow death of scaling

Summary

On the Slow Death of Scaling 30 Pages Posted: 12 Dec 2025 Last revised: 6 Jan 2026 Date Written: December 06, 2025 Abstract For the last decade, it has been hard to stray off the beaten path of accepted wisdom for what drives innovation. We have been held hostage to a painfully simple formula: scale model size and training data. A pervasive belief in scaling has resulted in a massive windfall in capital for industry labs and fundamentally reshaped the culture of conducting science in our field. Academia has been marginalized from meaningfully participating in AI progress and industry labs have stopped publishing. Yet, this essay will posit that the relationship between training compute and performance is highly uncertain and rapidly changing. Relying on scaling alone misses a critical shift that is underway, and ignores more interesting levers of progress. All this suggests that key disruptions lie ahead. Keywords: scaling, deep neural networks, efficiency, computer science, scientific progress, compute Suggested Citation: Suggested Citation