We have all freaked out about AI in the last few weeks. It’s difficult to say what triggered it. There is the post by ex-Anthropic researcher Jacob Coxon, OpenAI announcing a proposed solution to the Navier–Stokes problem, one of the Millennium Prize Problems in math, and the Hugging Face incident, which I already wrote about here. Scary stuff. But the part I want to write about today is, call me crazy, also fun.
I think part of why people very close to AI are freaking out is where they think we are on the progress curve. Today, progress feels something like this:
And then you zoom out:
And all hype aside, I want to make the case in a simple way for why I think we still have a long way to go up that curve. Spoiler: the standard ways to make AI better continue to bring results, but we have several others, with more being discovered. And more importantly, these improvements can make each other more useful.
More ways to get smarter
Scaling laws let us estimate how a model’s performance changes as we increase one of its inputs, like the compute used to train it. (gravity is a law... calling scaling a law gives me an ick, but let’s go with the industry)
Giving o1 more compute to reason improved its score on this math test. Adapted from OpenAI’s research. Points are approximate readings from the original chart.
The best-known example is giving a model more data or compute during training, the “OG” scaling law, and I wrote last week about the reasoning one. But there are a few others, and the interaction between them really matters. Two improvement factors are not additive to the total performance; they are multiplicative. In non-math terms, improving one can increase what we get out of another.
Hardware is an easy way to see this. Say new GPUs give you twice the compute in the same amount of time, and a better algorithm lets you run each experiment with half as much compute. Together, you can run four times as many experiments in that time. That doesn’t mean four times the intelligence, but it shows how two improvements can multiply.
I am a visual person, so I think this “simulation” (thanks, Astra) will make the point better than I can. The numbers are made up. The point is how they combine:
Try the interactive version: add one improvement at a time and see what happens.
What are the specific scaling laws we are riding?
Training bigger models and giving them more time to think are two ways to improve AI. There are several others, and I am sure the labs have a few tricks up their sleeves that we don’t know about. Let’s go through them one by one. Sorry if I sometimes anthropomorphize, but I think it helps.
Training compute
The OG. Give the model more time to learn and practice. How much of that it gets depends on the computing power we use and how long we run it. More training compute lets us train a bigger model, give it more examples to learn from, or both.
In 2020, OpenAI showed that increasing model size, data, and training compute improved performance in a fairly predictable way. That publication gave us a way to estimate what more training could buy us, before the whole AI explosion.
Training compute has grown at roughly 4–5X a year. To make it tangible, if you invested $1K in 2010 and multiplied it by four every year, today you would have $4.3 trillion. That’s the kind of growth we are talking about.
Training-compute estimates for frontier AI models. Each step up means 10X more computation, not 10X more intelligence. Source: Epoch AI.
With such insane growth - bananas as my kids like to say - many people expected this route to run out of room, mostly because of data, but also because of compute. But new datasets are created every day, and synthetic data (data created by other models) gives us another source of training material. In other words, we have ways to keep going, even if we don’t yet know how far they will take us.
Test-time compute (September 2024)
This is the reasoning scaling law. The fancy name refers to giving the model time to think at inference time (”test-time”). Basically, like people, the same model can do better when given more time to think before producing an answer. Those of you who were early adopters may remember the good old tricks of adding “take your time” or “think about it” to the prompt.
With the release of o1-preview in September 2024, this became much more explicit: OpenAI trained the model to make better use of that extra thinking time. As we discussed a couple of weeks ago, test-time compute scales both horizontally (letting the model explore more possibilities) and vertically (letting the model think for longer).
Multi-agent systems
We humans create teams of people with different skills to solve different problems. Multi-agent systems are the same idea. Create teams of agents to solve more complex problems, or sometimes use compute more efficiently. This is an extension of giving models more time to think, with the advantages of separate contexts and task specialization.
If this works and scales, why not create thousands or millions of agents? Well, that’s the goal, but like humans, agents have coordination problems, and there is a limit to how many can work together efficiently. Improving that coordination so we can throw more and more agents at a problem is the scaling opportunity.
This route is still early, but the last couple of months have brought two examples of multi-agent work to public attention: the Millennium Problem and the Hugging Face hack. For the Millennium result, though, Noam Brown gives less than 10% of the credit to multi-agent work. He mainly credits the underlying model.
A coordinated team did better on financial research, where the work could be divided, and worse on planning, where each action affected the next. Making larger teams useful depends on getting that coordination right. Source: Kim et al., Towards a Science of Scaling Agent Systems.
Small note: we tend to think about coordinating agents through harnesses, the software around the model that organizes their work. My bet is that most of that coordination logic will end up in the foundation models themselves. Noam Brown describes how OpenAI already trains agents together to cooperate. The hope is that, as the models get better at this, we can put them to work in much larger teams.
Hardware and efficiency
This is the hardware example from above. As hardware gets better, models can do more computation in the same amount of time or, even more importantly, with the same amount of energy.
My feeling before researching this post was that we were quite optimized already. 2nm technology, a ton of RAM... how much room could there be left? Well, NVIDIA announced the H100 in March 2022, and by February 2026, OpenAI had trained GPT-5.3-Codex on the newer Blackwell generation. That is... the H100 announcement was almost four years old by then, and newer hardware was already helping train models that were changing how software engineering works.
Not only that, companies like Tensordyne, Euclyd, and Extropic are designing different kinds of AI chips, with claims of efficiency gains that reach several orders of magnitude on specific tasks.
Company claims and estimates for different tasks and hardware, so the numbers are not directly comparable. Extropic’s estimate comes from simulations on small generative-model benchmarks.
These companies still have to deliver on those claims at scale. But the idea that hardware is already as efficient as it can get seems premature. If these approaches work, the same energy budget could buy us much more computation, which gives us more room to keep going up the other curves.
Recursive self-improvement (RSI)
This one is a bit odd and makes my consultant brain twitch. Use AI to improve the process of making better AI, which then helps you... make better AI and repeat.
A simple example is the training code for LLMs. The basic code can be surprisingly short. Karpathy’s nanoGPT has roughly 300 lines for the training loop and another 300 for the model itself. And AI is already helping with this kind of work. OpenAI used early versions of GPT-5.3-Codex to debug its own training and help develop later versions. In fact, automating AI research has been an explicit goal of all the AI labs for years. This same idea can apply to all the other scaling routes: AI for better hardware, AI to improve datasets (synthetic data), AI to coordinate other AIs...
That’s why RSI could offer another scaling route: give AI more compute to find better ways to build AI. While we are constrained by compute (hardware or energy), deciding to point resources to RSI is a bet that improving how we build AI will give us more back than spending those resources on the current process. It’s in some ways... a meta decision. Do I point my models to more training, or do I point them to improving the code?
A possible outcome of recursive self-improvement: better AI accelerates the development of still better AI. Conceptual illustration, not a measured trend or a timeline. Source: Tim Urban, Wait But Why, 2015.
Test-time training
We still have not cracked what I think is the holy grail: models that keep learning from experience and carry that learning forward. Models that get better at understanding you, your needs, or the work they do, with that learning stored in their weights beyond the context window. Test-time training is one possible step in that direction.
We already have part of the mechanism in fine-tuning, which adapts a model’s weights to a specific task after its main training.
Test-time training takes this same idea to inference time, after the model sees a new task. The system uses available examples to update some of its weights before answering. This is akin to human learning from experience. It’s what some would call true AGI. One ARC study improved results on puzzles with just 2–7 solved examples per task, using variations of those examples as extra training material. The updates can be temporary or retained, depending on the setup.
An ARC-AGI puzzle: two solved examples show a rule, and the model must apply it to a new input. In test-time training research, the system uses solved examples and variations of them to update task-specific weights before answering. The missing answer is not used for training.
Can this generalize? Can this be done at scale? Those are still open questions. But if we crack it, we would have another way to improve models and potentially make repeated work cheaper.
Beyond LLMs
So far, I’ve focused on LLMs, but they are by no means the only AI out there. World models learn how the world changes and can simulate what happens after an action. TypeSafe’s Jev focuses on classification and decisions with an AI that is too cheap to meter. Then there are neuro-symbolic systems like AlphaGeometry 2, which combines a language model’s suggestions with mathematical rules to find proofs.
These approaches, and many others, could help continue pushing performance in two ways: some could eventually replace LLMs, while others could work alongside them and make the overall system more efficient. I have no idea which approaches will take us furthest, but continued AI progress does not depend on LLMs being the final answer. And that is the core of my argument: there is a lot of room for growth, even though what we have today already seems quite good.
AI can complete increasingly long software tasks. The chart measures task length by how long a human expert would take, at a 50% success rate for the AI. Shown on a linear scale.
Source: METR, updated May 8, 2026, before Astra’s release.
Why does this feel so far away?
The reality is that even if you agree with all the above, estimating where the world is going to be five years from now is impossible. We humans are terrible at understanding exponentials. Even if we grok what scaling laws mean and think we can see the compounding nature of AI progress, imagining what much more capable models would let us do is an extreme example of the gap of imagination.
In all honesty, I spend 24/7 thinking about and working with AI, and I have no idea what models being 100X better than today would mean in practice. (Small confession: I actually have a nerdy strategy nervous breakdown every time a new generation of models comes out. What should my first question be?)
That difficulty in imagining what comes next is part of why I think we keep having conversations about AI progress stalling every 2–3 months (although acceleration is now so obvious that it has been a while since the last one).
And it doesn’t help that the progress we can see is so uneven. Models can propose solutions to Millennium problems and yet fail at stupid brain teasers. Those failures tend to go viral and reinforce this short-sighted view of model performance. I think we also hold AI to a higher standard than humans, treating a silly mistake as a verdict on the whole model. Their abilities remain uneven, even as they outperform humans in more and more areas.
AI can outperform humans on some tasks and still fail at seemingly simple ones. The frontier can keep expanding while remaining jagged. Conceptual illustration from Ethan Mollick, One Useful Thing.
So, to conclude: AI is not stalling. Progress is accelerating as the different ways to improve AI reinforce each other. Even with all that, we don’t see it, and most people are still using Luzia, ChatGPT, and Muse as search engines. In other words, AI usage is a mile wide and an inch deep.
An analysis of Luzia queries over three years showed almost no increase in the complexity of users’ first conversations, and I have seen similar data from other labs. All this while Astra is solving math problems that have never been solved before.
The good news is that there are endless problems to tackle in the world, from silly ones to existential ones, and so far we have been constrained by intelligence. What will the world look like when we eventually have virtually infinite intelligence? Again, no idea, but it looks pretty cool.