Instead of building a network at full size on day one, we let it add parameters as it trains — and let an AI agent work out the best way to do it.
Scaling laws tell you how many parameters to buy. They quietly assume you keep all of them, the whole time.
A model early in training is learning word frequencies and basic grammar. Paying for a large model from step zero buys capacity long before there is anything to use it for.
Being small early buys 19% more training for the same money. That is the whole mechanism.
Every growing model beat every baseline. The gap holds even after we gave the baseline its own better learning rate.
The baseline was mis-tuned. It ran at a learning rate chosen for the growing model. Given its own, it became more stable than ours — and the gap halved.
The finish line was too big. We pinned both arms to a size roughly 10× above what this budget wants. Remove that and a plain small model wins.
Growth wins when the target size is fixed — not in general.
Growing too early and growing too late were both worse. Splitting growth into more, smaller steps changed nothing.
What won was finishing growth about a third of the way in, and alternating between wider and deeper rather than doing all the widening first.
It then stopped and told us the remaining differences were smaller than the noise between runs.
Growth runs on a fixed timetable today. It should fire when progress stalls — the sign that the current size is used up.
The agent showed alternating beats widening first. A model could try both and keep whichever helps more right then.
These are small models trained briefly. If growth still wins on a model people would actually use, it changes what a training budget buys.
Everything below is live right now — the agent is still searching, and the results page updates as it goes.