Bigger neural networks are supposed to be smarter. More parameters, more training data, better results. That's been the iron law of AI scaling for a decade. Except it's not. According to recent research, the largest language models actively forget knowledge their smaller siblings learned—losing recall of rare facts by as much as 40 percent.
Everyone operates on the same intuition here: scale is a universal solvent. More compute, more weights, more training examples feeding into the system should equal monotonic improvement across the board. It's the logic that justified spending billions on larger models. The assumption is straightforward and, frankly, sensible. More parameters plus more data should equal better performance. But the data tells a different story. Research tracking how well models recall rare patterns reveals a strange, counterintuitive hump: performance improves rapidly up to 10 billion parameters, holds steady between 10B and 70B, then nosedives as models scale beyond 70 billion parameters.
The evidence is stark. According to analysis of recall performance across model sizes, improvement holds firm during early scaling—that's what we expected. But somewhere between 70 billion and the largest models in use today, the curve inverts. For the most obscure items in a model's training data, recall consistently dropped by as much as 40 percent in the largest models compared to mid-sized ones. This isn't noise or a measurement artifact. The pattern is reproducible and systematic. Models are literally unlearning rare knowledge as they grow larger.
Why would a bigger model be worse at remembering niche facts? The leading hypothesis points to compression and interference. As models scale, they develop more efficient internal representations—they learn to abstract and generalize. This is usually good: it lets them handle novel situations and find common patterns. But efficiency has a cost. When training data is shuffled into increasingly compressed representations, rare patterns get bulldozed. Think of it like urban development: as you optimize a city for traffic flow, you might pave over the quirky used bookstore that only mattered to a few people. The model has learned to prioritize what appears frequently, and in doing so, it's crowded out the uncommon edges.
There's also a training dynamics story here. Larger models, with vastly more parameters, can memorize entire datasets if left unsupervised. But modern training uses techniques like regularization and dropout to prevent overfitting. These techniques work differently at different scales. What prevents a 70B model from memorizing junk might actually be erasing its memory of rare-but-real facts by the time it reaches a trillion parameters. The model is being penalized for knowing things most examples don't prepare it for.
The implications are uncomfortable. We've built an industry assumption that scaling solves everything—that we can throw more data and compute at any problem and come out ahead. But if your goal is to build AI systems that retain specialized, rare, or niche knowledge, throwing scale at it might backfire. A smaller model might actually serve your purposes better. This doesn't kill the scaling agenda, but it does suggest we've been thinking about it wrong. The future might not be about building one enormous model that does everything, but about building ensembles of different-sized models, each retaining what the others forget.