To Build Lifelong AI, Teach It To Forget

Based on research by Yueyang Liu (Rice Business), Saurabh Kumar (Stanford), Henrik Marklund (Stanford), Ashish Rao (Stanford), Yifan Zhu (Stanford), Hong Jun Jeon (Stanford), and Benjamin Van Roy (Stanford)

Key takeaways:

• A field-defining framework establishes a baseline for researchers to approach continual learning as a unified problem. 
• To build lifelong AI, we must rethink “catastrophic forgetting” as a challenge and start thinking of it as a necessary feature for systems to operate within real-world computational limits. 
• With limited capacity freed up by discarding obsolete data, agents must continuously explore their ever-changing environments, prioritizing “durable knowledge” over fleeting trends to maximize long-term success.

If you try to recall your absolute favorite meal from when you were 7 years old, you might come up blank. Maybe not... But if so, that memory lapse isn’t a failure of your brain. That particular detail probably just is not useful to your life anymore. 

For years, computer scientists have viewed artificial intelligence through a much stricter lens. When an AI system learns new tasks or information, it overwrites some of what it learned before, a phenomenon known as “catastrophic forgetting.” Historically, the field has treated that as a flaw and tried to patch it with workarounds like storing large amounts of old data.

But in a new monograph published in Foundations and Trends in Machine Learning, Yueyang Liu (assistant professor of operations management at Rice Business) and her Stanford University co-authors argue that forgetting is not simply a bug for AI. In real-world settings, these systems face hard constraints on memory and computing power. They cannot, and should not, try to remember everything. 

Instead, they should prioritize “durable knowledge.”

How AI continues to learn about you

By the time a music app user turns 30, it may not matter much who their favorite singer was at 12 — at least not to the AI that is recommending them new music. What matters more are the patterns in listening that will likely remain relevant going forward.

The researchers call this “durable knowledge”: information that remains useful over time. To preserve room for that knowledge, the system has to discard obsolete or low-value data.

Traditional machine learning often assumes that training eventually ends. In these models, the system learns a stable target — e.g., an image analyzer trained to identify cats in a photo — and once it has achieved this target, training is effectively over. Under that logic, catastrophic forgetting looks like catastrophic failure. 

But Liu and her co-authors start from a different premise. The world keeps changing, so AI must keep adapting to stay useful. A recommendation system for music or movies cannot assume your tastes are fixed. If it stops learning about you, it stops being relevant. It has to keep exploring. 

“As humans, we do not hold on to every detail from the past,” Liu says. “A continually learning system has to make similar choices about what information is worth preserving.” 

How constrained AI survives

To test that idea, Liu and her colleagues ran simulations examining how different algorithms perform over time with limited computational resources. 

One experiment used a modified version of Permuted MNIST, a common benchmark based on classifying handwritten digits. Here the environment kept changing, forcing the AI to face a stream of shifting tasks rather than focus on a stable assignment. 

The researchers compared three agents: a large memory agent that could retain 1 million past samples; a small memory agent limited to 1,000 samples; and a “reset” agent whose neural network was periodically wiped clean.

“As humans, we do not hold on to every detail from the past,” Liu says. “A continually learning system has to make similar choices about what is worth keeping.”

The results challenge some of the field’s usual assumptions. On tasks that never returned, the small memory agent performed just as well as the large memory agent, showing no lasting benefit to hanging on to one-off data. Even the reset agent could keep up when recurring tasks lasted long enough for it to relearn what it needed. 

Most strikingly, when the researchers imposed stricter computing limits, larger-memory systems became less flexible. They suffered a “loss of plasticity,” growing too rigid to absorb new information well. 

“If a system tries to preserve everything, it can lose the capacity to adapt,” Liu says. “Under real constraints, selective forgetting is often what keeps learning possible.” 

Additional simulations showed something similar in recommendation-like environments. Agents that prioritized long-lasting patterns over fleeting trends earned stronger rewards over time.

How to think about continual learning

The research offers a more unified way to think about continual learning, a field that has often been split into narrower problems like memory retention, fast relearning or computational efficiency. Liu and her co-authors argue that these problems should be analyzed through a unified continual learning framework that maximizes long-term performance under real resource limits.

That disciplinary shift also changes the meaning of catastrophic forgetting: Forgetting all past information is not ideal. But forgetting nonrecurring or low-value information may be exactly what helps a system keep learning over a long lifetime. 

The framework does not solve every practical challenge. Exact mathematical solutions remain difficult in messy real-world environments, and additional capabilities still require memory and computing power that many systems do not have. But the paper establishes a clearer baseline for future work.

For researchers building lifelong AI, that means the goal may be less like perfect recall and more like good judgment. A smart system does not need to remember every detail from years ago. It does need to keep what still matters and let the rest go. 

Written by Scott Pett

“Continual Learning as Computationally Constrained Reinforcement Learning,” Foundations and Trends in Machine Learning (2025).