Researchers propose a self-distillation fix for 'catastrophic forgetting' in LLMs

""To enable the next generation of foundation models, we must solve the problem of continual learning: enabling AI systems to keep learning and improving over time, similar to how humans accumulate knowledge and refine skills throughout their lives," the researchers noted. Reinforcement learning offers a way to train on data generated by the model's own policy, which reduces forgetting. However, it typically requires explicit reward functions, which are not easy in every situation."

"SDFT suggests an alternative. Instead of inferring a reward function, it uses the model's in-context learning ability to generate on-policy learning signals from demonstrations."

Continual learning is presented as a necessary capability for foundation models to keep learning and improving over time, similar to human lifelong knowledge accumulation and skill refinement. Reinforcement learning can reduce forgetting by training on data generated by the model's own policy, but it often depends on explicit reward functions that can be difficult to specify. SDFT offers an alternative approach that bypasses reward inference. SDFT leverages the model's in-context learning ability to produce on-policy learning signals from demonstrations, enabling continued improvement without requiring handcrafted reward signals.

Read at Computerworld