Meta's SPICE framework pushes AI toward self-learning without human supervision

"Meta researchers have unveiled a new reinforcement learning framework called SPICE (Self-Play in Corpus Environments) that enables large language models (LLMs) to improve their reasoning skills without human supervision. Developed with the National University of Singapore, SPICE trains a single model to act as both a Challenger, which generates complex, document-based problems, and a Reasoner, which solves them. By grounding the learning process in real-world text corpora rather than synthetic data, the system avoids the hallucination loops that have plagued earlier self-play methods."

""Without external grounding, models inevitably plateau or collapse due to two critical issues," the researchers said in the paper. "(1) hallucination amplification, where factual errors in both generated questions and answers compound as models train on their own unverifiable synthetic data, and (2) information symmetry, where both the problem generator and solver share the same knowledge base, preventing genuine challenge and leading to simpler, more repetitive patterns.""

SPICE is a reinforcement learning framework that trains a single large language model to act as both a Challenger and a Reasoner using real-world text corpora. The Challenger generates complex, document-based problems and the Reasoner attempts to solve them, enabling self-play within corpus environments. Grounding the learning process in web documents prevents hallucination loops caused by training on synthetic, unverifiable data. The framework also reduces information symmetry by providing distinct, document-based challenges that force genuine difficulty. SPICE produces nearly 10% average improvements on mathematical and general reasoning benchmarks, demonstrating effective unsupervised self-improvement.

Read at Computerworld