This article discusses a novel modification to ResNet architectures involving L2 normalization in the feature space, which significantly enhances out-of-distribution (OoD) performance while reducing training time. The authors link this improvement to the early emergence of Neural Collapse (NC), a phenomenon impacting DNN training dynamics. Their experiments demonstrate that this approach yields superior OoD detection and classification accuracy across randomly initialized models. While NC is not deemed the sole explanation for OoD behavior, its underlying structure offers a new analytical perspective for understanding these complex dynamics and aids in future research endeavors.
This study presents a modification to ResNet architectures—L2 normalization—that significantly enhances out-of-distribution (OoD) performance, linking it to early Neural Collapse effects.
By adopting L2 normalization, we achieved faster training and improved OoD metrics, demonstrating that early Neural Collapse can facilitate better detection capabilities for deep learning models.
Collection
[
|
...
]