The article discusses a significant enhancement to the Deep Deterministic Uncertainty (DDU) benchmark through a simple code modification, leading to improved out-of-distribution (OoD) detection and classification results whilst reducing training duration. It establishes the relationship between L2 normalization, Neural Collapse (NC), and demonstrating that NC may enhance OoD performance. The authors advocate for further exploration into how NC relates to uncertainty in deep learning, acknowledging its potential impact on the understanding of neural networks' behavior in dynamic environments. This study paves the way for future research addressing robustness in deep neural networks.
We propose a simple, one-line-of-code modification of the Deep Deterministic Uncertainty benchmark that provides superior OoD detection and classification accuracy results in a fraction of the training time.
Although we do not suggest that NC is the sole explanation for OoD performance, we do expect that its simple structure can provide insight into the complex and poorly understood behaviour of uncertainty in deep neural networks.
We establish that L2 normalization induces NC faster than regular training, and that NC is linked to OoD detection performance under the DDU method.
This connection between Neural Collapse and Out-of-Distribution detection is a compelling area of future research into uncertainty and robustness in DNNs.
Collection
[
|
...
]