The performance degradation observed at later exits in NEO-KD is primarily due to adversarial examples applying greater cumulative losses, thus reducing accuracy in these later stages.
Implementing an ensemble strategy can effectively mitigate performance issues observed at later exits, especially when budgeted prediction setups are considered, enhancing overall model effectiveness.
Collection
[
|
...
]