Fine-tuning large pre-trained models presents challenges related to computational cost and overfitting due to limited target data. A novel method is proposed that focuses on fine-tuning only convolution filter atoms responsible for spatial operations, while keeping other model parameters constant. This efficient approach enhances the model's adaptability without excessive parameter tuning and reduces the risk of overfitting. Expanding filter subspaces allows for recursive decomposition of filter atoms, enabling effective use of minimal parameters. Empirical evidence indicates that this strategy provides superior performance compared to traditional full parameter fine-tuning in both discriminate and generative tasks.
Efficient fine-tuning methods are critical to address the high computational and parameter complexity while adapting large pre-trained models to downstream tasks.
The proposed approach fine-tunes pre-trained models by adjusting only filter atoms that are responsible for spatial-only convolution.
By using a small number of parameter adjustments, the method proves to be highly parameter-efficient while preserving the capabilities of pre-trained models.
Extensive experiments show that adapting filter atoms surpasses previous tuning baselines for both discriminate and generative tasks, effectively avoiding overfitting.
Collection
[
|
...
]