The article addresses the challenges in the development of vision-language models (VLMs), specifically focusing on the lack of justification for design choices that directly influence model performance. It details extensive experiments on various aspects of VLMs, including pre-trained models and architectural variations. A noteworthy contribution is Idefics2, a foundational VLM model with 8 billion parameters that demonstrates state-of-the-art results while maintaining efficiency. The authors emphasize the importance of systematic experimentation and have released the model and associated datasets to foster further research in the field.
The advancement of vision-language models (VLMs) relies on foundational design choices, yet many lack justification, hindering progress by obscuring performance improvements.
Idefics2, our 8 billion parameter vision-language foundation model, achieves state-of-the-art benchmarks and matches models much larger in size, reflecting efficient design and training.
Collection
[
|
...
]