#model-evaluation

#model-evaluation

The analysis reveals a long-tailed distribution of concept frequencies in pretraining datasets, with over two-thirds of concepts occurring at negligible frequencies relative to dataset size.

The MS MARCO Web Search dataset presents a multilingual landscape, uncovering significant data skew that may impact model performance and necessitates data-centric optimization techniques for improvement.

In evaluating Chameleon, we focus on tasks requiring text generation conditioned on images, particularly image captioning and visual question-answering, with results grouped by task specificity.

Our work identifies 12 important aspects in real-world deployments of text-to-image generation models, including alignment, quality, aesthetics, reasoning, bias, and efficiency.