Concept frequency in pretraining data significantly influences zero-shot performance. Evidence shows that a strong log-linear trend exists between the frequency of concepts in the training set and their subsequent performance in zero-shot tasks. This relationship is consistent across diverse prompting strategies, including simple class names and more complex phrases. Additional analyses confirm that concept frequency predicts performance not only in standard classification metrics but also across several retrieval metrics, reinforcing the idea that the quality and prevalence of concepts in pretraining are critical for efficient model performance.
The strong log-linear trend between concept frequency and zero-shot performance consistently holds across different prompting strategies, indicating that more frequently encountered concepts in pretraining data yield better performance.
Results demonstrate that concept frequency is predictive of performance across various retrieval metrics, with data supporting findings across multiple prompting strategies and different metrics used in evaluation.
Collection
[
|
...
]