In 2023, a group of researchers from the University of California, Berkeley, started Chatbot Arena, now called LMArena. It allows people to compare different AI models with prompts and determine which is better. Users can vote for how well models perform and compare them on a leaderboard. LMArena saw a tenfold traffic spike in August when a mysterious new AI text-to-image and image editing model, Nano Banana, went viral for churning out impressive images and photo edits.
In testing the predictive power of pretraining concept frequencies, the results show a clear scaling trend where increased frequency directly correlates with improved zero-shot performance across different prompting strategies and evaluation metrics.