"That is the immediate takeaway from the First Proof challengeperhaps the most robust test yet of the ability of large language models (LLMs) to perform mathematical research. Set by 11 top mathematicians on February 5, the results of the test were released early in the morning on Valentine's Day. It's too soon to conclusively say how many of the 10 math problems that were included in the challenge were solved by AIs without human help."
"The mathematicians behind First Proof presented the AIs 10 lemmasa math term for minor theorems that pave the way to a larger result. These problems are the working mathematician's stock-in-trade, the kind of mini problem one might hand off to a talented graduate student. The mathematicians aimed for problems that would require some originality to solve, not just a mash-up of standard techniques, according to Mohammed Abouzaid, a math professor at Stanford University and a member of the First Proof team."
"The challenge, while highlighting AI's limitations, also spotlights a budding AI-enthusiast subculture within the mathematics community. Online discussion boards and social media accounts dedicated to math were swamped with purported proofs from top mathematicians and rogue undergraduates alike. And it underscored how seriously AI startups, including ChatGPT maker OpenAI, are taking the challenge of teaching an LLM to do math."
The First Proof challenge presented ten lemmas to large language models to test their ability to perform mathematical research. Eleven mathematicians designed the problems and released results on February 14. The problems were chosen to require originality rather than standard technique recombination. Preliminary results show that none of the LLMs came close to solving all ten problems, and it remains unclear how many were solved without human help. The lemmas represented typical working-mathematician tasks and were comparable to assignments given to capable graduate students. The contest generated heavy online activity, with many purported proofs circulating on forums and social media.
Read at www.scientificamerican.com
Unable to calculate read time
Collection
[
|
...
]