To counter that, he revived oral exams and enlisted an AI agent to administer them at scale, in an attempt to "fight fire with fire." "We need assessments that evolve toward formats that reward understanding, decision-making, and real-time reasoning," Ipeirotis said. "Oral exams used to be standard until they could not scale," he added. "Now, AI is making them scalable again." In the blog post detailing the experiment, Ipeirotis said he and his colleague built the AI examiner using ElevenLabs' conversational speech technology.
The researchers analyzed the results of over 100,000 oral exams conducted at an Italian university and found a clear bell curve in pass rates that peaked at noon, regardless of the test taker's chronotype. (More on that in a moment.) Between 11 a.m. and 1 p.m. is the sweet spot; any earlier or later and the chances of passing significantly decreased. In fact, the earlier or later in the day students took a test, the less likely they were to pass.