Are We in an AI Bubble?

"In the study, published in July, the think tank Model Evaluation & Threat Research randomly assigned a group of experienced software developers to perform coding tasks with or without AI tools. It was the most rigorous test to date of how AI would perform in the real world. Because coding is one of the skills that existing models have largely mastered, just about everyone involved expected AI to generate huge productivity gains."

"But when the METR team looked at the employees' actual work output, they found that the developers had completed tasks 20 percent slower when using AI than when working without it. The researchers were stunned. "No one expected that outcome," Nate Rush, one of the authors of the study, told me. "We didn't even really consider a slowdown as a possibility.""

"On the one hand, the United States is undergoing an extraordinary, AI-fueled economic boom: The stock market is soaring thanks to the frothy valuations of AI-associated tech giants, and the real economy is being propelled by hundreds of billions of dollars of spending on data centers and other AI infrastructure. Undergirding all of the investment is the belief that AI will make workers dramatically more productive, which will in turn boost corporate profits to unimaginable levels."

A July METR study randomly assigned experienced software developers to complete coding tasks with or without AI tools. Experts predicted nearly a 40 percent productivity boost; participants afterward estimated a 20 percent speedup. Actual measured output showed developers were 20 percent slower when using AI compared with working without it. Researchers expressed surprise at the slowdown. Concurrently, the U.S. is experiencing an AI-fueled economic boom with massive investment in infrastructure driven by a widespread belief that AI will substantially raise worker productivity and corporate profits, while evidence mounts that practical gains may lag expectations.

Read at The Atlantic