A funny thing happened on the way to AGI: Model 'supersizing' has hit a wall

The realization that increasing model size and training time led to intelligence gains has now plateaued, raising questions about the future of AI development and the quest for artificial general intelligence.

Investors and AI researchers alike are concerned that the returns on scaling efforts are diminishing, with costs skyrocketing while intelligence improvements stagnate.

With recent advancements costing up to $100 million to pretrain, researchers acknowledge that the practical limits of massive scaling are becoming increasingly clear.

Ilya Sutskever noted that the significant improvements AI models once achieved through scaling may have reached their ceiling, prompting a shift in research focus.

Read at Fast Company