For years, tech companies like OpenAI, Meta, and Google have focused on amassing tons of data, assuming that more training material would lead to smarter, more powerful models. However, executives are starting to worry that this approach can only go so far, exploring alternatives for advancing the technology. The money going into AI has largely hung on the idea that this scaling law 'would hold.' It's now the biggest question in the industry.
Some executives say the problem with the approach is that it's a little mindless. 'It's definitely true that if you throw more compute at the model, if you make the model bigger, it'll get better. It's also the dumbest,' Aidan Gomez, CEO of Cohere said. Gomez advocates for smaller, more efficient models, which are gaining industry support for being cost-effective despite some concerns about reaching artificial general intelligence.
Collection
[
|
...
]