The new model, called Llama 3.1 405B, is the first openly available model that can compete available rivals in general knowledge, math skills and translating across multiple languages. The model was trained on more than 16,000 NVIDIA H100 GPUs, currently the fastest available chips that cost roughly $25,000 each, and can beat rivals on over 150 benchmarks, Meta claimed.
The '405B' stands for 405 billion parameters, which are internal variables that an AI model uses to reason and make decisions. The higher the number of parameters an AI model has, the smarter we perceive it to be. OpenAI's GPT-4 model, by comparison, is reportedly has roughly 1.5 trillion parameters, although the company has not disclosed the number so far.
In addition, Meta also released upgraded versions of existing Llama models that contain 70 billion and 8 billion parameters each, claiming that the newer versions had stronger reasoning abilities among other things.
'Llama 405B's improved reasoning capabilities make it possible for Meta AI to understand and answer your more complex questions, especially on the topics of math and coding,' Meta's blog post states.
[
Collection
]
[
|
...
]