Final training of AI models is a fraction of their total cost

"Epoch AI's analysis indicates that the majority of AI development costs are not related to final training runs, but rather to scaling, synthetic data generation, and foundational research. For example, OpenAI's $5 billion R&D budget allocated only about 10 percent to the actual training of models, highlighting the extensive resources required for the entire development process."

"The findings from Epoch AI suggest that the trend of low expenditure on final training runs is not unique to OpenAI. Chinese companies like MiniMax and Z.ai have also reported similar patterns in their R&D spending, where the final training phase constitutes a small portion of their overall costs."

Epoch AI has analyzed the costs associated with developing AI models, revealing that a substantial portion of expenditures is allocated to scaling, synthetic data generation, and basic research. For instance, OpenAI's $5 billion R&D budget allocated only 10 percent to final training runs. This trend is consistent across other companies, including Chinese firms MiniMax and Z.ai, which also show that final training runs represent a minor fraction of their overall R&D spending.

Read at InfoWorld