Meta's Optimization Platform Ax 1.0 Streamlines LLM and System Optimization

"Now stable, Ax is an open-source platform from Meta designed to help researchers and engineers apply machine learning to complex, resource-intensive experimentation. Over the past several years, Meta has used Ax to improve AI models, accelerate machine learning research, tune production infrastructure, and more. Ax is especially oriented to researchers who need to understand and optimize AI models or other systems with complex configuration."

"In such cases, the sheer number of possible configurations makes it nearly impossible to evaluate all of them efficiently in a linear way, say Meta researchers. The solution comes in the form of adaptive experimentation, which automatically evaluates configurations in a sequential way, using the insights from previous evaluations to guide exploration of the solution space. Adaptive experiments are incredibly useful, but can be challenging to run."

"Examples of problems for which Meta used Ax internally include hyperparameter optimization and architecture search in machine learning, discovering optimal data mixtures to train AI models, tuning infrastructure, optimizing compiler flags, and more. One particularly interesting application of Ax is in optimizing LLMs. Meta researchers have provided a comprehensive introduction to it, demonstrating how Ax can be used to write better prompts, select the most effective examples the AI should follow, and more."

Ax is an open-source platform from Meta for applying adaptive experimentation to complex, resource-intensive machine learning and infrastructure problems. Ax enables sequential evaluation of configurations, using insights from prior trials to guide exploration and reduce exhaustive searches. Ax requires sophisticated optimization methods and specialized infrastructure to manage experiment state, orchestration, analysis, and diagnostics. Internal uses include hyperparameter optimization, architecture search, finding optimal data mixtures, tuning production systems, and optimizing compiler flags. Ax also supports work on large language models, including prompt improvement and selection of effective examples, and supports multi-objective optimization under constraints and guardrails.

Read at InfoQ