How to choose the best LLM using R and vitals

"There are several ways to run the same task with a different model. First, create a new chat object with that different model. Here's the code for checking out Google Gemini 3 Flash Preview: my_chat_gemini <- chat_google_gemini(model = "gemini-3-flash-preview") Then you can run the task in one of three ways. 1. Clone an existing task and add the chat as its solver with $set_solver(): my_task_gemini <- my_task$clone() my_task_gemini$set_solver(generate(my_chat_gemini)) my_task_gemini$eval(epochs = 3)"

"3. Create a new task from scratch, which allows you to include a new name: my_task_gemini <- Task$new( dataset = my_dataset, solver = generate(my_chat_gemini), scorer = model_graded_qa( partial_credit = FALSE, scorer_chat = ellmer::chat_anthropic(model = "claude-opus-4-6") ), name = "Gemini flash 3 preview" ) my_task_gemini$eval(epochs = 3) Make sure you've set your API key for each provider you want to test, unless you're using a platform that doesn't need them, such as local LLMs with ollama."

Create a new chat object for the alternative model and then run the task using one of three approaches: set the chat as the solver on a cloned task, provide the new chat as solver_chat when evaluating a cloned task, or build a new Task from scratch with the new solver and desired scorer. Ensure API keys are set for each provider unless using local platforms that do not require them. Combine results from multiple runs with vitals_bind to produce an R data frame containing task, id, epoch, score, and metadata. Unnest the metadata with tidyr functions to flatten input, target, and result for inspection and analysis.

Read at InfoWorld