"Model selection and evaluation is not just done at the beginning, but is something that's repeated periodically," Sivasubramanian said. "We think having a human in the loop is important, so we are offering a way to manage human evaluation workflows and metrics of model performance easily."
"If humans are involved, users can choose to work with an AWS human evaluation team or their own. Customers must specify the task type (summarization or text generation, for example), the evaluation me
[
add
]
[
|
|
...
]