To prevent safety issues with large language models, companies use red-teaming where human testers create prompts to trigger toxic responses. However, missed prompts can lead to unsafe answers, prompting the need for improvement.
Researchers from MIT and IBM developed a machine learning technique to enhance red-teaming by training a model to generate diverse prompts that elicit toxic responses. This method outperformed human testers and other machine-learning approaches.
"Our method provides a faster and more effective way to ensure the safety of large language models compared to the current lengthy red-teaming process," commented researchers from Improbable AI Lab at MIT and the MIT-IBM Watson AI Lab.
[
add
]
[
|
|
...
]