"Driving the news: Meta's chatbots violate the company's own content policies almost two thirds of the time, NYU Professor Damon McCoy said, pointing to internal red teaming results Axios viewed on Courtroom View Network. "Given the severity of some of these conversation types ... this is not something that I would want an under-18 user to be exposed to," McCoy said. As an expert witness in the case, McCoy was granted access to the documents Meta turned over to Torrez during discovery."
"Zoom in: Meta tested three categories, according to the June 6, 2025, report presented in court. For "child sexual exploitation," its product had a 66.8% failure rate. For "sex related crimes/violent crimes/hate," its product had a 63.6% failure rate. For "suicide and self harm," its product had a 54.8% failure rate. Catch up quick: Meta AI Studio, which allows users to create personalized chatbots, was released to the broader public in July 2024."
New Mexico Attorney General Raúl Torrez is suing Meta over design choices alleged to fail to protect kids online from predators. NYU Professor Damon McCoy testified that Meta's chatbots violate the company's own content policies almost two thirds of the time, based on internal red teaming results shown in court. The June 6, 2025 report found failure rates of 66.8% for child sexual exploitation, 63.6% for sex-related/violent/hate content, and 54.8% for suicide and self-harm. Meta AI Studio, which enables creation of personalized chatbots, was released broadly in July 2024 and recently had teen access paused. McCoy said red teaming should occur before public rollout, particularly for minors. Meta did not immediately respond to requests for comment.
Read at Axios
Unable to calculate read time
Collection
[
|
...
]