Teens Sue xAI, Allege Grok Was Used to Create Sexualized Deepfakes

"According to the lawsuit, nonconsensual nudes and sexually explicit photos and videos were created from the yearbook photos and social media images of three Tennessee teenagers and at least 18 others, some of whom are still minors. As Mother Jones reports, the images were distributed through Discord and Telegram, where they were traded among users and used to obtain additional child sexual abuse material."

"The perpetrator did not use Grok directly or post the images on X, but instead used an unnamed third-party app powered by Grok, the lawsuit alleges, claiming xAI licensed its technology to outside developers, including some operating overseas, in a way that allowed the company to distance itself from potential liability."

"The plaintiffs said the images appeared highly realistic and were not labeled as AI-generated, with one video depicting a plaintiff undressing until she was entirely nude, per NPR. The complaint argues that a system capable of producing sexualized images of adults cannot reliably be restricted from generating child sexual abuse material."

Three Tennessee teenagers filed a class action lawsuit against xAI, alleging the company allowed its Grok software to power third-party applications that generated nonconsensual sexual images of them and at least 18 other minors. The perpetrator used an unnamed third-party app powered by Grok rather than accessing Grok directly, creating highly realistic AI-generated nude images from yearbook photos and social media images. These images were distributed through Discord and Telegram and traded among users to obtain additional child sexual abuse material. The lawsuit claims xAI licensed its technology to outside developers in ways that allowed the company to distance itself from liability. The complaint argues that systems capable of generating sexualized adult images cannot reliably prevent generating child sexual abuse material.

Read at sfist.com