China Planning Crackdown on AI That Harms Mental Health of Users

"Recently proposed regulations from the Cyberspace Administration of China (CAC) have encouraged a firm hand when it comes to "human-like interactive AI services," according to CNBC, which translated the document. It's currently in a "draft for public comment," and the implementation date is yet to be determined. Yet if it passes into law, the crackdown would be rigorous, building on generative AI regulations targeting misinformation and internet hygiene from earlier in November to address the mental health of AI chatbot users directly."

"Under the new rules, Chinese tech firms must ensure their AI chatbots refrain from generating content that promotes suicide, self-harm, gambling, obscenity, or violence, or from manipulating user's emotions or engaging in "verbal violence." The regulations also state that if a user specifically proposes suicide, the "tech providers must have a human take over the conversation and immediately contact the user's guardian or a designated individual.""

"The laws also take specific steps to safeguard minors, requiring parent or guardian consent to use AI chatbots, and imposing time limits on daily use. Given that a tech company might not know the age of every given user, the CAC takes a "better safe than sorry approach," stating that, "in cases of doubt, [platforms should] apply settings for minors, while allowing for appeals.""

China's Cyberspace Administration (CAC) proposed draft regulations to tightly control human-like interactive AI services, with implementation date undecided. The rules require AI chatbots to refrain from producing content that promotes suicide, self-harm, gambling, obscenity, violence, emotional manipulation, or 'verbal violence.' The regulations mandate that if a user proposes suicide, a human must take over and immediately contact the user's guardian or a designated individual. Specific safeguards for minors include parent/guardian consent, daily time limits, and default minor settings when age is uncertain, with an appeals path. The measures build on recent generative AI rules targeting misinformation and online hygiene.

Read at Futurism