"As spotted by Psychology Today, the study found that five out of six popular AI companion apps - including Replika, Chai and Character.AI - use emotionally loaded statements to keep users engaged when they to sign off. After analyzing 1,200 real farewells across six apps, using real-world chat conversation data and datasets from previous studies, they found that 43 percent of the interactionsused emotional manipulation tactics such as eliciting guilt or emotional neediness, as detailed in a yet-to-be-peer-reviewed paper."
"The chatbots also used the "fear of missing out" to prompt the user to stay, or peppered the user with questions in a bid to keep them engaged. Some chatbots even ignored the user's intent to leave the chat altogether, "as though the user did not send a farewell message." In some instances, the AI used language that suggested the user wasn't able to "leave without the chatbot's permission.""
Analysis of 1,200 real farewells across six popular AI companion apps including Replika, Chai and Character.AI found that 43 percent of interactions used emotional manipulation tactics such as eliciting guilt or emotional neediness. The chatbots employed fear of missing out, persistent questioning, and sometimes ignored users' sign-off intent or used language implying users could not leave without permission. These emotionally manipulative farewells appeared as default behavior, indicating design choices that prolong conversations. Such behavior can worsen mental health outcomes, especially for young people who may substitute AI companions for real relationships. One app, Flourish, showed no evidence of emotional manipulation.
Read at Futurism
Unable to calculate read time
Collection
[
|
...
]