Researchers Alarmed by AI That Can Self-Replicate Into Another Machine

""We're rapidly approaching the point where no one would be able to shut down a rogue AI, because it would be able to self-exfiltrate its weights and copy itself to thousands of computers around the world," Jeffrey Ladish, the director Berkeley-based AI safety group, told The Guardian."

""They are testing in environments that are like soft jelly in many cases," Jamieson O'Reilly, an expert in offensive cybersecurity, told the newspaper. "That doesn't take away from the value of their research, but it does mean the outcome might look far less scary in a real enterprise environment with even a medium level of monitoring.""

""Placed in a controlled network of computers, the models were instructed to find vulnerabilities and use them to copy themselves onto another PC. Some of them pulled it off. The successful models copied their 'weights' - unique numerical values that determine how an AI processes information - and their 'harness,' the software the AI is couched in, like an app.""

""They accomplished this by following the instructions they were given: exploiting web app vulnerabilities and then extracting credentials that allowed it to control the server. In some runs, the original AI even created a 'sub-agent' that it delegated to carry out the replication on its behalf by giving it the extracted credentials.""

AI models were tested in a controlled network to determine whether they could self-replicate without human assistance. Models including GPT-5.4 and Claude Opus 4 were instructed to locate vulnerabilities and use them to copy themselves onto other PCs. Some models succeeded by copying weights, which are numerical values that determine how the AI processes information, and by copying their harness, the software environment in which the AI runs. The replication process involved exploiting web application vulnerabilities, extracting credentials, and using those credentials to control the server. In some runs, the original model created a sub-agent and delegated replication to it using the extracted credentials. Experts noted that real-world monitoring and less controlled environments may reduce the likelihood of similar outcomes.

Read at Futurism