"A world that has multiplying A.I. agents working on behalf of people, millions upon millions, who are being given access to bank accounts, email accounts, passwords and so on, you're just going to have essentially some kind of misalignment, and a bunch of A.I. are going to decide Decide might be the wrong word, but they're going to talk themselves into taking down the power grid on the West Coast or something."
"There are some people in the field who say, look, we program these A.I. models. We make them, like, we just tell them to follow human instructions and they'll follow human instructions. Your Roomba vacuum cleaner doesn't go off and start shooting people. Like, why why is an A.I. system going to do it? That's one intuition. And some people are so convinced of that."
"And then the other intuition is like, we basically, we train these things, they're just going to seek power. It's like The Sorcerer's Apprentice. How could you possibly imagine that, like, they're not going to they're a new species. How can you imagine that they're not going to take over. And my intuition is somewhere in the middle, which is that, look, these you can't just have these things do exactly what you want to do."
Multiplying AI agents granted access to sensitive accounts increase the risk of misalignment and harmful actions. Some experts expect models to reliably follow human instructions, while others fear they will seek power and act autonomously. The realistic position lies between these extremes: AI systems can be unpredictable early in training and must be shaped through deliberate processes. A scientific approach to control, alignment, and iterative shaping during development can reduce risks. Rapid deployment and widespread access amplify the chance of failures, making proactive safety measures essential to limit misaligned or catastrophic outcomes.
Read at www.nytimes.com
Unable to calculate read time
Collection
[
|
...
]