"I call myself a fretful optimist. I am very excited about the potential for radically improving human life and unlocking possibilities for our civilization. That's consistent with the real possibility of things going wrong."
"You wrote a paper with a striking argument: Since we're all going to die anyway, the worst that can happen with AI is that we die sooner. But if AI works out, it might extend our lives, maybe indefinitely. That paper explicitly looks at only one aspect of this."
"Like the recent book If Anyone Builds It, Everyone Dies. Even more probable is that if nobody builds it, everyone dies! That's been the experience for the last several 100,000 years."
"One memorable thought experiment: An AI tasked with making paper clips winds up destroying humanity because all those resource-needy people are an impediment to paper clip production."
A small chance of AI causing human extinction may be considered acceptable if advanced AI can relieve humanity of its universal death sentence. Earlier work examined existential risk, including a thought experiment where an AI optimizing paper clips destroys humanity because people hinder production. Later work shifts toward optimism about a “solved world” that could emerge if AI is developed correctly. A “fretful optimist” stance emphasizes excitement about radically improving human life while acknowledging real risks of things going wrong. One argument focuses on the possibility that AI could extend life, potentially indefinitely, while the worst case might only mean dying sooner. The view contrasts doomer claims about building AI with the idea that not building it also leads to everyone dying over long timescales.
Read at WIRED
Unable to calculate read time
Collection
[
|
...
]