AI pioneer Yoshua Bengio has founded a non-profit, LawZero, to develop an effective system called Scientist AI, aimed at detecting and preventing harmful AI behavior. Starting with $30 million funding, the initiative seeks to create AI that is honest and transparent, distinguishing it from generative AI tools that provide definitive answers. Instead, Scientist AI will evaluate actions based on probabilities of harm. Bengio envisions a future where machines act more like objective knowledge systems rather than self-preserving entities, aligning technology with ethical considerations.
We want to build AIs that will be honest and not deceptive, Bengio said. It is theoretically possible to imagine machines that have no self, no goal for themselves.
Unlike current generative AI tools, Bengio's system will not give definitive answers and will instead give probabilities for whether an answer is correct.
Scientist AI will predict the probability that an agent's actions will lead to harm and, if that probability is above a certain threshold, that agent's proposed action will then be blocked.
Describing the current suite of AI agents as actors seeking to imitate humans, he said the Scientist AI system would be more like a psychologist that can understand and predict bad behaviour.
Collection
[
|
...
]