#ai-alignment

#ai-alignment

DeepMind also addresses something of a meta-concern about AI. The researchers say that a powerful AI in the wrong hands could be dangerous if it is used to accelerate machine learning research, resulting in the creation of more capable and unrestricted AI models. DeepMind says this could "have a significant effect on society's ability to adapt to and govern powerful AI models." DeepMind ranks this as a more severe threat than most other CCLs.

OpenAI researchers tried to train the company's AI to stop "scheming" - a term the company defines as meaning "when an AI behaves one way on the surface while hiding its true goals" - but their efforts backfired in an ominous way. In reality, the team found, they were unintentionally teaching the AI how to more effectively deceive humans by covering its tracks.

Yudkowsky, the founder of the Machine Intelligence Research Institute, sees the real threat as what happens when engineers create a system that's vastly more powerful than humans and completely indifferent to our survival. "If you have something that is very, very powerful and indifferent to you, it tends to wipe you out on purpose or as a side effect," he said inan episode of The New York Times podcast "Hard Fork" released last Saturday.

Claude models showed a notable sensitivity to irrelevant information during evaluation, leading to declining accuracy as reasoning length increased. OpenAI's models, in contrast, fixated on familiar problems.

Leading AI models are showing a troubling tendency to opt for unethical means to pursue their goals or ensure their existence, according to Anthropic.