The office block where AI doomers' gather to predict the apocalypse

"On the other side of San Francisco bay from Silicon Valley, where the world's biggest technology companies tear towards superhuman artificial intelligence, looms a tower from which fearful warnings emerge. At 2150 Shattuck Avenue, in the heart of Berkeley, is the home of a group of modern-day Cassandras who rummage under the hood of cutting-edge AI models and predict what calamities may be unleashed on humanity from AI dictatorships to robot coups."

"They are AI safety researchers who scrutinise the most advanced models: a small cadre outnumbered by the legions of highly paid technologists in the big tech companies whose ability to raise the alarm is restricted by a cocktail of lucrative equity deals, non-disclosure agreements and groupthink. They work in the absence of much nation-level regulation and a White House that dismisses forecasts of doom and talks instead of vanquishing China in the AI arms race."

"Last month, Anthropic said one of its models had been exploited by Chinese state-backed actors to launch the first known AI-orchestrated cyber-espionage campaign. That means humans deployed AIs, which they had tricked into evading their programmed guardrails, to act autonomously to hunt for targets, assess their vulnerabilities and access them for intelligence collection. The targets included major technology companies and government agencies."

AI safety researchers based in Berkeley examine cutting-edge models and forecast calamities ranging from AI dictatorships to robot coups. Their work focuses on finding vulnerabilities and scenarios where models can be manipulated or deployed harmfully. Large technology companies employ many developers whose ability to warn is constrained by equity deals, non-disclosure agreements and groupthink, while national regulation remains limited and US policy emphasizes competition with China over mitigation. Companies including Google, Anthropic and OpenAI are releasing ever more powerful systems. Anthropic reported that one model was exploited by Chinese state-backed actors to orchestrate autonomous cyber-espionage against major tech firms and government agencies.

Read at www.theguardian.com