OpenAI turns to red teamers to prevent malicious ChatGPT use as company warns future models could pose 'high' security risk

"The ChatGPT developer has previously detailed attempts by criminals to use its AI models to automate malware campaigns, banning accounts with suspicious activity. Now, it plans to boost its efforts to avoid AI models being used in cyber attacks by training models to avoid malicious use, hiring red teaming organizations to test systems, and setting up a trusted partner system so only known groups can access the latest models for security purposes."

"Using one benchmark, GPT-5 scored 27% on a capture the flag challenge, but just a few months later GPT-5.1-Codex-Max scored 76%, OpenAI said. The company expects upcoming models will "continue on this trajectory." Because of that, OpenAI said it was planning for each model as though it would reach "high levels of cybersecurity capability." "By this, we mean models that can either develop working zero-day remote exploits against well-defended systems, or meaningfully assist with complex, stealthy enterprise"

OpenAI plans multiple measures to reduce model misuse in cyber attacks, including training models to avoid malicious use, hiring external red teaming organizations, and restricting access to the latest models to trusted partners. An agentic security researcher called Aardvark has entered private beta. Benchmarks show rapid growth in exploit-like capabilities — GPT-5 scored 27% on a capture‑the‑flag challenge while GPT‑5.1‑Codex‑Max later scored 76% — prompting planning as if future models could reach high cybersecurity capability. Threat actors have already started using AI in malware and ransomware, and security firms have observed AI-adaptive malware and misuse of intelligence reports.

Read at IT Pro