Confidence in AI-powered cyber must be earned, not assumed | Computer Weekly

"Unlike deterministic systems, AI models behave probabilistically. Their outputs will vary depending on context, input structure and interaction history. This creates a big challenge for evaluation. Tools that may seem reliable in testing can quickly become unpredictable when attackers actively try to evade them or when confronted with imperfect real-world data."

"For security leaders, this means the usual, familiar testing and validation approaches are not enough. The question is not just whether an AI-powered tool works but how it actually behaves when it is stressed, manipulated, or forced to operate outside already known conditions."

"AI is being used on both sides of security operations. Offensive AI security tools support activities such as penetration testing, vulnerability discovery, attack simulation, and red teaming. They are designed to augment, not replace, the work security professionals already do, accelerating reconnaissance, exploring attack paths and identifying weaknesses at scale."

AI adoption in security is accelerating, but CISOs face challenges distinguishing genuine capabilities from marketing promises. Unlike deterministic systems, AI models behave probabilistically, producing variable outputs based on context and input structure. Tools appearing reliable in controlled testing may become unpredictable when attackers attempt evasion or when encountering imperfect real-world data. Traditional validation methods are inadequate for AI-powered security tools. Security leaders must evaluate not just whether tools work, but how they perform under stress, manipulation, and novel conditions. AI supports both offensive operations like penetration testing and vulnerability discovery, and defensive functions including detection, investigation, and response, augmenting rather than replacing human security professionals.

Read at ComputerWeekly.com