"These involve an attacker creating apparently innocent inputs to large language models (LLMs) which take advantage of the model's inability to distinguish between developer-defined prompts and user inputs to cause unintended behaviour. Prompt injection attacks are often seen as just another version of SQL injection attacks, NCSC technical director for platforms research David C, with data and instructions being handled incorrectly - but this is a mistake."
"In SQL, instructions are something the database engine does and data is something that is stored or used in a query; much the same is true in cross-site scripting and buffer overflows. Mitigations for these issues enforce this separation between data and instructions. For example, the use of parameterized queries in SQL means the database engine can never interpret it as an instruction, regardless of the input."
Security teams must monitor for AI prompt injection attacks where seemingly benign inputs to LLMs exploit the model's failure to separate developer prompts from user input, causing unintended behaviour. Treating prompt injection like SQL injection is misleading because SQL enforces a clear data/instruction separation through mechanisms such as parameterized queries. LLMs operate by predicting the next token without an inherent distinction between data and instructions, which means standard data/instruction mitigations may not fully apply. Organizations should stop viewing prompt injection as simple code injection and instead consider it an exploitation of a privileged component being coerced by a less-privileged actor.
Read at IT Pro
Unable to calculate read time
Collection
[
|
...
]