#llm-security tag

Information security

NCSC issues urgent warning over growing AI prompt injection risks - here's what you need to know

fromSecurityWeek

Artificial intelligence

ChatGPT Tricked Into Solving CAPTCHAs

Information security

New attack on ChatGPT research agent pilfers secrets from Gmail inboxes

LegalPwn: Tricking LLMs by burying flaw in legal fine print

Embedding adversarial instructions within legal-style text can bypass LLM guardrails, enabling prompt-injection attacks that force models to produce harmful or disallowed outputs.

fromLogRocket Blog

How to protect your AI agent from prompt injection attacks - LogRocket Blog

Prompt injection attacks exploit LLMs' instruction-following ability to manipulate agents, risking data exfiltration, unauthorized actions, and control-flow hijacking.

1 day ago

Information security

ChatGPT falls to new data-pilfering attack as a vicious cycle in AI continues

fromIT Pro

Information security

NCSC issues urgent warning over growing AI prompt injection risks - here's what you need to know

fromSecurityWeek

Artificial intelligence

ChatGPT Tricked Into Solving CAPTCHAs

Information security

New attack on ChatGPT research agent pilfers secrets from Gmail inboxes

Artificial intelligence

LegalPwn: Tricking LLMs by burying flaw in legal fine print

fromLogRocket Blog

Information security

How to protect your AI agent from prompt injection attacks - LogRocket Blog

more#prompt-injection

fromInfoQ

1 week ago

Open-Source Agent Sandbox Enables Secure Deployment of AI Agents on Kubernetes

Agent Sandbox provides a secure, stateful, declarative Kubernetes API to run isolated, persistent, single-pod environments for safely executing untrusted LLM-generated code and stateful workloads.

Critics scoff after Microsoft warns AI feature can infect machines and pilfer data

Microsoft's warning on Tuesday that an experimental AI Agent integrated into Windows can infect devices and pilfer sensitive user data has set off a familiar response from security-minded critics: Why is Big Tech so intent on pushing new features before their dangerous behaviors can be fully understood and contained? As reported Tuesday, Microsoft introduced Copilot Actions, a new set of "experimental agentic features"

Information security

EchoGram tokens like '=coffee' flip AI guardrail verdicts

EchoGram uses short tokens (for example =coffee) to bypass LLM guardrails, enabling prompt injection and jailbreaking of model safety filters.

LLM side-channel attack could allow snoops to guess topic

A side-channel attack named Whisper Leak can infer prompt topics from encrypted streaming LLM traffic by analyzing packet size and timing, exposing user communications.

fromSecurityWeek

Will AI-SPM Become the Standard Security Layer for Safe AI Adoption?

AI security posture management continuously detects, evaluates, and remediates AI and LLM security and compliance risks, enabling transparent, governed, and policy-aligned AI use.

Startup companies

fromBusiness Insider

This AI startup helps developers safely and cheaply build on top of LLMs from OpenAI and Anthropic. Read its pitch deck.

Requesty centralizes developer access to multiple AI providers, tightening security, enforcing governance, and optimizing costs across large language models.

fromSecuritymagazine

Generative AI Remains Growing Concern for Organizations

Generative AI adoption is outpacing enterprise security readiness, leaving LLM deployments under-assessed and high-severity vulnerabilities largely unresolved.

AI can't stop the sprint to adopt hot tech without security

Ollama provides a framework that makes it possible to run large language models locally, on a desktop machine or server. Cisco decided to research it because, in the words of Senior Incident Response Architect Dr. Giannis Tziakouris, Ollama has "gained popularity for its ease of use and local deployment capabilities." Talos researchers used the Shodan scanning tool to find unsecured Ollama servers, and spotted over 1,100, around 20 percent of which are "actively hosting models susceptible to unauthorized access."

Information security