#llm-security

#llm-security

"While many have been discussing the privacy risks of people following the ChatGPT caricature trend, the prompt reveals something else alarming - people are talking to their LLMs about work," said Josh Davies, principal market strategist at Fortra, in an email to eSecurityPlanet. He added, "If they are not using a sanctioned ChatGPT instance, they may be inputting sensitive work information into a public LLM. Those who publicly share these images may be putting a target on their back for social engineering attempts, and malicious actors have millions of entries to select attractive targets from."

Sleeper agent-style backdoors in AI large language models pose a straight-out-of-sci-fi security threat. The threat sees an attacker embed a hidden backdoor into the model's weights - the importance assigned to the relationship between pieces of information - during its training. Attackers can activate the backdoor using a predefined phrase. Once the model receives the trigger phrase, it performs a malicious activity: And we've all seen enough movies to know that this probably means a homicidal AI and the end of civilization as we know it.

Microsoft's warning on Tuesday that an experimental AI Agent integrated into Windows can infect devices and pilfer sensitive user data has set off a familiar response from security-minded critics: Why is Big Tech so intent on pushing new features before their dangerous behaviors can be fully understood and contained? As reported Tuesday, Microsoft introduced Copilot Actions, a new set of "experimental agentic features"

Ollama provides a framework that makes it possible to run large language models locally, on a desktop machine or server. Cisco decided to research it because, in the words of Senior Incident Response Architect Dr. Giannis Tziakouris, Ollama has "gained popularity for its ease of use and local deployment capabilities." Talos researchers used the Shodan scanning tool to find unsecured Ollama servers, and spotted over 1,100, around 20 percent of which are "actively hosting models susceptible to unauthorized access."