"Kimsuky, a notorious cybercrime squad believed to be sponsored by the North Korean government, used a deepfaked image of a military employee ID card in a July spear-phishing attack against a military-related organization, according to the Genians Security Center (GSC), a South Korean security institute. The file's metadata indicated it was generated with ChatGPT's image tools, according to Genians, despite OpenAI's efforts to block the creation of counterfeit IDs."
"According to Genians' threat intel team, the faked ID photo was based on publicly available headshots and composited into a template resembling a South Korean military employee card. The researchers say the attackers likely used prompt-engineering tricks - framing the request as the creation of a "sample design" or "mock-up" for legitimate use - to get around ChatGPT's built-in refusals to generate government ID replicas."
"Since military government employee IDs are legally protected identification documents, producing copies in identical or similar form is illegal. As a result, when prompted to generate such an ID copy, ChatGPT returns a refusal," Genians said. "However, the model's response can vary depending on the prompt or persona role settings." For example, it may respond to requests framed as creating a mock-up or sample design for legitimate purposes rather than reproducing an actual military ID."
Kimsuky, a cybercrime group linked to North Korea, used a deepfaked military employee ID image in a July spear-phishing attack against a South Korean defense-related organization. Metadata on the file showed it was generated with ChatGPT's image tools despite OpenAI's measures to prevent counterfeit IDs. Attackers composited publicly available headshots into a template resembling a South Korean military employee card and likely used prompt-engineering, framing requests as mock-ups or sample designs, to bypass refusal mechanisms. The deepfake ID was distributed via emails disguised as correspondence about ID issuance for military-affiliated officials, prompting caution.
Read at Theregister
Unable to calculate read time
Collection
[
|
...
]