Open-source AI models can be exploited for criminal activities: Study
A new study shows that open-source AI chatbots, like Meta's Llama and Google DeepMind's Gemma, are being used in ways researchers didn't expect—including by hackers.
After tracking thousands of servers running these models worldwide over 293 days, experts found thousands of deployments with security issues, noting hundreds of instances where guardrails had been removed and that system prompts were visible in roughly a quarter of the models they observed (with about 7.5% of those potentially enabling harmful activity).
Where things went wrong
Roughly 30% of these AI setups were operating out of China and about 20% in the US.
In about a quarter of cases, system prompts were visible—making it easier for bad actors to mess with them.
Hundreds had their safety guardrails turned off completely.
Why it matters
These exposed AIs could be hijacked to spread spam, phishing scams, hate speech, or even worse content.
As SentinelOne's Juan Andres Guerrero-Saade put it, there's an "iceberg" of unmonitored systems mixing good and bad uses.
Experts say labs need to get ahead of the risks before things spiral out of control.