Low-Cost Data Poisoning Attacks Target Open-Weight AI Models

A cybersecurity researcher demonstrated the feasibility of compromising open-weight AI models by injecting a backdoor for under $100 in less than an hour. By using just ten malicious training examples, the researcher was able to manipulate the model to produce code that is vulnerable to remote code execution. This attack method, known as data poisoning, involves steering the model towards malicious behaviors through targeted adversarial data. The implications of such vulnerabilities are significant, as they highlight the lack of observability in AI systems compared to traditional software. As AI supply chain attacks become more prevalent, the security community is increasingly focused on these risks. Current practices for tracking and mitigating malicious code in software do not adequately apply to AI models, which complicates defense strategies. The researcher’s findings align with previous warnings from the academic community regarding model subversion. The urgency of addressing these vulnerabilities is amplified as open-weight models are increasingly utilized in various applications.

Key Points: • Open-weight AI models can be compromised for under $100 using data poisoning. • Only ten malicious training examples are needed to create vulnerabilities in AI models. • Current practices for securing software do not effectively apply to AI models.