How indirect prompt injection attacks on AI work - and 6 ways to shut them down

Artificial intelligence (AI), and how it could benefit businesses, as well as consumers, is a topic you'll find discussed at every conference or summit this year.

AI tools, powered by large language models (LLMs) that use datasets to perform tasks, answer queries, and generate content, have taken the world by storm. AI is now in everything from our engines to our browsers and mobile apps, and whether we trust it or not, it's here to stay.

Also: These 4 critical AI vulnerabilities are being exploited faster than defenders can respond

Innovation aside, the integration of AI into our everyday applications has opened up new avenues for exploitation and abuse. While the full range of AI-related threats is not yet known, one specific type of attack is causing real concern among developers and defenders -- indirect prompt injection attacks.

They aren't purely hypothetical, either; researchers are now documenting real-world examples of indirect prompt injection attack sources found in the wild.

The LLMs that our AI assistants, chatbots, AI-based browsers, and tools rely on need information to perform tasks on our behalf. This information is gathered from multiple sources, including websites, databases, and external texts.

Indirect prompt injection attacks occur when instructions are hidden in text, such as web content or addresses. If an AI chatbot is linked to services, including email or social media, these malicious prompts could be hidden there, too.

Also: ChatGPT's new Lockdown Mode can stop prompt injection - here's how it works

What makes indirect prompt injection attacks serious is that they don't require user interaction.

An LLM may read and act on a malicious instruction and then display malicious content, including scam website addresses, phishing links, or misinformation. Indirect prompt injection attacks are also commonly linked with data exfiltration and remote code execution, as warned by Microsoft .

A direct prompt injection attack is a more traditional way to compromise a machine or software -- you direct malicious code or instructions to the system itself. In terms of AI, this could mean an attacker crafting a specific prompt to compel ChatGPT or Claude to operate in unintended ways, leading it to perform malicious actions.

Also: Use an AI browser? 5 ways to protect yourself from prompt injections - before it's too late

For example, a vulnerable AI chatbot with safeguards against generating malicious code could be told to respond to queries as a security researcher and then generate this output for "educational purposes." Or, it could be told to "ignore all instructions and..." leading to unintended behavior or data exposure.

Prompt injections may also be used to jailbreak LLMs and bypass developer safeguards.

The OWASP Foundation is a nonprofit that maintains the OWASP Top 10, a popular project that ranks the most prominent security threats to web and related applications.

Also: OpenClaw is a security nightmare - 5 red flags you shouldn't ignore

Threats against LLMs have now grown to have a potentially vast impact on our privacy and security, and as a result, the OWASP Top 10 for Large Language Model Applications project was born.

You'll find that prompt injection attacks are ranked at the top -- both direct and indirect -- as the highest threat to LLM security today.

If you scan Palo Alto Networks' Unit 42 advisory on indirect prompt injection attacks detected in the wild, you will notice that the researchers issued a directive for any LLM scanning the page to not follow any of the instructions listed, and to only treat the post as educational.

This alone can give you an insight into how indirect prompt injection attacks operate in the wild. An LLM scans a web page for authoritative, useful content, and may not be able to distinguish between legitimate content and malicious instructions.

In a recent deep-dive analysis of indirect prompt injection examples found in the wild, researchers at Forcepoint have given us insight into how these attacks are crafted.

Also: How a simple link allowed hackers to bypass Copilot's security guardrails - and what Microsoft did it

To begin with, many indirect prompt injection attempts begin with the following prompts:

There are interesting examples that use more sophisticated instructions that have been found on live websites, including:

As these examples reveal, indirect prompt injection attacks are far more than phishing links. They may become one of the most serious cyber threats online in the future.

The primary defenses against prompt injection attacks include input and output validation and sanitization, implementing human oversight and controls in LLM behavior, adopting the principles of least privilege, and setting up alerts for suspicious behavior. OWASP has published a cheat sheet to help organizations handle these threats.

Also: The biggest AI threats come from within - 12 ways to defend your organization

However, as Google notes, indirect prompt injection attacks aren't just a technical issue you can patch and move on from. Prompt injection attack vectors won't vanish anytime soon, and so companies must continually adapt their defensive tactics.

It's not just organizations that have to take steps to mitigate the risk of compromise from a prompt injection attack. Indirect ones, as they poison the content LLMs pull from, are possibly more dangerous to consumers, as exposure to them could be higher than the risk of an attacker directly targeting the AI chatbot you are using.

Also: Why enterprise AI agents could become the ultimate insider threat

You are at the most risk when a chatbot is being asked to examine external sources, such as for a query online or for an email scan.

I doubt indirect prompt injection attacks will ever be fully eradicated, and so implementing a few basic practices can, at least, reduce the chance of you becoming a victim:

To explore this topic further, check out our guide on using AI-based browsers safely.