Prompt injection protection: Detecting and blocking malicious AI instructions

In generative AI, a natural-language interface can create a new security problem: instructions and data can become mixed together. In prompt injection techniques, an attacker tries to place malicious instructions where an AI system may read them, so the model follows the attacker’s goal instead of the user’s or the company’s intended rules.

Author: Alexander Ivanyuk , Senior Director, Technology

Generative AI changes how people work with information. A user can ask a question, upload a document, summarize a ticket, draft an email or ask an AI assistant to help with a workflow. That is useful because the interaction feels natural. But the same natural-language interface also creates a new security problem: instructions and data can become mixed together.

That is the basic idea behind prompt injection. An attacker tries to place malicious instructions where an AI system may read them, so the model follows the attacker’s goal instead of the user’s or the company’s intended rules. Sometimes the malicious instruction is typed directly into the prompt. Sometimes it is hidden inside a document, email, webpage, support ticket, calendar invite, code , metadata or tool response. For a normal application, that text may just be content. For an LLM-powered application, it may become something the model interprets as an instruction.

According to OpenAI , “Robustness to adversarial attacks is a longstanding challenge for machine learning and AI, making this a hard, open problem.” It also asserts that defending against prompt injection is “a challenge across the AI industry” and that adversaries are expected to keep developing attacks.

Anthropic is even more explicit : “Prompt injection is far from a solved problem, particularly as models take more real-world actions.” Anthropic also states: “No browser agent is immune to prompt injection.”

This is why prompt injection protection is becoming a core part of GenAI security. It is not only stopping obvious prompts like “ignore all instructions.” That is the easy example. The harder problem is detecting harmful intent when the attack is subtle, hidden in business content, or designed to manipulate an agent that can use tools and access data.

Why prompt injection is different from ordinary input abuse

Traditional security already knows how to deal with many types of malicious input. SQL injection, command injection and cross-site scripting all involve attacker-controlled input being interpreted in the wrong way. Prompt injection follows a similar high-level pattern, but it works through language and context instead of only code syntax.

That makes it harder to control. A malicious prompt may not look like malware. It may look like a normal instruction, a polite request, a note inside a document or a hidden message inside content that the AI is asked to summarize. The model is not simply parsing a command. It is trying to interpret meaning. That is why a prompt injection attack can be phrased in many different ways while still aiming for the same outcome.

OWASP ranks prompt injection as the first risk in its LLM Application Top 10 for 2025 . The important point is not just that prompt injection exists, but that OWASP separates direct prompt injection, where the user directly enters malicious instructions, from indirect prompt injection, where hostile instructions are placed in external content that the model later processes. That distinction matters a lot for business use because employees often ask GenAI tools to summarize, analyze or act on content they did not write themselves.

For managed service providers (MSPs) and small and medium IT deployments (SMDs), the practical explanation is simple: Prompt injection is not only something a malicious user types into a chatbot. It can arrive through the content the business already handles every day.

Direct and indirect prompt injection

A direct prompt injection happens when someone intentionally gives the AI a harmful or manipulative instruction. The classic example is a user telling the model to ignore safety rules, reveal confidential information or bypass the intended task. These attacks can be crude, but they still matter because users may test limits, employees may misuse tools and attackers may interact with customer-facing AI systems.

An indirect prompt injection is often more dangerous in business settings. In this case, the malicious instruction is not entered by the user as a direct request. It is hidden inside something the AI reads: an email, a webpage, a document, a support ticket, a database field or content retrieved through RAG . The user may simply ask, “Summarize this message,” while the message contains hidden instructions telling the AI to ignore rules, extract sensitive information, or send data somewhere else.

The difference is important because indirect prompt injection attacks the trust relationship between user intent and external content. The user wants the AI to process the content. The attacker wants the AI to obey the content. Without protection, the model may not clearly separate those two roles.

This is why prompt injection becomes more serious as AI becomes more connected. A chatbot with no access to company data can still produce a bad answer. But an AI assistant with access to email, documents, tickets, CRM, file storage or model context protocol (MCP) tools has a much bigger attack surface. If it can read sensitive context and call tools, a malicious instruction hidden in the wrong place can become a data leak or an unsafe action.

Real cases show why this matters

This is not only a theoretical risk. Several public cases already show how prompt injection can affect connected AI systems.

One of the clearest examples is EchoLeak , a zero-click vulnerability reported by Aim Labs in Microsoft 365 Copilot. The researchers said the attack chain could automatically exfiltrate sensitive and proprietary information from Microsoft 365 Copilot context without user awareness and without relying on specific victim behavior. Microsoft mitigated the issue, and public reporting said there was no evidence of real-world exploitation, but the case still matters because it showed how AI connected to enterprise context can become a data-exfiltration path when prompt handling, retrieved data, and output channels interact in the wrong way.

Another useful example is Reprompt , reported by Varonis Threat Labs. The attack used a personal (not 365) Microsoft Copilot URL parameter to automatically populate a prompt from a link. Varonis said the attack required only a single click on a legitimate Microsoft link and could maintain control even when the Copilot chat was closed, allowing session data to be exfiltrated with no further interaction. The case shows a practical lesson: prompt injection does not always look like someone typing into a chat window. It can be delivered through ordinary user behavior such as clicking a link.

Agentic AI makes the same problem more serious. Trail of Bits demonstrated how prompt injection can lead toward remote code execution in AI agents that are allowed to run system commands. Their research focused on agent designs where some commands are treated as safe and allowed automatically, while others require approval. The lesson is simple: once an AI agent can call tools or execute commands, prompt injection is no longer only manipulating text output. It can become a way to misuse the agent’s permitted actions.

These cases point to the same pattern. Prompt injection becomes dangerous when three things come together: the AI can see sensitive or trusted context, it can process untrusted content, and it can communicate outward or take action. That combination is exactly why prompt injection protection needs to sit inside the business workflow, not only inside the model.

Why MSPs and SMDs should care

For MSPs, prompt injection is a customer-environment issue. Customers may not describe it in technical language, but they will feel the risk when AI becomes part of daily work. A customer may use AI to summarize tickets, internal documents, draft replies, review contracts or help with support workflows. Each of those use cases can involve untrusted content and sensitive business data at the same time.

For SMDs, the challenge is even more practical. Smaller teams adopt AI because it saves time. They may not have a dedicated AI security team, and they may not review every workflow in detail. That means employees can start using LLM-powered apps before anyone has defined where prompts are inspected, where sensitive data is blocked, and AI interactions should be logged.

The business does not need to treat every prompt as hostile. But it does need to understand that prompts are now part of the security surface. A prompt can carry sensitive data. A prompt can carry malicious instructions. A prompt can influence a model’s output. And in agentic systems, a prompt can influence which tool is called .

That is why prompt injection protection is not a “nice to have.” It is part of making GenAI safe enough for normal business use.

What prompt injection protection needs to do, and how Acronis helps

Prompt injection protection cannot be just a simple keyword filter. It has to deal with obvious attacks, subtle attacks, and attacks hidden inside normal business content.

The first job is to detect direct instruction override attempts, such as prompts that try to make the model ignore rules, reveal secrets, bypass safeguards, or behave outside the approved purpose. Acronis addresses this with Prompt Injection Protection that can detect potentially harmful prompts, record details in the event log, block those prompts, and notify the end user.

Prompt injections are addressed using a purpose-built engine that analyzes prompts for patterns and techniques commonly associated with attempts to manipulate LLM behavior. Acronis continuously tests and improves this engine against a large corpus of legitimate and potentially harmful prompts to help maintain effective detection while keeping false positives low

The second job is to catch more nuanced attacks where the wording changes but the intent is still malicious. Attackers may frame the prompt as a role play, a fictional scenario, a debugging request, a translation task or a request to test the model’s hidden rules. This is why prompt protection needs more than one detection method. Acronis uses an AI security engine as a high-speed first line of defense, based on discovered attack patterns, keywords, and regular expressions. The same security model also includes additional layers such as NLP-based intent analysis, LLM checks for higher-risk cases, and similarity against known attack patterns.

The third job is to reduce the risk of indirect prompt injection. This is where malicious instructions are hidden in content the AI is asked to process, such as a document, webpage, email, ticket or tool response. This is especially important for MSPs and SMDs because normal business workflows often involve untrusted or semi-trusted content. Acronis treats indirect prompt injection as part of the broader GenAI risk model, because LLMs can be manipulated when they process attacker-controlled external data sources.

The fourth job is to connect prompt protection with data protection. Many prompt injection attacks are dangerous because they try to make the AI expose sensitive information or send it somewhere it should not go. That is why prompt protection and DLP belong together. Acronis GenAI Protection can detect sensitive data sent to AI applications, record details in the event log, block sensitive data from being sent, and notify the user. This gives MSPs and customers both sides of the control: detect malicious instructions and reduce the chance that sensitive data leaves the environment.

The fifth job is to support different enforcement levels. Some customers may want to start with detection and logging so they can understand what is happening. Others may want to block risky prompts immediately. Acronis supports both modes: Prompt injection protection can be configured to detect harmful prompts or block them, while recording the event details in the log and notifying the end user when blocking is applied.

Finally, prompt injection protection needs to fit into the larger GenAI control model. A harmful prompt is not always isolated. It may be part of shadow AI use, sensitive-data exposure, agentic behavior, MCP tool access or abnormal automated activity. Acronis brings prompt protection together with usage monitoring, GenAI management for applications, agents, MCP servers, DLP, detection of denial-of-wallet attacks, dashboard widgets and the Generative AI security report. This makes prompt injection protection part of a managed security workflow rather than a separate filter that is hard for MSPs to operate at scale.

A Swiss company founded in Singapore in 2003, Acronis has 15 offices worldwide and employees in 60+ countries. Acronis Cyber Platform is available in 26 languages in 150 countries and is used by over 21,000 service providers to protect over 750,000 businesses.