Prompt Injection Explained

Prompt injection is an attempt to redirect an AI system through instructions supplied in its input. It becomes especially important when assistants read outside material and can access private information or take actions.

A useful beginner question is simple: does material the assistant is supposed to analyze gain the power to authorize something else?

A safe example

Imagine an assistant whose task is to summarize a fictional supplier document. Part of that document tells the assistant to stop summarizing and send an unrelated private file elsewhere.

The document is task material. It is not the user who authorized the task, and it should not be able to grant access or approve disclosure.

This illustrates indirect prompt injection: the attempted instruction comes through outside content the assistant reads. Direct attempts arrive in the interaction itself. OWASP discusses both forms and recommends layered defenses.

Why connecting tools changes the consequence

A misleading summary can be harmful, but an assistant with access to email, files, or business systems may also be able to perform consequential actions.

Separate the model's text from the application's authority. A generated request to send a message should still face the application's authorization rules. The model should not create its own permissions by producing persuasive text.

Ask which account performs the action, what resources it can access, and which checks happen before execution. Those are application design questions as well as model behavior questions.

Keep untrusted material in its proper role

Use clear structure to distinguish the task, permitted actions, and outside content. That can improve interpretation, but delimiters and warning sentences are not complete security boundaries.

Avoid treating a filter as a guarantee that all harmful instructions will be detected. The application should remain constrained even when input handling or model judgment fails.

For the fictional supplier assistant, the safest design may be to allow only a summary of that document. There is no reason to give it unrelated file access or an unrestricted sending tool.

Put controls around the workflow

Limit access to the data required for the task. Give tools narrow capabilities and validate their parameters. Keep authorization decisions in trusted application code or services.

Require appropriate review for consequential actions. Review needs enough context to be meaningful: the proposed recipient, information, and purpose should be visible before approval.

Log relevant actions without recording unnecessary sensitive content. A record of which tool ran, under which identity, and with what result can support investigation.

These controls reduce specific routes to harm. They do not establish that every possible injection attempt is prevented.

Evaluate with harmless training cases