AI Prompt Injection

What it is

Prompt injection places attacker-controlled instructions where an AI system should treat content as data. Direct injection comes through a user input; indirect injection appears in retrieved documents, webpages, messages, or other external material. The risk becomes more consequential when the AI can access private records or use tools, rather than only generate text.

How it works

A malicious source tells the model to disregard its task, disclose information, or call a tool under a false pretext. Because natural-language content and instructions can be mixed, the model may follow the hostile text. The attack does not grant privileges by magic: damage depends on what the surrounding system actually permits.

How people get tricked

The AI confuses a source's claims with trusted authority, and the user may trust a fluent answer or action. Instructions hidden in a document do not become legitimate merely because a retrieval system returns them.

Example scenario

A document being summarized contains a demand to send private notes to an external destination. A safe system treats that demand as document content and independently forbids the transfer.

Warning signs

• External material contains instructions aimed at the assistant rather than relevant facts.
• The proposed answer or tool action departs from the user's authorized task.
• Requests involve secrets or destinations unrelated to the source's purpose.

If it happens to you

Stop the unintended action, preserve the source and action logs, and investigate whether data or tools were actually exposed. Revoke affected access if necessary and tighten independent controls. Filtering one phrase does not eliminate the underlying trust-boundary problem.