How the Repeat-After-Me Technique Works
A team of researchers has demonstrated a straightforward method to manipulate six major AI agent frameworks into executing unauthorized tool calls. By prompting the underlying language model to repeat specific commands, they successfully induced LangChain’s LangGraph, Microsoft Agent Framework, Microsoft’s AutoGen, Google’s Agent Development Kit 2.0, and two other widely used systems to perform actions dictated by an attacker. The technique, dubbed „Repeat-After-Me,” relies on no complex exploits or advanced hacking tools—only a simple repetition request that the models follow without question. This vulnerability affects frameworks commonly used in building AI-powered applications, raising immediate concerns about the security of deployed agents in real-world environments.
Breaking news
Canonical’s Web Presence Hit by Second DDoS Assault This Year
Loops AI Secures $3.5 Million to Power a New AI Operating System for E‑Commerce
Winverse AI Secures Tens of Millions in Seed Funding from DeepSeek and Long‑Z
Google Adds Pixel 9 Series to Certified Refurbished StoreThe attack begins when a user inputs a prompt asking the AI model to repeat a string that includes a hidden tool invocation, such as a command to access files, send data, or modify system settings. Because the frameworks are designed to follow the model’s output faithfully, they interpret the repeated text as a legitimate instruction and execute the embedded tool call. The researchers tested this across multiple frameworks and found consistent success, indicating a shared design flaw in how these systems handle model-generated content. Notably, the method does not require jailbreaking the model or bypassing safety filters—it exploits the intended behavior of instruction following. This makes the vulnerability particularly insidious, as it blends in with normal interaction patterns and avoids triggering typical anomaly detection mechanisms.
Why Current Safeguards Fail to Block This Attack
Existing safety measures in AI agent frameworks often focus on filtering malicious inputs or monitoring for known harmful patterns in user queries. However, the Repeat-After-Me technique shifts the threat model: the dangerous payload originates not from the user’s direct input but from the model’s own output, which the system treats as trusted. Since the frameworks assume the model’s generated text is safe to act upon, they do not apply the same scrutiny to repeated phrases as they do to initial prompts. The researchers noted that none of the six frameworks had mechanisms to detect or prevent the execution of tool calls embedded in repeated model outputs, even when those outputs clearly deviated from the user’s original intent. This gap highlights a critical oversight in the trust assumptions underlying agent architectures.
If exploited in deployed applications, this flaw could allow attackers to steal sensitive data, manipulate connected systems, or carry out other malicious actions under the guise of normal AI interaction. For example, an agent designed to assist with email management might be tricked into forwarding messages to an external address after repeating a seemingly innocuous phrase. The researchers warn that as AI agents gain more autonomy and access to tools, the potential impact of such prompt-based manipulations grows. They urge developers to implement output validation, sandbox tool execution, and context-aware monitoring to distinguish between legitimate user-driven actions and model-generated commands that may conceal harmful intent.
Could This Lead to Real-World Harm?
How does the Repeat-After-Me attack differ from traditional prompt injection? Unlike standard prompt injection, which manipulates the model via deceptive user input, this attack uses the model’s own repetition behavior to generate and execute hidden tool calls within the framework’s trusted output stream.
Frequently Asked Questions
Which frameworks were found to be vulnerable? The researchers confirmed the technique worked on LangChain’s LangGraph, Microsoft Agent Framework, Microsoft’s AutoGen, Google’s Agent Development Kit 2.0, and two additional unnamed but widely used AI agent frameworks.
What can developers do to protect against this type of attack? Developers should treat all model-generated output as untrusted, enforce strict tool call validation, isolate agent environments, and monitor for anomalous behavior even when inputs appear benign.
