paperarXiv (systems, data, ML)·22 Oct 2026·Justin Tan

AgentPhishing: A Benchmark of LLM Email Agents Under Adaptive Attack

AgentPhishing benchmarks LLM email agents under adaptive phishing.

Depth 4/5Score 4.72

Paper (from PDF)

Preprint AGENTPHISHING: A BENCHMARK OF LLM EMAIL AGENTS UNDER ADAPTIVE ATTACK JustinTan,MiaoLi,DingZhao CarnegieMellonUniversity ABSTRACT Computer-use agents increasingly handle private information and act on users’ behalfacrossdesktopandwebapplications,includingemail. Phishingtherefore posesprivacyandsecurityrisksthatdependnotonlyonwhetheragentsrecognize malicious messages, but also on how they respond. Existing phishing datasets weredevelopedprimarilyforconventionaldetection,leavingagentsusceptibilityto persuasivemaliciousrequestsinsufficientlycharacterized.Toexposevulnerabilities thatpre-existingdatasetsmaymiss,weintroduceAgentPhishing,abenchmark forevaluatingwhetherpersuasivephishingattackscanevadedetectionandinduce LLM email agents to comply with malicious requests. We develop PhishOpt, whichoptimizesreusableemail-generationskillsagainstindividualagentsunder fixedbenchmarkconstraintsandarealismconstraint. Weanalyzethegenerated emailsusingCialdini’spersuasionprinciplestoidentifystrategiesassociatedwith agentfailuresandusethesefindingstoconstructauniversalgenerationskill. Our evaluationshowsthatagentscanhandlepre-existingphishingemailssafelyyet remainsusceptibletoattacksadaptedtoagentbehavior,whileinanAgentPhishing evaluation of nine agents, four exhibit phishing-detection rates below 10% and malicious-requestcomplianceratesabove45%. Theseresultshighlightthelimits ofinferringagentsafetyfromconventionalphishingevaluationsandtheneedto assesshowagentsrespondtopersuasivemaliciousrequests. 1 INTRODUCTION Computer-useagentspoweredbylargelanguagemodels(LLMs)andvision-languagemodels(VLMs) offerawaytodelegateeverydaytasksacrossdesktopandwebapplications(OpenAI,2025;Anthropic, 2024;GoogleDeepMind,2025;Zhouetal.,2026;KimiTeam,2026;Xieetal.,2024;Drouinetal., 2024). Tocarryoutthesetasks,agentsinterpretexternalcontentwhileaccessingprivateinformation andtakingactionsonusers’behalf.Adversarieswhoinfluencethiscontentmayredirectthoseactions, creatingprivacyandsecurityrisks(Debenedettietal.,2024;Zhanetal.,2024). Agentsmanaging email provide a concrete setting for studying these risks: unlike conventional phishing detectors thatclassifymessages,theymayalsocarryoutrequestswithinthem,includingrequeststodisclose sensitive information or perform other unauthorized actions (Wu et al., 2026; 1Password, …

Explained

What happened

AgentPhishing (Carnegie Mellon) tests whether LLM email agents can be talked into doing what a phishing email asks, such as sharing private data or taking actions for the attacker. In an evaluation of nine agents, four detected under 10% of the adapted phishing emails and complied with the malicious request over 45% of the time.

The key finding: agents that handled existing phishing datasets safely still failed against emails tuned to their own behaviour. Good scores on old benchmarks don't prove safety.

Why it works

The attack generator, PhishOpt, learns how to fool a specific agent:

Start from an email-generation "skill" (a reusable prompt or recipe for writing phishing emails).
Send generated emails to the target agent and watch what happens: did it flag the email, did it comply?
Use that feedback to refine the skill, under rules that keep emails realistic and forbid cheap tricks like fake system directives or "ignore previous instructions".
Analyse the winning emails with Cialdini's persuasion principles (authority, urgency/scarcity, reciprocity, social proof, liking, commitment) and combine the most effective framings into a universal skill that works across agents.

Note what's excluded: this is pure social engineering, not prompt injection. The emails don't contain hidden commands; they just persuade, like they would a human. Models trained to be helpful and to follow plausible requests turn out to be susceptible to the same levers humans are.

The bigger pattern

Static benchmarks measure yesterday's attacks. Real attackers adapt to the target, the same lesson as fraud detection and spam filtering. For agents, detection and compliance are separate: an agent might not flag an email yet still decline to act, or flag it and act anyway.

What it means in practice

Any agent that reads untrusted input (email, tickets, web pages, PR text) and can act needs guardrails outside the model: allowlisted actions, human confirmation for anything that sends data out, and no access to secrets it doesn't need.
Red-team your own agents adaptively, with an attacker model that learns from your agent's responses, rather than with a fixed list of bad examples.
For any product that moves money or points, the classic attacks (urgent "verify your account", fake partner requests to transfer funds) translate directly to support agents. Design flows so the agent can't move value on the strength of an email alone.