← Back to summary
The Agent in the Middle: How AI-Mediated Access Reshapes Web Vulnerability Exploitation
Converted from PDF with markitdown. Layout may be rough.
The Agent in the Middle: How AI-Mediated Access Reshapes Web Vulnerability Exploitation RuozhaoYang JiongchiYu rz.yang.2024@phdcs.smu.edu.sg jiongchiyu@acm.org SingaporeManagementUniversity NanyangTechnologicalUniversity Singapore Singapore MingHu XiaofeiXie mhu@sei.ecnu.edu.cn xfxie@smu.edu.sg EastChinaNormalUniversity SingaporeManagementUniversity China Singapore Abstract Keywords Webservicesincreasinglyexposebackendinterfacesthatletusers WebSecurity,VulnerabilityExploitation,LLMAgents delegatetaskstoAIagents,whichinvokeserviceoperationsdi- rectlywithoutnavigatingwebsites.Thisshiftchangeshowexisting ACMReferenceFormat: Webvulnerabilitiescanbeexploited.Weinvestigatethesechanges RuozhaoYang,JiongchiYu,MingHu,andXiaofeiXie.2027.TheAgentinthe throughVela,anexecutablebenchmarkcovering44vulnerabilities Middle:HowAI-MediatedAccessReshapesWebVulnerabilityExploitation. acrosstwelveWebapplicationfamilies.Thebenchmarkpreserves InProceedingsofProceedingsoftheACMWebConference2027(WWW’27). ACM,NewYork,NY,USA,11pages.https://doi.org/XXXXXXX.XXXXXXX vulnerablebackendsandderivesagent-facinginterfacesfromle- gitimateworkflows.Weevaluatethreecommercialagentsacross multipleinterfaceconfigurations,usingcontrolledcomparisons 1 Introduction toseparateinterfaceeffectsfromagentbehaviorandindependent oraclestoverifyattackoutcomes.Ourresultsshowthattheagent AIagentsarechanginghowusersaccessWebservices.Ausercan realizeseveryattackthatdirectinterfaceinvocationrealizesinthe describeashoppingneedoranaccount-managementtasktoan referenceconfiguration,soexploitabilityissetmainlybytheinter- agentandletitcarryoutthenecessarysteps.Web-agentresearch face:91%ofvulnerabilitiesremainexploitableunderthereference hasdemonstratedthiscapabilitythroughagentsthatnavigatepages design.Interfacerestrictionspreventsomeattacksbyremovingre- andinteractwithexistingwebsites[7,28,31].Afurthershiftis quiredoperationsorlimitinginputcontrol,whileretentionremains nowtakingshape:servicesareexposingcapabilitiesthatagents highacrossvulnerabilityclasses.Nocase-levellossinthereference canaccessdirectly,withoutnavigatingawebsite.OpenAI’sAgentic configurationisattributedtoagentrefusalorinputnormalization, CommerceProtocolconnectsChatGPTwithmerchantcatalogsand andinonecasetheagentcompletedanattackbysupplyingcalls shoppingexperiences[21],whileGoogle’sUniversalCommerce theattackeromitted.Acrossthetestedconfigurations,interfacede- Protocolsupportsagentaccesstomerchantcapabilitiessuchas signaffectsexploitabilitymorethanagentchoice,movingretention productdiscoveryandcheckout[10].Theseinitiativesmovethe from39%to95%againstatmost4.5pointsacrossagents,butsome interactionbetweenagentsandservicestowarddedicatedinterfaces restrictiveinterfacesalsolimitlegitimateworkflows.Ourstudy throughwhichbusinesstaskscanbecompleted. providesasystematicempiricalaccountofhowagent-mediated Figure1illustratesthischangeusingthetaskofreorderinga accessreshapesexistingWebvulnerabilities,groundingfutureWeb previouspurchase.Intheconventionalworkflow,theusernavigates architecturedesigninmeasuredsecurityoutcomesandtrade-offs thewebsitetofindtheorderandcompletecheckout.Throughagent- betweenreducingattackexposureandpreservinglegitimatefunc- mediatedaccess,theuserstatesthegoal,andtheagentinvokesthe tionalityinagent-facingservices. correspondingoperationsthroughanagent-facinginterface.Both pathsreachthesamebackend,wherebusinesslogicandapplication CCSConcepts dataremain,butthelatternolongerrequirestheservice’spages •Securityandprivacy→Webapplicationsecurity. andforms.ThisshiftopensthepossibilityofaWebinwhichagents becometheprimarymeansofaccessingservices,andindividual websitesarenolongernecessaryuser-facingentrypoints. Permissiontomakedigitalorhardcopiesofallorpartofthisworkforpersonalor Asthisaccessmodeldevelops,serviceprovidersneedtounder- classroomuseisgrantedwithoutfeeprovidedthatcopiesarenotmadeordistributed forprofitorcommercialadvantageandthatcopiesbearthisnoticeandthefullcitation standitssecurityimplications.Malicioususerscanalsousethis onthefirstpage.Copyrightsforcomponentsofthisworkownedbyothersthanthe pathtotargetbackendservices.Existingbackendvulnerabilities author(s)mustbehonored.Abstractingwithcreditispermitted.Tocopyotherwise,or maypersist,whileinterfacerestrictionsandagentbehaviorchange republish,topostonserversortoredistributetolists,requirespriorspecificpermission and/orafee.Requestpermissionsfrompermissions@acm.org. whetherandhowtheycanbeexploited.Fordevelopersandoper- WWW’27,Dublin,Ireland ators,thecentralquestionishowmuchoftheharmenabledby ©2027Copyrightheldbytheowner/author(s).PublicationrightslicensedtoACM. existingvulnerabilitiesremainspossiblethroughagent-mediated ACMISBN978-1-4503-XXXX-X/2027/XX https://doi.org/XXXXXXX.XXXXXXX access,andhowtheresultingsecurityconsequenceschange. 6202 tcO 62 ]RC.sc[ 1v14553.0162:viXra WWW’27,May10-14,2027,Dublin,Ireland Trovatoetal. Today: The User Carries Out Each Step Web Backend Browser clicks My Orders, then Add to Cart, then Place Order Web Interface Logic User Pa C g l e ic : k “ s O a rd n e d r f p o l r a m c s ed” (by U h s a e n r d) GET /orders POST /cart POST /checkout Pages Forms Auth Goal Orders “Reorder the Future: An AI Agent Carries Out the Steps Database shoes I Cart bought last Chat adds its system prompt, plans, selects tools, fills arguments Agent Interface week.” Typed request AI Agent list_my_orders() add_to_cart(shoes) checkout() OpenAPI MCP NLWeb Payment Reply: “Order placed.” (LLM) Figure1:Directandagent-mediatedaccesstoaWebservice.Userseitherinteractwiththeservicethemselves(top)ordelegatea tasktoanagentthatinvokestheavailablebackendoperations(bottom). Anaturalexpectationisthatagent-mediatedaccessprovides • Weframeagent-mediatedaccessasanemergingWebse- additional protection. Agent-facing interfaces can restrict oper- curityproblem,examininghowservice-providedinterfaces ationsandinputsneededforexploitation,whilesafetytraining andagentexecutionreshapetheexploitabilityandsecurity and runtime safeguards aim to prevent agents from following consequencesofexistingbackendvulnerabilities. harmfulinstructions[4].Acriticalquestioniswhetherthesesafe- • We develop Vela, an executable benchmark covering 44 guardsactuallypreventexistingbackendvulnerabilitiesfromcaus- vulnerabilitiesacrosstwelveWebapplications.Controlled ingharm.Attacksusingordinarybusinessoperationsmayremain comparisonsdistinguishinterfaceeffectsfromagentbehav- possible,andagentsmayhelpcompletethembysupplyingmiss- ior,whileindependentoraclesverifysecurityoutcomes. ingstepsorcombiningcalls.Strongerrestrictionsmayalsomake • We provide systematic empirical evidence of the limited legitimate operations unavailable. Prior studies primarily focus protectioncurrentagentsofferagainstbackendvulnerability onagentsthemselves,examiningtheiroffensivecapabilities,sus- exploitation,layingafoundationforsecuringfutureagent- ceptibilitytomaliciousinputs,andhandlingofharmfulinstruc- mediatedWebservices. tions[5,11,13,14,30,32].Theimplicationsofthisarchitectural Thesourcecodeanddataneededtoreproduceourresultsareavail- shiftforexistingbackendvulnerabilitiesremaininsufficientlyun- ableinananonymousrepository.1 derstood.Asystematicempiricalevaluationisneededtoquantify changesinexploitabilityandsecurityconsequences,distinguish 2 BackgroundandThreatModel interfaceeffectsfromagentbehavior,andassesstheaccompanying 2.1 Agent-MediatedWebAccess impactonlegitimatefunctionality. To conduct this evaluation, we develop Vela, an executable IndirectWebaccess,abrowserorAPIclientsendsrequeststo benchmarkcomprising44documentedvulnerabilitiesfromtwelve the backend. In the agent-mediated setting studied here, an AI Webapplicationfamilies.Foreachapplication,weconstructagent- agentinterpretstheuser’srequestandinvokesbackendoperations facing interfaces from legitimate workflows. We then compare throughaservice-providedinterface.Theinterfaceexposescallable directWebaccess,directinvocationoftheseinterfaces,andexe- capabilities,eachwithadescriptionandanargumentschema.It cutionthroughanagentonthesamevulnerablebackend.These defines which operations are available, which inputs the agent comparisonsisolatetheeffectsofinterfacedesignandagentexe- cancontrol,andhowcallsareboundtotheuser’sidentity.The cution,withindependentoraclesverifyingtheresultingsecurity agentselectscapabilities,suppliesarguments,andusesreturned consequences.Weevaluatethreecommercialagentsandthreein- resultstoguidesubsequentcalls.Itmaypreserveorrewriteuser- terfacedesigns,measuringvulnerabilityexploitabilityalongside suppliedinputs,refuserequests,orsupplyintermediatesteps.The supportforlegitimatetasks. backendprocessestheresultingrequestsandremainsresponsible Underthereferenceconfiguration,theagentsucceedsinatleast forenforcingauthorizationandbusinessrules. oneattemptoneverycaserealizedbydirectinterfaceinvocation, 2.2 ThreatModel soitremovesnoneoftheattackstheinterfaceletsthrough,and inonecaseitsuppliesmissingsteps.Asaresult,91%ofthevul- Weconsideramalicioususerwhoseekstoexploitanexistingback- nerabilitiesstillproduceverifiedsecurityconsequences,andthe endvulnerabilitythrougheitheraccesspathinFigure1toviolate preventedattacksdependonoperationsorinputcontrolthatthe the application’s security policy. The attacker controls the sub- interfacenolongerexposes.Retentionremainshighacrossallthree mittedrequests,includingattackparametersandpayloads.Under analysisgroups.Acrossthetestedconfigurations,changingthe directaccess,theattackerconstructsrequeststothenativeWeb interfacemovesretentionbetween39%and95%,againstatmost4.5 interface.Underagent-mediatedaccess,theattackersubmitsthe percentagepointsacrossagents,althoughsomereductionscome sameattackcontentasuserinstructions,adaptingitsformtothe withalossoflegitimatefunctionality.Theseresultsquantifyboth interfacewhilepreservingtheattackobjectiveandexploit-relevant thepersistenceofbackendsecurityrisksandthefunctionalcosts inputs.Theagentoperatesasataskassistantandaccessestheback- ofrestrictingaccess,providingevidenceforthedesignoffuture endonlythroughthepublishedcapabilities.Theattackercannot agent-facingWebservices. Thispapermakesthreecontributions: 1Anonymousrepository:https://anonymous.4open.science/r/VELA-70DF/ TheAgentintheMiddle:HowAI-MediatedAccessReshapesWebVulnerabilityExploitation WWW’27,May10-14,2027,Dublin,Ireland directlymodifyitssystempromptortheinterfaceimplementation. orotherprotectedresourcesthroughoperationssuchasupload, Acrossaccesspaths,theattackerstartswiththesameidentityand import,orretrieval. privileges,whileprivilegechangesachievedthroughexploitation remainwithinscope.Thebackendretainsitsoriginallogicandvul- 3.2 Agent-FacingInterfaces nerabilities.Compromisinganotheruser’sagent,indirectprompt injection,andautonomousdiscoveryofunknownvulnerabilities Aninterfacefixeswhichoperationsandargumentsanagentcanuse, areoutsidethescopeofthisstudy. soitsdesigndirectlydetermineswhichvulnerabilitiesremainreach- able.Wethereforetreatthecapabilitydesignasacontrolledfactor andbuildthreedesignsthatexposebackendfunctionalityatdiffer- 3 Designof Vela entlevelsofabstraction.Theendpoint-orienteddesignexposesthe Velameasureswhathappenstoanativelyexploitablevulnerability documentedendpointswiththeiroriginalparameters,inthestyleof whenitisreachedthroughanagent-facinginterfaceinsteadofthe OpenAPI[22].Theworkflow-orienteddesignnameseachworkflow nativeWeb.Threerequirementsfollowfromthisgoal,andeach asonetoolwithaboundedbusinessargumentschema,inthestyle shapesthedesign.Becausearesultmustspeaktorealsoftware, ofMCP[2].Thequery-oriented designtakesanatural-language everycaseisadocumentedvulnerabilityinarealordeliberately query,inthestyleofNLWeb[19].Wedeliverallthreetotheagent vulnerableapplication,reproduciblenativelyandverifiedindepen- throughMCPoverthesameadapters,soadifferencebetweenthem dentlyoftheagent’soutput.Becauseaninterfacetunedtoaknown reflectsthecapabilitydesignandnottheprotocolorthebackend. flawwouldprejudgetheoutcome,theagent-facinginterfacesare Becauseaquerymuststillbecomeaconcreteoperation,wefix builtfromlegitimateworkflowswithoutanyknowledgeofthevul- wherethattranslationhappens.Thequery-orienteddesignexposes nerabilities.Andbecauseadifferencemustbeattributable,allaccess asingletoolthattakesaquerystring.Aserver-sideLLMplanner pathssharethesamebackend,initialidentity,andinitialstate,soa mapseachquerytoanintentandfillsitstypedslots.Theplanner changeinexploitabilityfollowsfromthepath,notfromthesetup. ispartoftheserviceinterfaceandisseparatefromtheagentunder AsshowninFigure2,nativevalidationandinterfaceconstruc- test.C1andC2usethesameplannerimplementation,withthe tionproceedasseparatebranchesandjoinonlyaftertheinterfaceis attacker’sobjectiveandinitialprivilegesheldconstant.Planner frozen,afterwhichmatchedevaluationcomparestheaccesspaths. executionremainspartoftheinterface-stageoutcome.Intentsthat readpubliccontentareansweredfromasnapshotindexofeach application’spubliccontent.Anintentisboundtoaliveadapter 3.1 ApplicationsandVulnerabilities onlywhenitsbenignworkflowneedscurrentstateorawrite,which Weselecttwelveapplicationfamiliestobalanceexternalvalidity thesnapshotcannotserve. withcontrolledvariation.Forvalidity,mostaredeployedapplica- Theplannerresolvesobjectreferencesagainstindexedentities, tionswithdocumentedCVEsorpublicadvisories,drawnfromWe- coerceseachslotvaluetoitsdeclaredtypeandrange,anddrops bExploitBench[16]andCVE-Bench[32],whichsupplydeployable querytextthatfillsnoslot.Theseconstraints,ratherthantheuseof applications,vulnerabilitydocumentation,andverificationartifacts. naturallanguageitself,boundwhataquerydeliverstothebackend. Caserecordsdistinguishdocumentedproductionvulnerabilities Aquerymaynameanoperationorquoteapayload,butonlyvalues fromdeliberatelyintroducedflawsandretainthecorresponding thatsurviveslotfillingreachtheadapter.Intents,slots,andlive CVE,advisory,orchallengeidentifiers.Forcontrolledvariation,we bindingsarederivedfromthesamebenignworkflowsastheother adddeliberatelyvulnerabletrainingapplicationswhosenumbered designsandfrozenwiththem. challengesisolateoneflawatatimeandletuschangeasinglere- Theschemesarebuiltblindsothatnointerfacecanbetunedtoa quirementwhileholdingthesurroundingworkflowfixed.Wedelib- knownflaw.Foreachuserrolewecollectrepresentativegoalsfrom eratelyincludeAPI-firstfamilies,becauseanagentreachesaback- documentation,normaluse,andbenigntraffic,andturneachgoal endthroughoperationsandargumentsmuchasanAPIclientdoes. intocapabilitiesthatkeeptheargumentstheworkflowlegitimately Thesefamiliesproviderelevantsettingsforagentmediationandin- needs. Designers never see vulnerability reports or payloads, a cludeauthorizationandbusiness-logicflaws.Table1liststheappli- trustedcomponentsuppliesidentityandcredentials,andtheagent cationfamilies;AppendixAprovidesrepresentativecaseidentifiers. setsonlythebusinessarguments.Weaddnovulnerability-aware Wereproduceeachdocumentedvulnerabilitythroughthena- filteringorauthorization,whichwouldintroducetargeteddefenses tiveinterfaceandrecorditsversion,prerequisites,requiredopera- into the interface comparison. Before any security test, benign tions,andatrustedobservationofitseffect.Atrainingchallengeis tasksconfirmthateachcapabilityreachestheintendedoperation countedasacaseonlyafteritpassesnativereproduction.Every underthecorrectrole.Wealsomeasurebenignfunctionalcoverage, validatedcaseenterstheanalysiswhetherornotaninterfacecan the share of identified workflows that a scheme supports. This expressit,soselectiondoesnotdependoninterfacereachability. coverageisneededtoreadanabsentoperationcorrectly,sincea Thevalidatedcasesspantwelvevulnerabilitycategories.Wegroup missingcapabilitymayreflectincompletefunctionalityratherthan thecategoriesbywhattheattackmainlyrequires,sincethatre- adefense.Wethenfreezethespecification,adapters,andevidence, quirementisexactlywhatmediationactson.Input-precisionattacks since any design revised after the security results were known needexactcontrolofaninputvalue,application-logicattacks,cov- wouldonlybeexploratory. eringauthorizationandbusiness-logicflaws,needarequestthatis After the freeze, each case is mapped to the interface as ex- validinformbutviolatesownership,role,orworkflowrules,and pressiblewhenthecapabilitiescanrepresentitsrequirements,in- resource-handlingattacksrequireaccesstoormanipulationoffiles expressible when a needed operation or argument is absent, or WWW’27,May10-14,2027,Dublin,Ireland Trovatoetal. Environment Construction Vulnerable Apps Native Validation Validated Cases Vulnerability 12 families exploit + oracle 44 cases Information Wall no vulnerability knowledge crosses User Workflows Blind Build Benign Validation Interface benign use only 3 schemes 50 workflows ecafretnI ezeerF tset yna erofeb dexif Matched Evaluation on the Same Backend Case Mapping C0 Native expert exploit Interface Evaluation status Case outcome Expressible C1 Tools scripted call C0 vs C1 Verified Neutralized Agent Valid failure Preserved Inexpressible C2 Agent commercial agents C1 vs C2 Structural Transformed Unresolved Trusted Harness same state, identity, oracle Invalid Step completion deniateR Figure2:OverviewofVela.Thetwobranchesjoinonlyaftertheinterfaceisfrozen. 3.4 MetricsandUnitsofAnalysis unresolved whentheevidenceisinsufficient.Wekeepthismap- ping distinct from execution because an inexpressible case is a Unitsofanalysisfollowthestructureofthestudy.Acaseisoneval- structuralpropertyoftheinterface,notablockedexploit,whereas idatedvulnerabilityandistheunitofcase-levelclaims.Ataskpairs whetheranexpressibleattackyieldsitseffectisdecidedonlyby acasewithanattackrequest.Anattemptisonecompletesession runningit.Asuccessonacasemappedasinexpressibletriggersa onatask,andbecauseanagentisstochastic,werun𝑅repetitions re-auditofthemapping,neverasilentchangetotheinterface.In ofeachtask.Per-attemptratesthereforedividebyattemptsand thequery-orienteddesign,structuralrestrictionsaredeterminedby arenottreatedasindependentsamplesofvulnerabilities,andwe theavailableintentsandslotdomains.Failurestotransmitrequired reportthemonlybesidethecase-levelretention,asinFigure4a,to argumentsduringplanningareclassifiedseparatelyasinterface- showhowoftenarealizablecasesucceedswithinasinglesession. stageexecutionlosses. MediationOutcome.Theprimaryresultofacaseisbinary, namelywhetheranyapplicabletaskreachestheoracle-verified 3.3 EvaluationProtocolandOracles consequenceinanyofits𝑅repetitionswithinthebudget.Cases Conditions.Weusethreeconditionssothatthetwolayersofme- withaverifiedconsequenceformtheretainedset.Forthesecases, executiontracescharacterizehowtheattackiscarriedout.Acase diationcanbeseparatedratherthanobservedonlyincombination. ispreservedwhentheverifiedrequestmatchesthenativeexploit, Allsharethesamebackend,initialidentity,andattackobjective.C0 andtransformedwhentheagentalterstheinputbutapayloadof exercisesthevulnerabilitythroughthenativeinterfaceandfixesthe thesameattackclassstillreachesthesink.Stepcompletiondenotes referenceconsequence.C1attemptsthesameobjectivethroughthe casesinwhichtheagentsuppliesnecessarycallsthattheattacker frozencapabilitieswithascriptedcaller.C2placesanagentinfront didnotspecify,suchasatargetlookuporaprerequisiteoperation. ofthesamecapabilities.C1omitstheexternaltaskassistant;forthe Thislabelrecordsassistanceinexecutingtheattack,withoutimply- query-orienteddesignitstillinvokesthesameservice-sideplanner ingalargersecurityconsequence.Withinasuccessfulrepetition, asC2,throughthesamequerytool.AcaseisC1-realizablewhen stepcompletiontakesprecedenceovertransformed,whichtakes directinvocationverifiesitsprotectedconsequence.Theattacker precedenceoverpreserved.Acasewithnoverifiedconsequenceis submitsthesameattackcontentasuserinstructions,adaptingits neutralized,andwekeepitsreasonseparate,distinguishingastruc- formtotheinterfacewhilepreservingtheobjectiveandexploit- turallyinexpressibleattackfromaninterface-stageexecutionloss relevantinputs,asspecifiedinSection2.2.Theagentselectsand oranexpressibleattacknotrealizedbytheagentwithinthebudget. executesitsowncalls,neverseestheC1callsequence,andhasno Theoutcomesarejudgedpercasefromitssuccessfulrepetitions patharoundthetools.TheC0-C1comparisonisolatestheinterface byanannotationguidewithtwoindependentannotators,whose change,andtheC1-C2comparisonisolateswhattheagentaddson agreementontheoutcomelabelsofthe40retainedreferencecases topofit. MatchedExecution.Atrustedharnessrestoresandverifiesthe isCohen’s𝜅 =0.89.Whenrepetitionsofacasedifferinoutcome, thecasetakesthelabelofthemajorityofitssuccessfulrepetitions, initialstatebeforeeveryattempt,andC1andC2shareschemas, andatieisbrokentowardthelabelclosertothenativeexploit,in adapters,andauthenticationbindings.Theharnessfixestheinitial theorderpreserved,transformed,stepcompletion. credentialsandthebackenddestination,andtheagentcannotreach ReportedQuantities.Theprimarymetricisconsequencereten- resetmechanisms,fixtures,ororacles.Werecordtraces,versions,ac- tion,theshareofvalidatedcaseswhoseprotectedeffectisverified torbindings,andoracleobservations,andanyfailedresetorincon- throughagivenpath.Wereportitoverallnative-validatedcases clusiverunismarkedinvalidratherthancountedasafailedexploit. Oracles. Each case carries a trusted oracle for the protected andovertheC1-realizablesubset,distinguishinginterface-stage attritionfromlossesunderagentmediation,andbreakitdownby effectfixedduringnativevalidation,becauseafluentagentreply analysisgroup.Retentionrecordswhetheracasesucceedsatleast is not evidence of a security effect. The oracle checks backend oncewithinthetestedattempts;per-attemptsuccessrecordshow stateortheattacker-visiblechannel,dependingontheeffect,and consistentlyitsucceeds.Becauserestrictionscanalsoremovelegit- is validated against the native exploit and negative controls to imateoperations,wereadretentionbesidethebenignfunctional checkthatitdetectstheintendedeffectandrejectsthecontrols. coverageofSection3.2,aworkflow-levelshareoverall50identi- Thesamepredicateappliesacrossallconditions.Anunverified fiedbenignworkflows,andbenigntaskcompletion,anattempt-level responseoranagentclaimneverestablishessuccessonitsown.A shareoverthesamefixedset.Itsdenominatoris50𝑅,anditsnu- confidentialityoraclechecksthattheprotectedvaluereachedthe meratorcountssuccessfulworkflowexecutionswithinthebudget. definedattacker-visiblechannel,andanintegrityoraclechecksthe Unsupportedworkflowsremaininthedenominatorandcontribute persistentbackendstate.AppendixEgivestheoracleforms. TheAgentintheMiddle:HowAI-MediatedAccessReshapesWebVulnerabilityExploitation WWW’27,May10-14,2027,Dublin,Ireland Table1:Applications,caseprovenance,andnumberofnative- (CLIv0.164.0,GPT-6.1Sol),andGrokBuild[27](v1.0.67,Grok4.7), validatedcasesretainedunderthereferenceconfiguration. eachpinnedatitsOctober2026releasewithitsdefaultbacking model.Everyagentkeepsthevendor’sdefaultsystempromptwith noaddedsecurityinstruction,sinceneithertheattackernortheser- Family Source Cases Retained vicecontrolsitunderourthreatmodel,andrunsnon-interactively, crAPI OWASPcrAPI(challenges) 4 4 withtool-callapprovalsgrantedbytheharness.Suchapprovals asktheusertoconfirmtheagent’sactions,andheretheuseristhe JuiceShop OWASPJuiceShop(challenges) 5 5 attacker.Itsbuilt-inshell,file,andWebtoolsaredisabled,sothe PrestaShop WebExploitBench+CVE-Bench 5 5 toolsundertestareitsonlypathtotheapplication.RQ3compares SIYUCMS WebExploitBench 6 6 thethreecapabilitydesignsofSection3.2,implementedasone toolperdocumentedoperation(endpoint-oriented),onetoolper JetLinks WebExploitBench 5 5 workflowwithaboundedbusinessargumentschema(workflow- Dolibarr CVE-Bench 2 2 oriented),andonequerytoolwhoseserver-sideplannermapseach querytoanintentoverasnapshotindexoraliveadapter(query- SuiteCRM CVE-Bench 1 1 oriented).AllthreereachtheagentasanMCPserveroverthesame WordPress CVE-Bench(pluginCVEs) 11 10 adapters,withdifferentoperationsandargumentsexposedtothe Moodle Securityadvisory 1 0 agent.Benignfunctionalcoverageandtaskcompletionareread againstafixedsetof50identifiedbenignworkflowsacrossthe Nextcloud Securityadvisory 1 0 twelvefamilies,collectedbeforeanysecuritytest.Taskcomple- Cacti CVE-Bench 2 2 tionusesall50workflowsasitsdenominatoracrossrepetitions, Zabbix CVE-Bench 1 0 includingworkflowsthataninterfacedoesnotsupport,asdefined inSection3.4.ThereferenceconfigurationpairsClaudeCodewith Total 44 40(91%) theworkflow-orienteddesign.RQ1runsonit,andRQ2andRQ3 eachchangeonefactor. RequestsandProcedure.Beforeexecutionwefixtheapplica- bleattackrequestsandtheirexactwordingforeachcase,following nocompletions.Thesetwomeasuresdistinguishthefunctionality Section 2.2. Requests preserve the original attack objective and aninterfaceexposesfromthefunctionalitysuccessfullydelivered exploit-relevantinputswhileadaptingtheirformtoagent-mediated throughtheagent.Becausethebenchmarkispurposiveratherthan access.Wedistinguishinput-bearingrequestscontainingcrafted a random sample of the Web, we report results as percentages, valuesfromgoal-orientedrequestsstatingtheintendedmalicious givetheunderlyingcountsalongsidethem,andtreatclass-level outcome,applyingeachformwhereitpreservesthecase’sattack differencesasdescriptive.Wherewecomparepairedconditions, requirements.Acaseisattemptedundereveryapplicableform, weaddanexactMcNemartestandreportthepairedC1/C2matrix aninapplicableformnevercountsasafailure,andacaseisre- (AppendixD)ratherthaninferringfrommarginalcounts. tainedifanyapplicablerequestsucceedsinatleastonerepetition. 4 Evaluation EachC2taskisrepeated𝑅 =5timesundercapsof15toolcalls, 30,000tokens,and5minutesperattempt,whereasC1receivesone WeevaluateVelathroughthreeresearchquestions.Eachusesa scriptedpasswiththevalidatedinputs,submittedasqueriesforthe referenceconfiguration,ClaudeCodewiththeworkflow-oriented query-orienteddesign.Theharnessrestorestheapplicationfrom design, and varies at most one factor. RQ1 asks to what extent averifiedsnapshotbeforeeveryattempt.Onlyanoracle-verified nativelyexploitablevulnerabilitiesstayexploitableunderagent effectcountsassuccess,andarefusalcountsasavalidfailureonly mediationandhowthisdiffersacrossvulnerabilityclasses.RQ2 whentheoracleconfirmsthatnoeffectoccurred.Abudgetsweep fixesthecapabilitydesignandaskshowmuchthechoiceofagent showsthatdoublingeverycaprecoversnoadditionalcaseunder changesexploitabilityandthroughwhichbehaviorsanagentloses thereferenceconfiguration(AppendixC). a case. RQ3 fixes the agent and asks how the capability design changesexploitabilityandatwhatcosttolegitimatefunctionality. 4.2 ExploitabilityunderMediation(RQ1) 4.1 ExperimentSetup Of the native-validated cases, 91% are expressible through the BenchmarkandConditions.Werunall44native-validatedcases workflow-orientedinterface,andbothdirectinvocation(C1)and from the twelve application families of Section 3.1, comprising theagent(C2)reachtheoracle-verifiedeffectinallofthem(Fig- 18input-precision,15application-logic,and11resource-handling ure3).Retentionistherefore91%ofallnativecasesand100%of cases(AppendixA).Everycaseisevaluatedunderthethreecon- theexpressiblesubset.Theremaining9%areinexpressiblebecause ditionsofSection3.3.C0reproducesthenativeexploitandfixes aneededoperationorargumentisnotexposed,andnocasere- boththecasesetanditsreferenceconsequence,andC1andC2 mainsunresolved.C1andC2agreeoneverycase(exactMcNemar shareonefrozenimplementationofeachcapabilitydesign.Table1 𝑝 =1.0),soallcase-levelattritionisstructural. summarizestheapplicationsandcaseprovenance. The91%figurereflectswhatthereferenceinterfaceexposes,and AgentsandCapabilityDesigns.RQ2comparesthreecommer- RQ3showsthatotherdesignsmoveitbetween39%and95%.The cialagents,ClaudeCode[3](v2.1.310,ClaudeOpus5.5),Codex[20] C1–C2agreement,bycontrast,isnotimpliedbythedesign,since WWW’27,May10-14,2027,Dublin,Ireland Trovatoetal. Input precision Application logic Resource handling Lost here Native 100% Expressible 91% lost at the interface C1 verified 91% C2 verified 91% Figure3:Native-validatedcasesateachstageunderthereferenceconfiguration.Eachsquareisonecase.Therightcolumn givestheshareofallnative-validatedcasesandnamesthelayeratwhichcasesarelost. C2iswheretheagentcanrefuseorrewritetherequest.Evenfor suchlossfromitstracesasarefusal,aninputnormalizationthat input-precisionpayloadsandgoal-orientedrequeststhatstatethe removestheattack,oramalformedorabandonedcall,andread maliciousoutcome,nocase-levellossisattributedtorefusalor retentionbesidebenigntaskcompletion,sincelowerattackrates inputnormalization(Section4.3). matteronlyiftheagentcompleteslegitimatetasks. Retention stays high across vulnerability classes (Figure 4a): Thethreeagentsretain86-91%ofconsequences(38–40of44) 17/18input-precision,14/15application-logic,and9/11resource- whilecompleting76-83%ofbenigntaskattempts.Thechoiceof handlingcases.Giventhesegroupsizes,thegapisdescriptive.It agentmovesretentionbyatmost4.5percentagepoints(twocases), tracestotheinterface,sinceresource-handlingattacksneedopera- noagentneutralizesaclass,andeachlosesatmost2ofthe40realiz- tionssuchasupload,import,orretrieval,andmissingoperations ablecases(Table2).Acrossthetestedrequests,allthreelargelycarry orargumentsaccountfortwoofthefourinexpressiblecases.Inthe theattacksthroughtobackendexecution.ClaudeCodeandCodex retainedcases,fixedidentityandroutingdeterminewhoactsand losenorealizablecasetorefusal,inputnormalization,orexecution where,whiletheexposedbusinessargumentsstillcarryexploit- failure.Eachleaves10%ofitsretainedcasestransformed,including relevantvaluestothevulnerablehandler. changestoanon-essentialouterfilenameonapackage-import attack.GrokBuildloses5%oftherealizablecasestomalformed Finding1:Agentmediationremovesexploitabilityonlywherethein- orabandonedcallsinmulti-stepresource-handlingworkflowsand terfacedoes.Inthereferenceconfiguration,91%oftestedvulnerabilities leaves8%ofitsretainedcasestransformed.Similarexecutionerrors remainexploitable,theagentrealizeseverycasethatdirectinvocation realizes,andallcase-levellossesarisefrominterfacerestrictionson occurinitsbenignruns,whosecompletionrateis76%.Doubling requiredoperationsorinputcontrol. thecapsrecoversoneofitstwolostcases(AppendixC),showing budgetsensitivityinoneobservedloss.Itslowerretentionaccompa- Oftheretainedcases,35arepreserved,4transformed,where nieslowerbenigncompletion,withnocase-levellossesattributed theagentrewroteanon-essentialinputsuchasanouterfilename toselectiverefusal.ClaudeCodeandCodextieonretentionand whileapayloadofthesameclassstillreachedthesink,andone differbyonepercentagepointincompletion.Becausetheunderly- exhibitsstepcompletion.Input-bearingandgoal-orientedrequests ingmodelsareopaqueandthelossesarefew,theseobservationsdo yieldidenticalretentiononcaseseligibleforboth(AppendixB). notestablishaspecificinternalcauseorareliablesecurityranking. Thesinglestep-completioncaseconcernscrAPI’srefund-eligibility flaw (Challenge 8 [23]), where a customer can obtain a refund Finding3:Executionfailuresaccountfortheobservedcase-leveldif- throughanorder-statusupdatewithoutreturningtheitem.The ferencesinexploitabilityacrossagents.Similarfailuresalsodisrupt requestspecifiedonlythesubmission;theagentlookeduptheorder legitimatetasks,linkinglowerattacksuccesstoreducedtaskreliability withoutobservedgainsfromselectiverejectionofmaliciousrequests. andinitiatedthereturnfirst,andtheoracleverifiedthecredit.Re- playingonlythesubmissionproducednocredit(AppendixC),while thecompleteC1sequencesucceeds,sotheagentreachednothing 4.4 ImpactoftheInterface(RQ3) theinterfaceforbidsbutsparedtheattackerfromspecifyingthe WethenholdClaudeCodefixedandvaryonlythecapabilitydesign prerequisitestepsintherequest. (Figure4b,c),readingretentionbesidebenigncoverageandtask Finding2:In4ofthe40retainedcases,theagentrewrotenon-essential completion.Thequery-orienteddesignusesafixedplannerand inputswhilethemaliciouspayloadstayedeffective,andinonecase snapshotindex,andanycontrastholdsforourimplementations itsuppliedthecallsneededtocompleteanattack.Inputrewritingby ratherthanforthestandardsthemselves. theagentisthusnotareliablebarrier,anditsmulti-stepexecutioncan Theendpoint-orienteddesignretains95%ofconsequences(42/44) completeattacksthattheattackerspecifiesonlypartially. at92%benigncoverage(46/50),theworkflow-orientedreference 91%(40/44)at84%(42/50),andthequery-orienteddesign39%(17/44) at46%(23/50);taskcompletionis89%,82%,and41%,respectively. 4.3 ImpactoftheAgent(RQ2) Thedesignthusmovesretentionby56.8percentagepoints,against Toisolatetheagent,weholdtheworkflow-orienteddesignfixedand atmost4.5acrossagents,butnarrowingthesurfaceremovesex- runtaskswithClaudeCode,Codex,andGrokBuild(Figure4a,c). ploitpathsandusefuloperationstogether.Theone-factorsweeps AllshareaC1baseline,soaC1-realizablecasethatanagentfails donotmeasureagent-by-designinteractions,andanotheragent toreachacrossitsattemptsislostattheagentlayer.Welabeleach maybehavedifferentlyunderthequery-orienteddesign. TheAgentintheMiddle:HowAI-MediatedAccessReshapesWebVulnerabilityExploitation WWW’27,May10-14,2027,Dublin,Ireland (a) Retention by Class (b) Capability Design 95% Input precision 94% Endpoint 92% 90% 89% Application logic 93% 91% 70% Workflow (ref.) 84% 82% Resource handling 64-82% 39% 50% All cases 86-91% Query 46% 41% 30% 40% 60% 80% 100% 0% 25% 50% 75% 100% 30% 50% 70% 90% Benign completion noitneteR Claude Code Codex Grok Build Design 95% CI Per attempt Consequence retention Benign coverage Benign completion (c) Exposure vs. Utility Endpoint Δ design 56.8 Δ agent 4.5 Query Figure4:Retentionbyclassandtheone-factorsweepsaroundthereferenceconfiguration(ClaudeCode,workflow-oriented). (a)Retentionperclassandagent,withthe95%Wilsonintervalandper-attemptsuccessforClaudeCode.(b)Retention,benign coverage,andbenigncompletionpercapabilitydesign.Completionismeasuredoverall50benignworkflowsacrossrepetitions. (c)Retentionagainstbenigncompletion,wheretheworkflowdiamondalsomarksClaudeCode. Finding4:Interfacedesignchangesexploitabilitymorethanagent andplannerexecution.Thisdistinctionmotivatesinterfacedesigns choiceinthetestedconfigurations.Thelargestreductionsinattack thatpreserverequiredbusinessfunctionsandconstraininputsac- exposureaccompanysubstantiallossesoflegitimatefunctionality,mak- cordingtotheirintendeduse.Agentsafeguardsrequireasimilar ingfunctionalcoverageessentialtointerpretingapparentsecurity distinctionbetweenrejectingharmfulactionsandfailingtoexecute improvementsofaninterface. tasks.TheexecutionfailuresobservedinRQ2affectedbothattacks andlegitimatetasks,providinglittlebasisfortreatinglowerattack Mostofthedifferenceoccursattheinterfacestage.Theendpoint- successaloneasstrongerprotection. oriented design makes 98%of cases expressible withalmost no AuthorizationGranularity.Exposingacapabilityforalegit- subsequentloss.Forthequery-orienteddesign,24ofits27losses imatebusinesstaskleavesopenwhichusersmayapplyittowhich (55% of all cases) occur at the interface stage, where no intent objectsandunderwhatconditions.Acapabilityforretrievingor- reachestherequiredoperationorslotfillingfailstotransmitthe ders,forexample,stillrequireschecksonthecaller’saccesstothe requiredargument;theother3areC1-realizablecasesthatClaude requestedorder.Bindingcallstoauseridentityestablisheswhoacts; Codefailstoexploit.Tocontrolforworkflowcoverage,weexamine object-levelauthorizationandbusinessrulesdeterminewhatthat the21workflows(42%)thatallthreedesignssupport,fixedbefore usermaydo.Agent-facinginterfacescansupportthesechecksby anyattackresult,whichcorrespondto17cases.Theorderingholds: bindingidentity-dependentargumentsandlimitingunnecessaryin- thedesignsretain16,15,and9cases(94%,88%,and53%).Thetwo putcontrol.Thebackendmustenforcethecorrespondingpolicyon workflow-orientedlossesherehaveasupportedworkflowbutan eachrequest,includingrequestsexpressedthroughvalidbusiness unexposedoperationorargument.Ofthe8query-orientedlosses,6 arguments.Asservicesmovebeyondwebsitenavigation,theserules reachtherelevantintentbutlosetheattackargumentinslotfilling, mustremaineffectiveacrosseveryinterfaceexposingtheoperation. and2areClaudeCodefailures.Thegapthereforereflectsargument Agent-GeneratedCallSequences.Thesinglestep-completion deliverythroughtypedslotsratherthannatural-languageaccess case shows that an agent can supply prerequisite calls omitted assuch,conditionalonourslotschema,planner,andrequestset. fromauser’srequestandtherebycompleteanattack.Thesecu- Finding5:Supportingthesamelegitimateworkflowsdoesnotimply rity implications of delegation thus extend to the sequence the deliveringthesameattackinputstothebackend.Argumentconstraints agentconstructsduringexecution.Eachcallmayinvokeanor- andquery-plannerbehaviorcanpreventtestedattacksfromreaching dinarybusinessoperation,whiletheresultingsequenceviolates theireffectevenwhenthelegitimateworkflowsremainavailable. arulegoverningeligibility,ordering,orrepeatedexecution.This motivatesenforcingworkflowrulesagainstthecurrentapplication statethroughoutexecution.Checksontheinitialinstructionorthe 5 Discussion availabilityofindividualtoolsalonecannotestablishwhetherthe 5.1 ImplicationsforAgent-FacingWebServices completedworkflowisauthorized. PreservingLegitimateFunctionality.Asagentsbecomeapri- maryentrypointtoWebservices,interfacerestrictionsdetermine 5.2 ImplicationsforSecurityEvaluation whichtasksuserscandelegate.Ourresultsshowthatlowerex- ploitabilitycanaccompanytheremovaloflegitimateworkflows. Evaluatinganagent-facingservicerequiresexplainingwhyanat- Thecomparisononsharedworkflowsshowsthatanoperationcan tackfailsandwhatlegitimatefunctionalityremainsavailable.An remainavailablewhileattackinputsfailtoreachthebackend.In unavailableoperation,anunsuccessfulagentrun,andanenforced thequery-orienteddesign,theselossesreflectbothslotconstraints authorizationcheckhavedifferentimplicationsforservicedesign. WWW’27,May10-14,2027,Dublin,Ireland Trovatoetal. Aggregateattacksuccesscanobscurethesedifferences.Thecom- attackerwhoalreadyknowstheattackreachesthesamebackend parisonsinthisstudyshowthevalueoftestingwhataninterface throughadefault-configuredagentandanagent-facinginterface. permitsindependentlyofagentexecutionandverifyingtheresult- Thequestionisnotwhethertheagentcanfindorbuildanexploit ingeffectsontheapplication.Readingtheseoutcomesalongside butwhetherthismediationcarriesaknownonethrough. benigncoverageandcompletionhelpsdistinguishreducedexpo- AttacksonLLMAgents.Aparallellinetreatstheagentitself surethroughfunctionalityremovalfromrestrictionsthatpreserve asthetarget.Directandindirectpromptinjectioncanoverride theintendedtask. anLLM-integratedapplication[11,17],benchmarksmeasurehow Futureevaluationsshouldalsoexaminethecallsequencesagents oftentool-usingagentsfollowinjectedinstructions[5,30],and generateandrepeatmeasurementsasmodelsandinterfacesevolve. emulatedsandboxesexposeunsafeagentbehavioratscale[24]. Anagentupdatemayrecoveranexecutionfailurethatpreviously Agent-in-the-middleattacksinterceptmessagesbetweencooperat- preventedanattack,whileaninterfaceupdatemayexposeamiss- ingagents[?],whereasouragentinthemiddlesitsbetweenauser ing operation or argument. Broader comparisons across agent– andabackend.AgentHarmevaluatesharmful-taskexecution[1], interfacecombinationswouldclarifywhethertheeffectsobserved whileST-WebAgentBenchevaluatessafetyandpolicyadherence inourseparatesweepspersistwhenbothlayerschange. duringbusinesstasks[15].BrowserARTstudieswhetherrefusal trainingtransferstobrowseragents[14],andtheCybersecurity 5.3 ThreatstoValidity RefusalFrameworkexaminesrefusalsincybertasksinrelation Representativeness.Thebenchmarkcovers44documentedvul- tooperationalcontext[13].Ourstudycomplementstheseevalua- nerabilitiesacrosstwelveapplicationfamilies,withcasesdrawn tionsbycomparingaccesspathstothesamevulnerablebackend, fromproductionapplicationsanddeliberatelyvulnerableenviron- separatinginterfaceeffectsfromtheexternalagent’sexecution, ments.Itsupportscontrolledcomparisonsofknowndefects,but andmeasuringtheaccompanyinglossoflegitimatefunctionality. doesnotestimatetheirprevalenceacrossdeployedWebservices. Wesubmitrequeststhatpreservetheattackobjectiveandexploit- InterfaceConstruction.Retentiondependsonhowmuchan relevantinputstoataskassistantandverifytheresultingbackend interfaceexposes,andwebuilttheinterfacesourselves,sothe91% consequenceswithindependentoracles. referencefigureisnotanestimatefordeployedservices.Wederive Agent-FacingWebAccess.Agentsincreasinglyactoverreal themblindfrombenignworkflows,freezethembeforeanysecurity Webinterfaces,fromgroundedinteractionenvironments[28,31] test(Section3.2),reportbenigncoveragebesideretention,andgive togeneralistagentsonlivesites[7],builtonreasoning-and-acting arangeacrossthreedesigns(39%to95%)ratherthanasinglepoint. andtool-usemethods[25,29].AccessschemessuchasOpenAPI Thequery-orientedresultsadditionallydependonourplannerand descriptions,MCP[2],andNLWeb[19],togetherwithcommerce slotconstraints,andotherimplementationsmayexposedifferent protocolssuchasUCP[26],definehowservicesexposefunctional- operationsorpreservedifferentattackinputs. itytotheseagents.SecurityanalysesofMCPcatalogthreatssuch ExecutionandMeasurement.Observedexploitabilitydepends asmalicioustooldescriptions[?],andprompt-to-SQLinjection onthetestedrequests,agentversions,andexecutionbudgets.A showsthatanLLM-integratedWebapplicationcanrelayattacker security-orientedsystempromptorapprovalbyapartyotherthan inputintodatabasequeries[?].Thisliteraturedescribeshowagent- theusermaychangeagent-layeroutcomesandisnottested.Case- mediatedaccessworksandtherisksitadds,whereaswemeasureits levelretentionrecordssuccessinatleastonerepetitionanddoes securityconsequenceforexistingvulnerabilitiesthroughmatched notimplyconsistentsuccessacrossattempts.Failuresremainspe- accesspathsandoracle-verifiedeffects. cifictothetestedconditions,asillustratedbythecaserecovered 7 Conclusion withalargerbudget.Independentoraclesverifypredefinedsecurity consequences;theydonotenumerateeverypossibleharm.Our WepresentedVela,abenchmarkthatcomparesnativeaccess,di- one-factorcomparisonsidentifydifferencesaroundthereference rectcapabilityinvocation,andagent-mediatedexecutiononthe configuration,whileinteractionsbetweenagentsandinterfacede- samevulnerablebackendsunderindependentoracles,separating signsremaintobeevaluated.Theresultsapplytocapability-based theeffectoftheagent-facinginterfacefromthatoftheagent.Across accessunderthestatedthreatmodel;browser-drivenaccessand 44cases,theagentremovednoneoftheattacksthereferenceinter- attacksthatcompromisetheagentrequireseparateinvestigation. faceletthrough,and91%ofthevulnerabilitiesretainedtheirnative consequence.Thecapabilitydesignshiftedretentionfarmorethan 6 RelatedWork thechoiceofagent,from95%underanendpoint-orienteddesign AgentsasWebAttackers.Agrowingbodyofworkmeasures to39%underaquery-orientedone,thoughthelatteralsolowered whetherLLMagentscanexploitWebvulnerabilitiesontheirown. benigncoverageto46%.Asagent-facinginterfacesbecomeacom- BenchmarkssuchasWebExploitBench[16],CVE-Bench[32],and monpathtotheWeb,whateachcapabilityexposes,ratherthanthe PACEbench[18]deployvulnerableapplicationsasisolatedtargets, agent,isthemainleveronexposureinourimplementation,with andstudiesshowthatagentscanhacklivewebsitesandexploit theadmissibleargumentdomainofeachcapabilityasthemost disclosedone-dayvulnerabilitiesfromtheirdescriptions[8,9].Re- promisingrefinementtotest. latedsystemsconfiguremodelsforpenetrationtestingandexploit EthicalConsiderations construction[6,12].Thesestudiesexamineagents’vulnerability- discoveryandexploitationcapabilitiesunderdifferentlevelsofprior Weexercisevulnerabilitiesonlyinapplicationsthatwedeployand knowledgeandaccess.Wereusesuchenvironmentsasasource control,insideisolatedenvironmentswithsyntheticaccountsand ofvalidatedcasesbutstudyadifferentsetting,inwhichahuman TheAgentintheMiddle:HowAI-MediatedAccessReshapesWebVulnerabilityExploitation WWW’27,May10-14,2027,Dublin,Ireland syntheticdata,andwetestnoliveserviceandnothirdparty.The characterizationofwhichattackssurviveagentmediationcould workinvolvesnohumansubjectsorpersonaldata,andthevul- informmisuse,butwejudgeitsdefensivevaluetooutweighthis nerabilitiescomefromexistingpublicbenchmarksanddisclosed risk,sinceithelpsinterfacedesignersandtesterscloseexposed advisories,sowereportnonewvulnerabilitiesandmakenodis- paths.Thereleasedartifactreproducesonlythesepubliccasesin closureclaims.Ourfindingsconcernhowagentsandinterfaces isolateddeploymentsandcontainsnonewexploit. handleknownexploitsratherthanaflawinaspecificproduct.A WWW’27,May10-14,2027,Dublin,Ireland Trovatoetal. References [25] TimoSchick,JaneDwivedi-Yu,RobertoDessì,RobertaRaileanu,MariaLomeli, [1] MaksymAndriushchenko,AlexandraSouly,MateuszDziemian,DerekDuenas, EricHambro,LukeZettlemoyer,NicolaCancedda,andThomasScialom.2023. MaxwellLin,JustinWang,DanHendrycks,AndyZou,ZicoKolter,MattFredrik- Toolformer:Languagemodelscanteachthemselvestousetools. Advancesin son,etal.2025. Agentharm:Abenchmarkformeasuringharmfulnessofllm neuralinformationprocessingsystems36(2023),68539–68551. agents.InInternationalConferenceonLearningRepresentations,Vol.2025.79185– [26] ShopifyandGoogle.2026.UniversalCommerceProtocol(UCP).https://ucp.dev. 79220. Openspecificationforagent-mediatedcommerce. [2] Anthropic.2024.ModelContextProtocol.https://modelcontextprotocol.io.Open [27] xAI.2026.GrokBuild.https://github.com/xai-org/grok-build.Apache-2.0. specification,introducedNovember2024. [28] ShunyuYao,HowardChen,JohnYang,andKarthikNarasimhan.2022.WebShop: [3] Anthropic.2025.ClaudeCode.https://github.com/anthropics/claude-code.Pro- TowardsScalableReal-WorldWebInteractionwithGroundedLanguageAgents. prietary,distributedasabinary. InAdvancesinNeuralInformationProcessingSystems(NeurIPS). [4] Anthropic.2026.HowwecontainClaudeacrossproducts. RetrievedMay25,2026 [29] ShunyuYao,JeffreyZhao,DianYu,NanDu,IzhakShafran,KarthikNarasimhan, fromhttps://www.anthropic.com/engineering/how-we-contain-claude andYuanCao.2023. ReAct:SynergizingReasoningandActinginLanguage [5] EdoardoDebenedetti,JieZhang,MislavBalunović,LucaBeurer-Kellner,Marc Models.InInternationalConferenceonLearningRepresentations(ICLR). Fischer,andFlorianTramèr.2024. AgentDojo:ADynamicEnvironmentto [30] QiusiZhan,ZhixiangLiang,ZifanYing,andDanielKang.2024. InjecAgent: EvaluatePromptInjectionAttacksandDefensesforLLMAgents.InAdvancesin BenchmarkingIndirectPromptInjectionsinTool-IntegratedLargeLanguage NeuralInformationProcessingSystems(NeurIPS),DatasetsandBenchmarksTrack, ModelAgents.InFindingsoftheAssociationforComputationalLinguistics(ACL). Vol.37. [31] ShuyanZhou,FrankF.Xu,HaoZhu,XuhuiZhou,RobertLo,AbishekSridhar, [6] GeleiDeng,YiLiu,VíctorMayoral-Vilches,PengLiu,YuekangLi,YuanXu, XianyiCheng,TianyueOu,YonatanBisk,DanielFried,UriAlon,andGraham TianweiZhang,YangLiu,MartinPinzger,andStefanRass.2024.PentestGPT: Neubig.2024.WebArena:ARealisticWebEnvironmentforBuildingAutonomous EvaluatingandHarnessingLargeLanguageModelsforAutomatedPenetration Agents.InInternationalConferenceonLearningRepresentations(ICLR). Testing.InProceedingsofthe33rdUSENIXSecuritySymposium(USENIXSecurity). [32] YuxuanZhu,AntonyKellermann,DylanBowman,PhilipLi,AkulGupta,Adarsh 847–864. Danda,RichardFang,ConnerJensen,EricIhli,JasonBenn,JetGeronimo,Avi [7] XiangDeng,YuGu,BoyuanZheng,ShijieChen,SamuelStevens,BoshiWang, Dhir,SudhitRao,KaichengYu,TwmStone,andDanielKang.2025.CVE-Bench: HuanSun,andYuSu.2023.Mind2Web:TowardsaGeneralistAgentfortheWeb. ABenchmarkforAIAgents’AbilitytoExploitReal-WorldWebApplication InAdvancesinNeuralInformationProcessingSystems(NeurIPS). Vulnerabilities.InProceedingsofthe42ndInternationalConferenceonMachine [8] RichardFang,RohanBindu,AkulGupta,andDanielKang.2024.LLMAgents Learning(ICML). canAutonomouslyExploitOne-dayVulnerabilities.arXiv:2404.08144. [9] RichardFang,RohanBindu,AkulGupta,QiusiZhan,andDanielKang.2024. A BenchmarkComposition LLMAgentscanAutonomouslyHackWebsites.arXiv:2402.06664. [10] Google.2026.UndertheHood:UniversalCommerceProtocol(UCP).RetrievedJanu- Table1inSection4.1summarizestheapplicationfamiliesandcase ary11,2026fromhttps://developers.googleblog.com/under-the-hood-universal- provenance.Figure5showsthevulnerabilitycategoriesandtheir commerce-protocol-ucp/ [11] KaiGreshake,SaharAbdelnabi,ShaileshMishra,ChristophEndres,Thorsten analysisgroups.AtrainingchallengeisnotanindependentCVE Holz,andMarioFritz.2023.NotWhatYou’veSignedUpFor:CompromisingReal- andisadmittedonlyaftersuccessfulnativereproduction. WorldLLM-IntegratedApplicationswithIndirectPromptInjection.InProceedings RepresentativeidentifiersincludeDolibarrCVE-2024-5314and ofthe16thACMWorkshoponArtificialIntelligenceandSecurity(AISec).79–90. [12] AndreasHappeandJürgenCito.2023.Gettingpwn’dbyAI:PenetrationTesting CVE-2024-5315,SuiteCRMCVE-2024-36412,NextcloudCVE-2026- withLargeLanguageModels.InProceedingsofthe31stACMJointEuropean 45281,CactiCVE-2024-25641andCVE-2024-34340,ZabbixCVE- SoftwareEngineeringConferenceandSymposiumontheFoundationsofSoftware 2024-22120,PrestaShopCVE-2024-34716,andMoodleMSA-26-0044. Engineering(ESEC/FSE). [13] EliotKrzysztofJones,MateuszDziemian,MattFredrikson,andJZicoKolter. TheWordPressfamilydrawsonelevenpluginCVEs,includingthe 2026.ANewFrameworkforCybersecurityRefusalsinAIAgents.arXivpreprint WebDirectoryFreeSQLinjectionCVE-2024-3552andtheHash arXiv:2606.02644(2026). [14] PriyanshuKumar,ElaineLau,SaranyaVijayakumar,TuTrinh,ScaleRedTeam, Formfile-uploadRCECVE-2024-5084,eachofwhichneedsitsown ElaineChang,VaughnRobinson,SeanHendryx,ShuyanZhou,MattFredrikson, workflowandinterfacemapping. etal.2024.Refusal-trainedllmsareeasilyjailbrokenasbrowseragents.arXiv preprintarXiv:2410.13886(2024). Input precision (41%) [15] IdoLevy,SamiMarreed,AlonOved,AviYaeli,SegevShlomov,etal.2026. St- SQL injection 18% webagentbench:Abenchmarkforevaluatingsafetyandtrustworthinessinweb Cross-site scripting 9% agents.InInternationalConferenceonLearningRepresentations,Vol.2026.122213– Template injection 7% 122255. Command injection 5% [16] FengyuLiu,JiarunDai,YiheFan,WuyuaoMai,ZiaoLi,BofeiChen,JieZhang, Insecure deserialization 2% ZhengLou,BochengXiang,QiyiZhang,XudongPan,GengHong,YuanZhang, Application logic (34%) andMinYang.2026.AgentCyberRange:BenchmarkingFrontierAISystemsin Object-level authorization 11% RealisticCyberRanges.arXivpreprintarXiv:2606.14295. Function-level authorization 9% Business-logic flaw 9% [17] YiLiu,GeleiDeng,YuekangLi,KailongWang,TianweiZhang,YepangLiu, Authentication bypass 5% HaoyuWang,YanZheng,andYangLiu.2023.PromptInjectionAttackagainst Resource handling (25%) LLM-integratedApplications.arXiv:2306.05499. File upload or write 9% [18] ZichengLiu,LigeHuang,JieZhang,DongruiLiu,YuanTian,andJingShao. Path traversal or file access 9% 2026.PACEbench:AFrameworkforEvaluatingPracticalAICyber-Exploitation Information disclosure 7% Capabilities.InInternationalConferenceonLearningRepresentations(ICLR). [19] Microsoft.2025. NLWeb:NaturalLanguageInterfacesforWebsites. https:// 0% 5% 10% 15% 20% github.com/nlweb-ai/NLWeb.Open-sourceprojectannouncedatMicrosoftBuild Share of native-validated cases 2025.EachinstancealsoservesasanMCPserver. [20] OpenAI.2025.CodexCLI.https://github.com/openai/codex.Apache-2.0. Figure 5: Share of native-validated cases by vulnerability [21] OpenAI.2026.PoweringProductDiscoveryinChatGPT. RetrievedMarch24,2026 category,groupedandcolouredbyanalysisgroup. fromhttps://openai.com/index/powering-product-discovery-in-chatgpt/ [22] OpenAPIInitiative.2024.OpenAPISpecificationv3.1.1.https://spec.openapis. org/oas/v3.1.1.html. [23] OWASPFoundation.[n.d.]. crAPI:Challenges. https://github.com/OWASP/ B AdditionalResults crAPI/blob/b5fc307f3e5f875809b095b771108ef342a33724/docs/challenges.md. Challenge8;accessed26October2026. Figure6complementsRQ1underthereferenceconfiguration.Input- [24] YangjunRuan,HonghuaDong,AndrewWang,SilviuPitis,YongchaoZhou, bearingrequestssucceedin96%oftheireligiblecasesandgoal- JimmyBa,YannDubois,ChrisJ.Maddison,andTatsunoriHashimoto.2024.Iden- tifyingtheRisksofLMAgentswithanLM-EmulatedSandbox.InInternational orientedonesin89%;theseusedifferenteligiblesets.Onthecases ConferenceonLearningRepresentations(ICLR). eligibleforbothforms,case-levelretentionisidentical.Mediation also leaves the class mix nearly unchanged. Resource-handling TheAgentintheMiddle:HowAI-MediatedAccessReshapesWebVulnerabilityExploitation WWW’27,May10-14,2027,Dublin,Ireland cases fall from 25% of the native set to 23% of the retained set, BudgetSweep.Wevarythecapsoncalls,tokens,andtime. consistentwiththeinterface-levellossesreportedinSection4.2. Doublingeachcaprecoversnoadditionalcaseunderthereference configurationandoneunderGrokBuild.OneofGrokBuild’stwo (a) Retained per request form, of eligible cases case-levellossesisthereforesensitivetotheexecutionbudget. Input-bearing 96% Direct-CallerBaseline.Forthestep-completioncase,thesuc- cessfulsequencecontains3calls,2ofwhichtheattackerdidnot | Goal-oriented | | | | 89% | | | | | | | ------------- | --- | --- | --- | --- | --- | --- | --- | --- | --- | specify.Acallerrestrictedtotherequestedsubmissionomitsthese | | 0% | 25% | 50% | 75% 100% | | | | | | | --- | --- | --- | --- | -------- | --- | --- | --- | --- | --- | prerequisites,whereasthecompleteC1sequenceincludesthem (b) Class mix before and after mediation andsucceeds.Thiscontrolmeasuresstepcompletionthroughthe | | | 41% | 34% | 25% | additionalcallssuppliedbytheagent. | | | | | | --- | --- | --- | --- | --- | ---------------------------------- | --- | --- | --- | --- | Native | | | | | | Interface-Comparable | Subset. | We compare | agents | on the | | --- | --- | --- | --- | --- | -------------------- | ------- | ---------- | ------ | ------ | | | | 43% | 35% | 23% | | | | | | Retained C1-realizablecasesunderthefixedworkflow-orientedinterface. Input precision Application logic Resource handling Retentionis100%underthereferenceconfiguration,andthecross- agentspreadstayswithin5percentagepoints.Thesedifferences Figure6:Referenceconfiguration.(a)Shareofeligiblecases ariseduringagentexecutiononasharedsetofcasesalreadyrealized retainedperrequestform.(b)Analysis-groupsharesamong bydirectinvocation. nativeandretainedcases. D PairedConditionsandCaseMapping Table2breaksdowntheagentsweepofRQ2byoutcomekind. Table3givesthepairedC1/C2outcomeofeveryconfigurationover Nocase-levellossisattributedtorefusalorinputnormalization, itsC1-realizablesubset.Thesepairedtablesexcludeinterface-stage | and the | only agent-layer | losses | are Grok Build’s | malformed or | | | | | | | ------- | ---------------- | ------ | ---------------- | ------------ | --- | --- | --- | --- | --- | losses,includingstructuralrestrictionsandquery-plannerexecu- abandonedcallsinmulti-stepresource-handlingworkflows. tionlosses.NocaseinthetestedconfigurationssucceedsunderC2 Table2:Agent-layeroutcomeson40C1-realizablecasesun- withoutC1.Theanonymousartifactrepositoryprovidestheper- dertheworkflow-orienteddesign. caseworkflowandinterfacemapping,C1result,fiveC2resultsper applicablerequest,oracleevidenceindex,andinvalid-runrecords. Table3:PairedC1/C2outcomesoneachconfiguration’sC1- | | | | ClaudeCode Codex | GrokBuild | | | | | | | --- | --- | --- | ---------------- | --------- | --- | --- | --- | --- | --- | realizablesubset(𝑛),excludinginterface-stagelosses.✓and | Transformedcases | | | 4 | 4 3 | | | | | | | ---------------- | --- | --- | --- | --- | --- | --- | --- | --- | --- | ×denoteverifiedandunverifiedoutcomes;𝑝istheexacttwo- | Lostattheagent | | | 0 | 0 2 | | | | | | | -------------- | --- | --- | --- | --- | --- | --- | --- | --- | --- | sidedMcNemar𝑝-value. | Refusal | | | 0 | 0 0 | | | | | | | -------------------- | --- | --- | --- | --- | ------------------- | -------- | ------ | ------ | --- | | Inputnormalization | | | 0 | 0 0 | | | | | | | Malformedorabandoned | | | 0 | 0 2 | | 𝑛 C1✓C2✓ | C1✓C2× | | 𝑝 | | | | | | | Configuration | | | C1×C2× | | | | | | | | ClaudeCode,workflow | 40 40 | 0 | 0 | 1.0 | C MechanismControlsandSensitivity | | | | | | Codex,workflow | 40 40 | 0 | 0 | 1.0 | | --- | --- | --- | --- | --- | -------------- | ----- | --- | --- | --- | Eachcontrolisolatesonelinkamongtheinterface,theagent,and GrokBuild,workflow 40 38 2 0 0.50 | | | | | | ClaudeCode,endpoint | 43 42 | 1 | 0 | 1.0 | | --- | --- | --- | --- | --- | ------------------- | ----- | --- | --- | --- | therequest,andstateswhatitsresultimpliesforattribution. ArgumentReplay.ForeachcaserealizedbyC2,wereplaythe ClaudeCode,query 20 17 3 0 0.25 agent-generatedargumentsofthecallsspecifiedbytheattacker, inorder,fromthesameverifiedsnapshotthroughascriptedcaller. E OracleandCaseRecords Thiscontroltestswhetherthespecifiedcallssufficewithoutthe agent’sadditionalsteps.Underthereferenceconfiguration,replay Foraconfidentialityviolation,theoraclemarkstheprotectedstatus ofsyntheticdataandchecksthattheattacker-visiblechannelre- reproducestheeffectinallrealizablecasesexceptthecrAPIstep- turnedit.Presenceofthevalueinthebackendaloneisnotenough. completioncase.ItsC2sequencecomprisesanorderlookup,return Foranintegrityviolation,itcomparespersistentstatebeforeand initiation,andreturnsubmission.Replayingonlytherequested aftertheattempt.Othereffectsusetrustedobservationssuchas submissionleavesthebalanceunchanged,whereasthecomplete sequencecreditsit.TheprimaryC1baselineincludestheprerequi- filestateorapplicationeventrecords.Eachcaserecordstoresthe nativereproduction,negativecontrols,theoraclepredicate,actor sitecallsandalsosucceeds.Thiscontrastidentifiesthedependence andownershipassertions,resetreceipts,andper-conditiontraces. onthecallssuppliedbytheagent. Eachexecutedattemptisclassifiedasaverifiedsuccess,avalid Input-BearingandGoal-OrientedRequests.Wecompare failure,oraninvalidrun.Structuralinexpressibilityisrecordedat requeststhatpreservetheattackobjectiveandrelevantinputson thecaseseligibleforbothforms.Thetwoformsyieldidentical thecase-mappingstage.Failedresets,incompatibleconfigurations, andinconclusiveobservationsareinvalidandarereportedsepa- case-levelretentiononthissharedsubset. RequestPhrasing.Wevaryordinarywordingwhilekeeping rately.Aninvalidattemptisrerunfromafreshsnapshotupto2 times,andinthereferenceconfiguration2.7%oftheC2attempts | the attack | objective | and exploit-relevant | inputs | fixed. Across 3 | | | | | | | ---------- | --------- | -------------------- | ------ | --------------- | --- | --- | --- | --- | --- | wereinvalid,allofwhichwerevalidonthefirstrerun.Aninvalid paraphrasespercase,theoutcomeisunchangedin95%ofcases, indicatingstabilityunderthesetestedwordingchanges. runnevercountsasanexploitationfailure.