Rethinking Query Optimization for Multi-Agent Systems
NOMA treats multi-agent LLM pipelines as a query optimization problem, jointly choosing topology, model and execution engine against latency, cost and accuracy instead of hand-wiring fixed agent graphs.
Paper (from PDF)
Read full textRethinking Query Optimization for Multi-Agent Systems ZoiKaoudi IoanaGiurgiu ITUniversityofCopenhagen IBMResearchEurope Denmark Switzerland zoka@itu.dk igi@zurich.ibm.com ABSTRACT resultinginpipelinesthatareofteninefficient,brittle,andunable Theproliferationoflargelanguagemodels(LLMs)hasaccelerated toadapttoevolvingworkloadrequirements.Inthesesettings,the theadoptionofagent-baseddatapipelines.Yetcurrentapproaches primaryunitofcomputationisnolongertheoperatorbuttheagent, remainadhoc,relyingonfixedstructures,predefinedLLMs,andsin- andtheprimaryartifactisnotafixedqueryplanbutadynamically gleexecutionengines,withoutsystematicoptimizationacrosshet- structuredpipelineofinteractingagents.Thisshiftcallsforanew erogeneousdatasourcesandengines.ThispaperpresentsNOMA, direction:datasystemsforagents.Wemustdesignsystemswhere aqueryoptimizationframeworkformulti-agentdatapipelines.We multi-agentpipelinesthemselvesaretheobjectofoptimization.As arguethatoptimizingagenticpipelinesisafundamentallydifferent aresult,decisionsaboutpipelinetopology,modelselection,and queryoptimizationproblem,withcentralchallenges:(i)amulti- executionenginescannolongerbemadeindependentlyorahead dimensionalsearchandobjectivespace,wheretopology,model, oftimebutmustbejointlyoptimizedundercompetingobjectives andenginechoicesmustbeoptimizedjointlyacrosslatency,cost, suchaslatency,cost,andaccuracy. andaccuracy;(ii)avariablepipelinetopology;(iii)theco-existence Challenges. Thisperspectiveisanalogoustoearlier“database ofdiversedatamodels,leavingnocommonoperatoralgebra;and formachinelearning”efforts,whichapplieddatabaseprinciples (iv)thesignificantcostofexecutingthesepipelines.Ourcontrolled tooptimizeMLpipelines[18,25].However,multi-agentpipelines experimentoverareal-world10-agentpipelinerevealsextreme introducenewchallenges.First,theoptimizationspaceisinher- variance(153×cost,5×latency,25%quality)andthatoptimalplans entlymulti-dimensional,requiringjointreasoningovertopology, areheterogeneousconfigurationsnouserwouldconstructmanu- models,andenginesundercompetingobjectives.Second,pipeline ally.Ouranalysisofrealdeploymentsconfirmstheseinefficiencies topologyisitselfadecisionvariable,meaningthattheoptimizer aresystematic.WepresentNOMAasanintegratedoptimization mustdeterminenotonlyhowtoexecuteapipelinebutalsohow loopinwhichplangeneration,costestimation,runtimerefinement, tostructureit.Third,thecoexistenceofstructureddata,unstruc- andsemanticcachingreinforceoneanotheracrossexecutions,set- …
Explained
What happened
A PVLDB 2026 vision paper (IT University of Copenhagen and IBM Research) argues that multi-agent data pipelines need a real query optimizer, and sketches one called NOMA.
Their experiment on a real 10-agent pipeline (about 2.8 GB of data, customer-support style tasks: sentiment, urgency, entity extraction, report generation):
- Each deterministic agent can run on Spark or Presto with 1 or 4 workers; each LLM agent can use Llama 3.1 8B, Qwen2.5 72B or Claude Sonnet 3.5.
- That gives 186,624 possible plans for one fixed topology. Sampling 10,000 showed 153x cost, 5x latency and 25% quality differences.
- The best plans were mixed configurations no human would pick by hand, and changing the topology itself found even better ones.
Why it works
Classic query optimizers work because they have a common algebra (relational operators), cost models and fixed plan shapes. Agent pipelines break all of that:
- Many dimensions at once: pipeline shape, which model per agent, which engine per step, judged on latency, cost and accuracy together (a Pareto problem, not a single cost).
- Topology is a variable: the optimizer must decide how to structure the pipeline, not just how to run it.
- No common algebra: tables, text, embeddings and streams mix, so classic pruning rules don't apply.
- Expensive, random execution: every trial run costs real money and gives different quality each time.
NOMA's proposal: generate candidate plans, estimate cost per agent, re-optimize at runtime as real costs show up, and use semantic caching to reuse artifacts across runs.
The bigger pattern
This is "databases for ML" again, now applied to agents: when a workflow gets expensive and repeated, it becomes an optimization problem with plans, cost models and caches. Expect LLM routers and agent frameworks to grow optimizer-like features.
What it means in practice
- In any LLM pipeline you build, model choice per step is the biggest cost lever. A cheap model for classification and a strong model only for final synthesis can cut cost by orders of magnitude with little quality loss.
- Measure pipeline-level quality, since errors compound across agents.
- Cache intermediate results (classifications, extractions) keyed by input hash, just like memoizing expensive background jobs.
- A useful framing: treat an agent workflow like a query plan and optimize it with data.