paperarXiv (systems, data, ML)·22 Oct 2026·Maria Gabriela Alves Zuppardo

Beyond Accuracy: Disentangling Decomposition from Typed-State Quality Assurance in Multi-Agent LLM Pipelines

Separates task decomposition from typed-state QA to explain gains in multi-agent LLM pipelines.

Depth 4/5Score 4.61

Paper (from PDF)

Beyond Accuracy: Disentangling Decomposition from Typed-State Quality Assurance in Multi-Agent LLM Pipelines MariaGabrielaAlvesZuppardo ViniciusCardosoGarcia CentrodeInformática,UniversidadeFederalde CentrodeInformática,UniversidadeFederalde Pernambuco(UFPE) Pernambuco(UFPE) Recife,PE,Brazil Recife,PE,Brazil mgaz@cin.ufpe.br vcg@cin.ufpe.br ABSTRACT Noneofthestudieswesurveyedisolatesthecontributionof Multi-agentLLMpipelinesareoftencreditedwithgainsinsoft- anarchitecturalmechanism,suchasatypedinter-agentcontract, wareengineeringtasks,butitisrarelyclearwhetheragainstems fromthatofpromptdesign.Whenapipelineoutperformsasingle- fromthetypedcontractbetweenagentsorfromthespecialized agentbaseline,thegainisoftenreadasevidencethatthecontract prompts that decomposition enables. We study this attribution improvesoutputquality;whetherpromptingalonewouldproduce probleminRequirementsEngineering,comparingthreearchitec- thesamegainisrarelychecked,andsystematicreviewsconfirm turesforclassifyingrequirementsasfunctionalornon-functional thatthisconflationremainsunresolved6,18.F1isnotthis andthenprioritizingthemwithMoSCoW:asingle-callbaseline;a paper’sobjectofinterestbuttheinstrumentweusetoruleout two-callvariantthatreusesthepipeline’sspecializedpromptsbut oneexplanationbeforeaskingamorespecificquestion:weshow passesanunvalidatedintermediate;andapipelinewithaPydantic- thatthetypedcontractbarelymovesF1(Δ s ≈0),whichpointsto validatedtypedstate.The18conditionscoverPROMISENFR+(625 decompositionanditstask-specificpromptsastheaccuracydriver requirements), three locally hosted small language models, and andleavesopenwhatdifferencesaccompanythetypedcontract,a twolanguages.TheF1gainoverthebaselineisassociatedwith questionF1cannotanswer(§6.5,§6.6). thetwo-call,specialized-promptvariant(upto+0.075),whereas Wethereforeasknotonlywhetheramulti-agentpipelinere- thetypedstateaddsasmalldifference(−0.003to+0.013)thatis finesclassificationaccuracy,butwhicharchitecturalelementdrives significantinoneofsixcomparisons.Beyondaccuracy,thetyped anygainandwhatdifferencesaccompanythetypedcontract.We pipelinecontainsout-of-rangemodeloutputsattheagentbound- operationalisequalityassurancenarrowly,asearlystructuralval- ary,whiletheunvalidatedvariantfailslateranddropstherequire- idationandcross-modelbehaviouralconsistency,looselyrelated ment,anditshowsasmall,reproducibleincreaseininter-model …

Explained

What happened

A study from UFPE (Brazil; SBQS 2026) asks a question most multi-agent papers skip: when a multi-agent LLM pipeline beats a single call, what actually caused the gain? Is it the typed contract between agents, or just the better, task-specific prompts that splitting the work allowed?

Task: classify 625 software requirements (PROMISE NFR+) as functional or non-functional, then prioritize them with MoSCoW. Three architectures, run on three small local 7-8B models in two languages (18 conditions):

Single call baseline.
Two calls with the pipeline's specialized prompts, passing an unvalidated intermediate result.
Two agents in LangGraph with a Pydantic-validated typed state between them.

Results:

The F1 gain (up to +0.075) comes from decomposition and specialized prompts (variant 2).
The typed state adds almost nothing to accuracy (-0.003 to +0.013, significant in only 1 of 6 comparisons).
But the typed state catches bad model outputs at the agent boundary, one step earlier. Without it, the pipeline fails later and silently drops the requirement. It also gives a small, reproducible boost in consistency across models.

Why it works

Splitting a task lets each prompt focus on one decision, which helps small models most. That's a prompting effect, not an architecture effect.

A validated schema doesn't make the model smarter; it makes failures visible and contained. Out-of-range values (an invalid category, a missing field) are rejected where they occur, instead of flowing downstream and corrupting later steps.

The bigger pattern

This is the attribution problem: an end-to-end win bundles several changes, so you need ablations to know which one mattered. And it's a reminder that accuracy is not the only quality metric. Reliability (fail fast, fail loud) is a separate property that F1 doesn't measure. Caveat: small study, one dataset, 7-8B models only.

What it means in practice

When adding LLM steps to a background job pipeline, get the accuracy from focused prompts per step, and the reliability from validating every model output against a schema (dry-schema, JSON Schema, Pydantic) at the boundary.
On validation failure, retry or route to a fallback, never pass the bad value on.
When you claim an architecture change improved results, run the ablation.