The Thin Orchestrator: How We Made Our AI Agents More Reliable by Giving the Router Less to Do
Meta made data pipeline agents more reliable by keeping the orchestrator thin: it only routes work and enforces checks, while specialist agents hold the knowledge.
Extract
Sometimes, giving an AI agent all of the knowledge it needs is the wrong approach. At Meta, we’ve seen notable improvements in efficiency and accuracy by keeping the orchestration layer of our data pipeline agents deliberately thin: It routes work and enforces reliability checks, while specialized agents do the domain reasoning. We’re sharing our own process and insights, plus how any team can apply this to their production AI agents. We built a 1,300-line orchestrator that routes 20+ workflows to 14 specialized AI agents. It performs zero domain work itself. That’s the point. When we first deployed AI agents to automate data column onboarding across Meta’s recommendation systems — a process spanning three code repositories, three programming languages, and up to six pipeline stages that must stay in sync — we did what most teams do: We built one smart agent that knew everything. It worked in demos. In production, it started hallucinating. The fix wasn’t better prompts or a bigger context window. It was giving the central agent less to do. We call this the thin orchestrator pattern: a router that owns sequencing and reliability checks, and leaves every domain decision to specialists. The Problem: More Context, Worse Results Our pipeline is config-as-code: Python configurations drive C++ code generation, which feeds data processing services, which populate analytics systems. Onboarding a single data column requires coordinated changes across configuration registries, backend services, and data pipelines, and each column moves through up to six stages before its data arrives. Our first approach was a single AI agent loaded with all domain knowledge, configuration schemas, code generation rules, pipeline routing logic, validation requirements, and population patterns for 10+ data sources. The agent had everything it needed. That was the problem. Past a certain context threshold, additional knowledge degraded rather than improved performance. The agent would cross-contaminate concepts, applying field naming conventions from one subsystem to another, or using a code generation pattern meant for one data source when populating from a different one. A preprint from the Windreich Department of Artificial Intelligence and Human Health at Mount Sinai found that orchestrated multiagent systems held 65.3% accuracy at 80-task batches where single agents fell to 16.6%, while reducing token usage up to 65-fold. The authors attribute this to the …
Explained
What happened
Meta rebuilt an AI agent that onboards new data columns into its recommendation pipelines. One column touches three repos, three languages and up to six pipeline stages.
- The first version was one agent that knew everything: schemas, code generation rules, routing, validation, and patterns for 10+ data sources. It worked in demos and hallucinated in production, for example applying naming rules from one subsystem to another.
- Another single-agent attempt hit 70 to 80% success only when an engineer wrote a custom prompt per task. Without that, it fell to 10 to 20%.
- The fix was a 1,300-line orchestrator that does zero domain work. It routes 20+ workflows to 14 specialist subagents.
- Every request follows a fixed 7-step protocol. Six steps are deterministic (read the subagent file, check for duplicates, gather fields, infer what it can, check reliability gates, handle results). Only the delegation step uses LLM reasoning, inside a subagent that loads only its own knowledge files.
- New use cases are workflow templates that reorder existing subagents. Pipeline migration shipped with no new agent code.
- Column prep went from 3 to 4 days by hand to under an hour, and about two thirds of changes land without major human edits.
Why it works
More context made the model worse because related but different rules interfered with each other. Splitting knowledge so each subagent only sees its own domain removes the chance to mix them up. Making routing deterministic removes the most dangerous failure: a confused router that sends work to the wrong place or skips checks, which then spreads to every later step.
The bigger pattern
This is the old control plane vs data plane split applied to agents. Keep the coordinator simple and predictable, and put the complex, failure-prone work in isolated workers. It also echoes context rot: past some size, adding context hurts accuracy. The idea of multi-agent systems is not new. The useful part is the evidence from production and the rule of thumb: if a decision can be made with code, do not ask the model.
What it means in practice
- Draw a clear line between deterministic steps (routing, validation, duplicate checks, approvals) and probabilistic ones (writing code or config). Implement the first kind in plain code.
- Give each subagent a small, scoped knowledge set loaded at run time, so updates do not need a redeploy.
- Model workflows as state machines, similar to a job pipeline in Sidekiq or Temporal, where each stage is a specialist worker with retries and escalation.
- Measure success without per-task prompt tuning. If accuracy depends on an engineer writing a custom prompt, the architecture is the problem.