Meta Networking at the 2026 OCP Global Summit: Open Innovation for Scale-Up, Scale-Out, and Scale-Across AI Networks
Meta details AI networking at OCP 2026: liquid-cooled racks with 102.4T switches of 128 by 800G ports running FBOSS, plus fabrics for its gigawatt Prometheus cluster.
Extract
At the 2026 Open Compute Project (OCP) Global Summit, Meta shared the networking innovations that are directly supporting our latest AI models, including Muse Wonder and Muse Spark , and our gigawatt-scale Prometheus cluster . Highlights IcePack & IceCube: Meta’s first liquid-cooled network rack with 102.4T switch blades (128×800G, Broadcom Tomahawk 6C and NVIDIA Spectrum-6 variants), running FBOSS. Flattened Planar Fabric (FPF) for scale-out: Meta’s multi-plane scale-out fabric where each GPU connects to eight network planes. 3-tier → 2-tier Clos, NIC-level packet spraying, and a combined host+switch control plane. Built with NVIDIA and shared with our cloud partners. Scale-across: BAG, our petabit-scale super-spine joining buildings, campuses, and regions in our gigawatt-scale Prometheus cluster , plus SRv6 regional backbones on FBOSS + SAI. Ethernet for scale-up : Tahan, the Ethernet scale-up switch for MTIA 300 based on Broadcom Tomahawk5; a preview of 72 accelerators per rack on 102.4T silicon; and ESUN progressing from 1.0 → 1.1. OSPA: a new OCP initiative co-led by Meta, Microsoft, and Alibaba to standardize the switch platform layer . v1.0 spec submitted to OCP. MetaRoCE: our RDMA transport for AI networks , now going open source. We shared insights into key networking-related initiatives through talks from many of the Meta network and hardware teams. Those talks and their slide decks are now available to everyone! Keynote and Executive Sessions Concrete to Code: Engineering AI Infrastructure at Gigawatt Scale ( video ) AI has changed the shape of infrastructure. Santosh Janardhan, Head of Infrastructure at Meta, shares how Meta is designing, building, and operating at a scale that requires rethinking the network, the facility, and the systems that hold it all together. Speaker: Santosh Janardhan, Head of Infrastructure, Meta Betting on the Network: Meta’s Case for Full-Stack, Open AI Networking ( video ) AI progress now depends on the network as much as on compute. The path forward is a full-stack, open approach built in the open ecosystem rather than behind proprietary walls. Speaker: Omar Baldonado, Senior Director, Data Center & AI Networking, Meta Breaking the Walls: Open Scale-Across for the AI Era ( video ) Training compute is compounding faster than infrastructure can scale. This talk lays out why open scale-across networking is needed to connect AI capacity across data centers and regions. Speakers: Vipul Deokar, VP, Core …
Explained
What happened
At the 2026 OCP Global Summit, Meta detailed the network stack behind its gigawatt-scale Prometheus AI cluster. The pieces, grouped by how AI networks are now layered:
- Switch hardware.
IceCube, a 102.4T switch (128 x 800G, Broadcom Tomahawk 6C or NVIDIA Spectrum-6), in the liquid-cooledIcePackrack, PHY-less and running Meta FBOSS. - Scale-up (GPUs within a rack):
Tahan, an Ethernet scale-up switch for the MTIA 300 accelerator, plus theESUNeffort (Ethernet for Scale-Up Networking) moving from 1.0 to 1.1, and a preview of 72 accelerators per rack. - Scale-out (across racks):
Flattened Planar Fabric, where each GPU connects to eight network planes, collapsing a 3-tier Clos to 2-tier, with NIC-level packet spraying and a combined host-plus-switch control plane. - Scale-across (across buildings, campuses, regions):
BAG(Backend Aggregation), a petabit super-spine, plusSRv6regional backbones on FBOSS, deliberately using shallow buffers at WAN distance. - Openness:
MetaRoCERDMA transport going open source, andOSPA, a Meta/Microsoft/Alibaba initiative to standardize the proprietary switch-platform layer that SAI left fragmented.
Why it works
AI training traffic is dense, bursty and all-to-all, so one flat network cannot serve it. Meta splits the problem into three tiers with different physics: microsecond, ultra-high-bandwidth scale-up inside a rack; packet-sprayed multi-plane scale-out across a hall; and long-haul scale-across between sites. Betting on Ethernet everywhere, including scale-up where proprietary interconnects dominated, lets one software stack (FBOSS plus SAI) and one supply chain span all three, which is why standardizing the platform layer (OSPA) and the transport (ESUN, MetaRoCE) matters as much as the silicon.
The bigger pattern
This is the industry consolidating on open Ethernet for AI fabrics against proprietary stacks, and on multi-plane, multi-tier topologies as the way to scale past a single building when compute demand compounds faster than any one datacenter can hold. Packet spraying plus shallow-buffer WAN design reflects a shift from TCP-style congestion assumptions to fabric-level load balancing. The deeper idea: at gigawatt scale the network, not the GPU, is often the bottleneck and the reliability risk.
What it means in practice
- For large-scale AI, design the network in tiers with distinct goals: bandwidth and latency inside the rack, load-balanced many-path fabric across racks, and routed, buffer-aware transport across regions. One uniform design will not fit all three.
- Packet spraying across many planes beats single-path hashing for all-to-all collectives. If you run GPU training, look at whether your fabric spreads a flow across paths or pins it to one.
- Watch the scale-up interconnect decision. Ethernet-based scale-up (
ESUN,Tahan) is maturing as an open alternative to proprietary links, which affects vendor lock-in and cost. - Standard, config-driven network software (FBOSS, SAI, OSPA) cuts per-platform bring-up from months. The same lesson applies broadly: a portable control plane over commodity hardware beats bespoke stacks as fleets grow.
- Reliability is a network property at this scale. Operating the largest clusters means treating drivers, collective libraries, FBOSS and the scheduler as one system, not independent layers.