Lossless, RDMA, and the Case for InfiniBand

Scale-out joins the coherent domains into a cluster. The distances are longer, the participants number in the thousands, and the workload is unforgiving in a specific way that dictates the design. To understand the scale-out fabric, understand first what the traffic demands, then judge the technologies against that demand.

The traffic is synchronous and bursty. A collective operation across the cluster means many nodes transmit at nearly the same instant, and the operation cannot finish until every transfer lands. This has two consequences that drive everything. First, latency matters enormously, because the step waits on the last arrival. Second — and this is the subtle one — a single lost packet is catastrophic to the tail. In ordinary networking, a dropped packet is routine: the sender notices, waits, and retransmits, and for a file download nobody cares about the small delay. In a synchronous collective, that retransmission stalls not one transfer but the entire operation, because everyone is waiting on the slow one. A rare drop, harmless in general traffic, becomes a straggler in a collective. The fabric must therefore not merely recover from loss quickly. It must not lose packets at all.

This is the meaning of lossless. A lossless fabric uses flow control: before a sender floods a receiver or a switch buffer past capacity, the fabric signals backward to slow the source, so the packet is never dropped in the first place. Prevention, not recovery. It is a different design philosophy from the one that built the general internet, which assumed loss was normal and made recovery cheap. For AI collectives you want the opposite assumption: loss is intolerable, so architect to prevent it.

The second demand is remote direct memory access — RDMA. Ordinarily, data arriving over a network is handled by the host processor's networking stack: it is copied, processed, and delivered up through layers of software before the application sees it. That path adds latency and consumes host compute. RDMA removes the host from the path. It lets one node's network hardware write directly into another node's memory, and read directly from it, without the host processor's involvement on either end. The data moves memory-to-memory across the fabric. This is the scale-out echo of the memory-semantic trick from scale-up: get the general-purpose host out of the way and let the specialized hardware move data directly between memories. RDMA is what makes the scale-out fabric fast enough and cheap enough in host cycles to keep thousands of nodes fed.

InfiniBand is the fabric built from the start on exactly these principles. It was designed as a lossless, low-latency interconnect with RDMA as a native, first-class capability, for the specific job of making many machines cooperate as one — the high-performance computing lineage from which large-scale AI training descends. Because it was purpose-built for this traffic, it does the job with little adaptation: the lossless flow control, the low latency, the direct memory access are what it is, not features bolted on. This is why InfiniBand became the default fabric of the most demanding training clusters. It is the specialist, and the workload is its specialty.

But the specialist has costs, and they are not only monetary. InfiniBand is a distinct ecosystem — its own switches, its own adapters, its own operational expertise — separate from the Ethernet that runs essentially everything else in the data center and the world. Adopting it means running and staffing a second kind of network, from a narrower supply base, at a price the narrowness supports. For the highest-performance clusters that price has been worth paying. But it creates an obvious opening. Faraday's read: whenever a specialist commands a premium for doing one job supremely well, the generalist that does everything else at scale will come for that job — not by being better, but by becoming good enough while carrying the crushing advantages of ubiquity and volume. That is precisely the pressure now bearing on the scale-out tier, and it is why the more interesting story is not InfiniBand's strength but Ethernet's adaptation. That is the next lesson.