Scale-Out: Joining the Cores
Faraday's read: scale-out is where the two great networking traditions collide over the AI workload. InfiniBand was built for exactly this — a lossless, low-latency fabric for machines that must cooperate. Ethernet was built for everything else, and it won everywhere else on ubiquity and price. Now Ethernet is being retooled to do what InfiniBand always did, because the prize is too large to concede. The contest is not really InfiniBand versus Ethernet. It is a specialist versus a generalist learning the specialty. Whoever moves ten thousand accelerators into agreement with the shortest tail and the lowest cost per bit sets the economics of the whole cluster.
Knowledge check
Why does the scale-out fabric for AI require lossless delivery and remote direct memory access, and why is standard best-effort Ethernet — as originally designed — a poor fit for the workload?
Show the answer
AI collective operations are synchronous and bursty: many nodes transmit at once and the step cannot complete until every transfer does, so a single dropped packet forces a retransmission that stalls the whole collective and lengthens the tail. The fabric therefore needs lossless delivery (flow control that prevents drops rather than recovering from them) to keep the tail short, and remote direct memory access (RDMA) to move data between node memories without involving the host processor, which removes latency and CPU overhead. Standard best-effort Ethernet was designed to tolerate and recover from drops rather than prevent them, and to move data through the host networking stack — both acceptable for general traffic but costly for tightly synchronized collectives, which is why Ethernet must be adapted (lossless mechanisms plus RDMA, and the Ultra Ethernet effort) to compete with InfiniBand.