The Switch Silicon and Ethernet's Counterattack

Underneath every scale-out fabric, whatever its name, sits a switch chip. It is the least discussed and most decisive component in the cluster network, and the counterattack against the specialist is being fought largely in this silicon. To see the contest clearly, look at the chip and the two numbers that define it.

A switch chip's job is to receive bits on many ports and send them out the right ports, as fast as possible, all at once. Two properties govern its power. The first is total switching bandwidth — the aggregate rate at which the chip can move data across all ports combined, measured in bits per second. Each generation of these chips has roughly doubled this figure, and the leading merchant switch silicon — Broadcom's Tomahawk line is the prominent example — now moves on the order of tens of terabits per second through a single chip. The second property is radix: the number of ports the chip supports. Radix is quietly the more strategic of the two, and here is why.

Radix determines how many nodes you can connect with how few layers of switching. To join more machines than a single switch has ports, you build a tree of switches — nodes connect to switches, those switches connect to higher switches, and so on. Every additional layer a packet must traverse adds latency, adds cost, adds power, and adds more places to drop a packet or create congestion. A higher-radix chip — more ports per switch — connects the same number of nodes in fewer layers. Fewer layers means a shorter, cheaper, lower-latency path between any two accelerators, which means a shorter tail. Faraday's read: radix is the lever that quietly sets the cost and the latency floor of the entire cluster fabric. A doubling of radix can remove a whole layer of switching from a large cluster, and the layer you remove is pure overhead — latency and expense that produced no compute. The switch chip's port count shapes cluster economics more than any spec sheet headline suggests.

Now the counterattack. Ethernet's disadvantages against InfiniBand were specific: it was best-effort rather than lossless, and it lacked InfiniBand's native, tuned RDMA. Its advantages were general and enormous: it is everywhere, made by many vendors, produced in staggering volume, understood by every network engineer, and therefore cheaper and more flexible per bit. The strategy writes itself — do not ask Ethernet to abandon its advantages; teach it the specialist's tricks. Add lossless flow control so it stops dropping packets under collective bursts. Refine RDMA over Ethernet so memory-to-memory transfer works well across it. Improve how it spreads traffic across many paths and manages congestion, so ten thousand nodes bursting at once do not collide into stragglers.

This is the substance of the industry effort to make Ethernet a first-class AI fabric — the Ultra Ethernet initiative, a consortium of many companies collaborating on a version of Ethernet tuned specifically for AI and high-performance workloads. Read it structurally: it is the generalist deliberately acquiring the specialist's capabilities while keeping the generalist's economics. The goal is not for Ethernet to become InfiniBand. It is for Ethernet to become good enough at InfiniBand's job that the specialist's premium can no longer be justified for a growing share of clusters — at which point ubiquity and volume decide the rest.

Why does this contest matter beyond the vendors fighting it? Because the scale-out fabric is the part of the cluster where cost per bit and tail latency are set for the majority of the machines that will ever be built, and those two quantities bound how much usable compute a cluster of a given price can deliver. If Ethernet's counterattack succeeds, the cost of the scale-out tier falls toward commodity levels and large clusters become cheaper to build and to widen. If the specialist holds its lead, the premium persists and shapes who can afford the largest machines. Either way, the outcome is decided in the switch silicon and the protocols around it — in radix, in lossless flow control, in RDMA — not in the accelerator.

Hold the module. The scale-out workload demands lossless delivery and RDMA because synchronous collectives cannot tolerate drops or host-stack latency. InfiniBand is the specialist built for exactly that and became the default of the most demanding clusters. Ethernet, the generalist, is being retooled through the switch silicon and coordinated standards to acquire the same capabilities while keeping its overwhelming economic advantages. And beneath both, the switch chip's radix and bandwidth set the latency floor and the cost per bit of the entire cluster. This is the tier where the price of moving ten thousand accelerators into agreement is decided. There remains one more layer beneath even the switch — the physical medium the bits travel on, which at these speeds is increasingly light. That is the final module.