From Chip to Rack: When the Rack Becomes the Computer

The unit of sale has changed. For most of data center history, the unit was the server: a box of processors and memory that slid into a rack alongside dozens of others. The rack was furniture. It held servers, routed cables and carried power strips. The computing happened inside each box.

The previous module explains why that stopped being enough. Large models do not fit on one chip. Chips in a group must exchange data at scale-up speed. Scale-up links are fastest over short distances, and copper cable is cheapest and most reliable over short distances. So the designers pulled the group of chips as close together as physics allowed. The group grew until it filled a rack. The rack became the computer.

The reference case is NVIDIA's GB200 NVL72, announced at NVIDIA's annual developer conference in March 2024. The name encodes the design. GB200 is the superchip: one Grace central processing unit, or CPU, paired with two Blackwell graphics processing units, or GPUs. NVL72 means seventy-two GPUs joined in a single NVLink domain. NVIDIA describes the rack as thirty-six Grace CPUs and seventy-two Blackwell GPUs, connected through NVLink Switch chips, with 130 terabytes per second of GPU bandwidth inside that domain. Its marketing line is that the rack acts as a single large GPU. Strip away the adjective and the engineering claim is precise: any GPU in the rack can reach any other at scale-up speed.

That design has consequences that did not apply to a rack of ordinary servers.

First, the rack is now a single system that ships and fails as one. The backplane of copper cabling that ties the GPUs together is part of the product. It cannot be assembled casually on site from mixed parts.

Second, the rack is liquid cooled by design. NVIDIA describes the GB200 NVL72 as a liquid-cooled rack, and states that liquid cooling is what allows the density and the short, fast connections. Air cannot remove that much heat from that little space. The next lesson follows the heat out of the building.

Third, the rack design became something the industry had to standardise. In October 2024, at the Open Compute Project, or OCP, Global Summit, NVIDIA contributed large portions of the GB200 NVL72 electro-mechanical design to OCP. According to NVIDIA's announcement, the contribution covered the rack architecture, the compute and switch tray mechanicals, the liquid-cooling and thermal environment specifications, and the volumes of the NVLink cable cartridges. OCP is the open hardware body that Facebook founded in 2011 to publish data center designs. At the same event NVIDIA announced a joint reference design for the rack with Vertiv, the power and cooling equipment maker. Chip companies now publish building-services documents. That is the clearest sign that the boundary between chip and facility has moved.

The pattern is not unique to NVIDIA. Advanced Micro Devices, or AMD, unveiled its Helios rack-scale system in June 2025, built around its Instinct MI400 series accelerators, and launched it in July 2026, saying it was in full production, with first shipments at the end of the third quarter of 2026 and volume ramping through the fourth quarter into 2027. Amazon Web Services, or AWS, made its Trainium3 UltraServers generally available in December 2025, joining up to 144 of its own Trainium3 chips in one system. NVIDIA's own successor rack, Vera Rubin NVL72, launched at the Consumer Electronics Show in January 2026 with partner availability announced for the second half of 2026. Different vendors, same structure: the purchase is a rack-scale system with a large scale-up domain inside it.

What does this mean for someone who is not an engineer?

For an investor, the unit of analysis has moved up a level. Chip counts still matter, but the system that customers buy, finance and deploy is the rack. Its delivery depends on cooling equipment, cabling, power shelves and integration capacity at the server makers, not only on chip supply.

For an operator, a rack-scale system is a commitment to the building around it. It arrives needing liquid, high power and a floor strong enough to hold it. A hall designed for a mix of ordinary servers may not be able to host one without work.

For an engineer, the scale-up domain is now a design parameter chosen by the vendor. Seventy-two GPUs in the Blackwell generation, 144 Trainium3 chips in the AWS design. The size of that domain shapes which models run well on the system and how the software splits them.

Faraday's read is that the rack is the new chip. Vendors compete on what they can join together at scale-up speed in one enclosure, and the building has to follow. The question is how far the building can follow before it too has to be redesigned.