From Chip to Rack to Building
The rack is the new chip. Scale-up links work best over short copper runs, so vendors pack as many accelerators as physics allows into one enclosure and sell the enclosure as a single system. NVIDIA's GB200 NVL72, AMD's Helios and Amazon's Trainium3 UltraServers share that structure. The consequence lands on the building: a rack drawing well over a hundred kilowatts needs liquid cooling, higher distribution voltage and a floor that can carry it. Faraday's read: facilities should now be judged by the rack density they can reach and what that costs, because the roadmap sets the date and the building sets the price.
Knowledge check
Why did the rack, rather than the individual server, become the unit that artificial intelligence systems are designed and sold as?
Show the answer
Large models must be split across many accelerators that exchange data constantly, and scale-up links such as NVLink are fastest and cheapest over short copper distances. Vendors therefore packed as many accelerators as possible into one enclosure joined by a single scale-up domain, as in NVIDIA's GB200 NVL72 with seventy-two GPUs. The rack ships, cools and fails as one system, so it is now the practical unit of design and purchase.