The Limits of Air
Air cooling did not fail at a line. It became steadily more expensive, louder and bulkier as chips got hotter, and the operators who stretched it with containment and rear-door coils bought the industry years. What ended the argument for the densest racks was not a cooling decision at all. It was a networking decision: accelerator makers wanted many chips tightly coupled over short copper links, which meant packing them into one rack, which set a power density that air could not handle sensibly. Faraday's read: cooling is downstream of system architecture. To forecast a building's cooling needs, watch what the chip and system vendors decide about how many processors must sit side by side.
Knowledge check
Why did the GB200 NVL72 have to be so dense in the first place?
Show the answer
Its GPUs communicate over NVLink, a very fast interconnect that runs over copper inside the rack, and copper at those speeds works only over short distances. Keeping the chips close together concentrated roughly 120 kilowatts in one rack, as reported at launch, and that density required liquid cooling.