Why Air Runs Out
The Uptime Institute has asked operators about rack power in its annual surveys for years, and the pattern in the responses has been consistent: most racks in most data centers draw modest amounts of power, commonly under ten kilowatts, even as a growing minority of racks draw far more. That minority is the story of this module. To understand why it matters, start with the fluid that has carried the industry's heat for decades, which is air.
Air is a poor carrier of heat. It is light, so a given volume holds little thermal energy. Water, by contrast, carries on the order of three thousand times more heat per unit volume for the same temperature rise. That ratio is physics, not marketing, and it means that to remove a kilowatt of heat with air you must move a great deal of it. Double the power in a rack and, at the same temperature rise, you must roughly double the airflow through it. Fans pay for that airflow, and fan power climbs steeply as speed increases. Air cooling does not fail at a single threshold. It becomes progressively more expensive, noisier and harder to control.
Operators have stretched air a long way. The first improvement was to stop mixing hot and cold air. In a hot aisle and cold aisle layout, servers face each other across a cold aisle and exhaust into a shared hot aisle, and physical containment, with doors and roof panels, keeps the two streams apart. The second was to move the cooling coil closer to the heat, with in-row coolers placed between racks and rear-door heat exchangers bolted to the back of a rack so that hot exhaust passes through a water coil before it reaches the room. A rear-door heat exchanger is a useful halfway house: the server still uses air, but water carries the heat away from the rack.
The standards bodies noticed the pressure. The fifth edition of the Thermal Guidelines published by the American Society of Heating, Refrigerating and Air-Conditioning Engineers, or ASHRAE, which appeared in 2021, added a new air-cooled class, called H1, for high-density equipment. It carries a narrower and cooler recommended range than the general-purpose classes. The logic is straightforward. When a processor sits very close to its temperature limit, there is less margin for warm inlet air, so the room must be kept cooler, which undoes some of the economization savings described in the previous module. ASHRAE was in effect recording that the most powerful air-cooled parts were running out of thermal headroom.
The deeper constraint is at the chip, not the room. NVIDIA rates its Blackwell B200 graphics processing unit, or GPU, at about a kilowatt in its standard server configuration, up from about 700 watts for the Hopper H100 before it, and that heat leaves a package a few centimeters across. The heat must cross from the silicon through a lid and a thermal interface material into a heat sink. With air, the heat sink must be large, with many fins, and the fins take space. Space in a server is contested by memory, power delivery and the high-speed links between chips. As chips got hotter, air-cooled heat sinks grew taller and servers grew thicker, which meant fewer servers per rack. That is the opposite of what the designers of large accelerator systems wanted.
There is also a question of temperature grade. With air, the heat reaches the outdoors only after passing through several exchanges, each costing a few degrees. By the time the building rejects it, the heat is cooler than it started, and more of the year requires refrigeration. A liquid that touches the chip's cold plate can carry heat out at a higher temperature, which makes rejection easier. This point connects the chip-level problem to the facility-level efficiency discussed earlier.
None of this means air is finished. Storage, networking and general-purpose servers still run comfortably on air, and even liquid-cooled racks usually contain components, such as power supplies and some memory, that are cooled by fans. Faraday's read is that the industry is moving toward hybrid halls, in which liquid handles the densest heat and air handles the remainder, rather than toward the disappearance of air. Anyone who predicts the end of the computer room air handler should be asked what will cool the switches.
The practical takeaway is a habit of mind. When you hear a rack density figure, translate it into airflow and fan power, and ask where the heat goes next. A figure that sounds like an incremental step can imply a change of cooling medium, and a change of medium changes the building.