AI Chips and Compute Density Foundations

Glossary

Accelerator

A chip that speeds up one family of operations, chiefly the matrix math of neural networks, by giving up general-purpose flexibility. Graphics processing units, Google's tensor processing units and Amazon's Trainium chips are all accelerators.

Direct-to-chip liquid cooling

A cooling method in which coolant flows through metal cold plates mounted on the hottest components. A coolant distribution unit transfers the heat to the building's water system.

Graphics processing unit (GPU)

A processor built from thousands of simple arithmetic units working in parallel, originally for shading pixels. Its parallel structure made it the first mainstream accelerator for neural networks.

High-bandwidth memory (HBM)

Memory built from stacked dies placed beside the processor on the same package and connected over a very wide interface. It is standardised by JEDEC; the HBM4 standard was published in April 2025.

Matrix multiplication

The operation of multiplying two grids of numbers to produce a third, in which each cell of the result can be computed independently. It is the dominant arithmetic in both training and running neural networks.

Memory wall

The performance ceiling that appears when processors improve faster than memory can supply them with data, leaving arithmetic units idle. The term comes from William Wulf and Sally McKee's 1995 paper.

Rack power density

The electrical power drawn by, and heat released from, one rack of equipment, usually stated in kilowatts. It drives the design of cooling, power distribution and structure in a data center.

Scale-up domain

The group of accelerators joined by a fast, short-reach link, such as NVIDIA's NVLink, so tightly that software can treat them almost as one large chip. Its size is a design choice set by the vendor, such as seventy-two GPUs in a GB200 NVL72 rack.