The Machines: Custom Silicon, Density, and the Thermal Envelope
An accelerator is not a computer; it is a machine tool built to do one thing — dense matrix arithmetic — at massive scale, and its usable output is set by how well it is fed and cooled, never by its arithmetic peak alone. The memory wall means the array starves unless high-bandwidth memory, on-chip reuse, and reduced precision keep operands flowing. Density then decides how much of that silicon can run at full clock, and density is governed by two utilities that never make the spec sheet: power delivery and heat removal. Liquid cooling and efficient power conversion are not overhead — they are the services that convert rated capacity into real tokens-per-watt.
Knowledge check
What is the "memory wall," and why does it make high-bandwidth memory and reduced numeric precision central to accelerator design?
Show the answer
The memory wall is the gap between how fast an accelerator can compute and how fast memory can supply operands: the arithmetic array can multiply far faster than ordinary memory can feed it, so it idles. High-bandwidth memory, stacked in-package with wide interfaces, narrows the gap by delivering operands quickly. Reduced precision narrows it further — fewer bits per number means more numbers moved per unit of bandwidth and more arithmetic per watt — so both are direct levers on keeping the machine tool fed.