Why Artificial Intelligence Runs on Accelerators
Start with the math, not the chip. Neural networks are matrix multiplication at scale, and matrix multiplication is parallel by construction. Hardware that trades generality for parallel arithmetic wins that job, and it keeps winning as long as the math keeps its shape. The graphics processor won first because it already had the right structure and, from 2006, the software to reach it. Faraday's read: the central processor did not lose a race; it was assigned a different job, host rather than engine. Anyone judging an accelerated system should count engines, then ask what feeds them, because a stalled engine is the costliest idle asset in the building.
Knowledge check
Why does matrix multiplication favour a graphics processing unit over a central processing unit?
Show the answer
Each cell of a matrix product can be computed independently, so the work is almost entirely parallel. A graphics processing unit devotes its silicon to thousands of simple arithmetic units, while a central processing unit devotes much of its silicon to making one instruction stream fast. Amdahl's law says parallel hardware pays off most when the serial part of the work is small, which is the case here.