What the CPU Still Does, and What It Stopped Doing

Take the division of labour as the model. In an artificial intelligence server, the central processing unit, or CPU, is the host. The graphics processing unit, or GPU, or another accelerator, is the engine. The host runs the operating system, loads data, schedules work and talks to the network. The engine does the arithmetic. Neither is optional.

Why did the CPU lose the arithmetic job? Amdahl's law gives the frame. Gene Amdahl made the argument in a 1967 conference paper: the speedup you get from parallel hardware is capped by the part of the work that cannot be done in parallel. Most general computing has a large serial part, so a chip that makes serial work fast is the right tool. Neural-network math has a very small serial part. For that workload the cap barely binds, and the chip with the most parallel units wins.

The second reason is energy per operation. A CPU spends energy on decisions: fetching instructions, predicting branches, working out what to do next. That overhead is paid on every instruction. An accelerator spreads it thin. One instruction can command a large block of arithmetic. For repetitive math, more of each watt goes into the arithmetic itself.

The third reason is precision. Scientific computing has traditionally used 64-bit numbers. Neural networks tolerate far less. Training commonly uses 16-bit formats, and inference increasingly uses 8-bit and 4-bit ones. Fewer bits per number means more numbers per unit of memory, more operations per unit of silicon and less energy per operation. Accelerator designers built hardware around these small formats. NVIDIA added dedicated matrix units, which it calls Tensor Cores, with its Volta generation in 2017, and it lists 4-bit floating-point support as a feature of its Blackwell generation, announced in March 2024. General-purpose CPUs were never organised around that trade.

Google's history makes the point from another direction. At its annual developer conference in May 2016, Google announced the tensor processing unit, or TPU, a chip built only for neural-network math, and said it had already been running in its data centers for more than a year. Google did not build a better CPU. It built a narrower chip. That decision has since been copied across the industry.

Now the part of the CPU story that still holds. The host matters more as systems grow. Data has to be staged, fed and moved, and if the host cannot keep pace, the accelerators wait. Idle accelerators are the most expensive idle equipment in the building. That is why NVIDIA pairs its own Grace CPU with its GPUs in the GB200 superchip, joined by a high-speed link, rather than treating the CPU as a commodity part. Advanced Micro Devices, or AMD, which sells both EPYC CPUs and Instinct accelerators, makes the same argument from the other side of the market.

For an investor or operator, the model has three practical readings.

First, count accelerators, but ask about the ratio. A server sold as an eight-GPU box, or a rack sold as a seventy-two-GPU system, tells you the engine count. The CPUs, memory and network around those engines decide whether they run at full load or sit waiting.

Second, the value in the box has moved. Faraday's read is that in a traditional server the CPU and its memory were most of the cost, and that in an accelerated server the accelerators and their memory dominate. It follows that chip roadmaps now move data center budgets: the most expensive component in the building is also the one that changes every year.

Third, the power profile moved too. A fleet of CPU servers draws power fairly evenly, because thousands of unrelated jobs average out. Accelerators working on one synchronised training job can raise and drop their draw together. Grid planners have noticed. In 2024 the North American Electric Reliability Corporation, or NERC, set up a Large Loads Task Force, and in July 2025 it published a white paper on emerging large loads that highlights rapid fluctuations in demand. The chip's work pattern now shows up on the transmission system.

The summary model is compact. General work stays on the CPU. Parallel math goes to the accelerator. The system is the unit that matters, and it performs only as well as its slowest part. Open the accelerator and you find that the slowest part is rarely the arithmetic.