Quiz

Question 1 of 10

Why is the usable compute of a large training cluster bounded by communication rather than by the peak throughput of its accelerators?

• AAccelerators are slower than their marketed peak throughput
• BTraining is synchronous: every step waits on a collective exchange, so the step runs at the pace of the slowest communication, not the average compute
• CNetworks are always faster than processors, so they dominate
• DCommunication and computation are unrelated to cluster speed