Quiz

Question 1 of 10

What is the primary advantage of continuous batching over static batching in an inference serving engine?

• AIt permanently reduces the size of the model weights stored in memory
• BIt frees a finished sequence's slot and admits a waiting request at each decode step, so the batch never sits half-empty
• CIt eliminates the decode phase entirely by precomputing all output tokens
• DIt routes every request to a smaller model to save cost