Catalog · Orchestration Intelligence & Control Plane

Inference Serving & Model-Routing Infrastructure

What’s inside

1. Module 1

   The Inference Serving Stack

   • Serving Is a Scheduling Problem: Engines and Continuous Batching · 3 min · narrated
   • The Key-Value Cache: Memory Reuse, Paging, and Prefix Sharing · 3 min · narrated
   • Module review

2. Module 2

   Model Routing Across Models and Hardware Tiers

   • Routing Across Models: Cascades, Classifiers, and Quality-Cost Frontiers · 3 min · narrated
   • Routing Across Hardware Tiers: Heterogeneous Fleets and Accelerator Classes · 3 min · narrated
   • Module review

3. Module 3

   Token-Economics-Aware Scheduling

   • The Economics of a Token: Prefill, Decode, and Cost per Token · 3 min · narrated
   • Scheduling Against Power and Price: SLO-Aware and Power-Aware Serving · 3 min · narrated
   • Module review

4. Module 4

   Inference as the Dominant Load and Data-Center Design

   • Why Inference Overtakes Training as the Dominant Load · 3 min · narrated
   • How Serving Infrastructure Shapes Data-Center Design and Utilization · 3 min · narrated
   • Module review