AWS reviewed 13 Amazon SageMaker AI inference capabilities launched in 2026 across two paths: managed endpoints for lower operational overhead and Kubernetes-native HyperPod Inference for greater control. The update spans inference recommendations, capacity-aware instance pools, OpenAI-compatible APIs, container caching, observability, inline async payloads, KV caching, data capture, disaggregated prefill and decode, model caching, and prefix-aware routing. AWS's latency and throughput figures come from examples or benchmarks and should not be treated as guarantees for every workload.

Editor's note

This summary is based on the primary source. Open the original for full context, data, and qualifying details.