Agents1 min read
Amazon SageMaker AI shipped 13 inference launches in year-to-date across two deployment paths: fully managed endpoints and Amazon SageMaker HyperPod Inference. This post reviews each launch, from inference recommendations and capacity-aw...
By OpenSmartRoute editorial · attributed excerpt
Amazon SageMaker AI shipped 13 inference launches in year-to-date across two deployment paths: fully managed endpoints and Amazon SageMaker HyperPod Inference. This post reviews each launch, from inference recommendations and capacity-aware instance pools to tiered KV caching and disaggregated prefill and decode.
Read the original at AWS machine learning blog: Amazon SageMaker Inference: 2026 year-to-date launches in review
Keep reading
Agents1 min read
Deploy production-ready Hugging Face models on Amazon SageMaker AI using six open-source agent skills. Point a coding agent at a model and get back a real-time endpoint with the right serving container, autoscaling, Amazon CloudWatch ala...
Agents1 min read
Amazon SageMaker HyperPod Inference Gateway is a Kubernetes-native, GPU-aware routing add-on for Amazon EKS. It uses real-time GPU signals to send each inference request to the best-suited pod, cutting first-token latency by up to 82% wi...
Source
AWS machine learning blog
Amazon SageMaker Inference: 2026 year-to-date launches in review
Read the original
Agents1 min read
Moonshot AI launched Kimi K3 on Amazon Bedrock as its largest open model.
