"SageMaker Inference Now Supports Multi-Instance GPUs to Maximize GPU Utilization at Model Endpoints"
Amazon SageMaker Inference now supports GPU partitioning, using NVIDIA multi-instance GPU (MIG) to divide physical GPUs into independent partitions. Each partition gets a fixed share of GPU compute units and a dedicated memory block, enforced by hardware isolation. This changes two limitations: different models can be placed in each partition, and part of the new generation accelerators can be selected instead of the entire generation of accelerators.
Why it matters: The original text provides changes in capabilities and open entry, allowing readers to judge how it will change the existing workflow.