AWS Machine Learning Blog· Shantanu Tripathi·· 11d agoFeaturedAI score70
"SageMaker Inference Now Supports Multi-Instance GPUs to Maximize GPU Utilization at Model Endpoints"
SageMaker Inference now supports Multi-Instance GPU to maximize GPU utilization for model endpoints
AI digest
Amazon SageMaker Inference now supports GPU partitioning, using NVIDIA multi-instance GPU (MIG) to divide physical GPUs into independent partitions. Each partition gets a fixed share of GPU compute units and a dedicated memory block, enforced by hardware isolation. This changes two limitations: different models can be placed in each partition, and part of the new generation accelerators can be selected instead of the entire generation of accelerators.
Why it matters
The original text provides changes in capabilities and open entry, allowing readers to judge how it will change the existing workflow.
Source:AWS Machine Learning Blog · aws.amazon.com