Skip to content
AWS Machine Learning Blog· Sitaraman Vijay Krishna·· 5d agoAI score45

Amazon Bedrock and LiteLLM Build Self-Service LLM Gateway

Building a self-service LLM gateway with Amazon Bedrock and LiteLLM

AI digest

The self-service LLM gateway built by Amazon Bedrock and LiteLLM allows developers to access foundational models on Amazon Bedrock via a single API, while also achieving user-level cost control and model performance monitoring. The gateway runs LiteLLM on AWS Fargate and provides services through OpenAI-compatible APIs, supporting models such as Claude Sonnet and Claude Haiku.

Source:AWS Machine Learning Blog · aws.amazon.com