AWS Machine Learning Blog· Sitaraman Vijay Krishna·· 5d agoAI score45
Amazon Bedrock and LiteLLM Build Self-Service LLM Gateway
Building a self-service LLM gateway with Amazon Bedrock and LiteLLM
AI digest
The self-service LLM gateway built by Amazon Bedrock and LiteLLM allows developers to access foundational models on Amazon Bedrock via a single API, while also achieving user-level cost control and model performance monitoring. The gateway runs LiteLLM on AWS Fargate and provides services through OpenAI-compatible APIs, supporting models such as Claude Sonnet and Claude Haiku.
Source:AWS Machine Learning Blog · aws.amazon.com