LiteLLM

LiteLLM is an AI Gateway that simplifies access to 100+ LLM providers by offering a unified OpenAI-compatible API. It enables platform teams to provide developers with model access while managing cost, spend tracking, budgets, rate limits, and fallbacks.

Key Features

• Unified API: Interact with models from OpenAI, Azure, Gemini, Bedrock, Anthropic, and more through a single endpoint.
• Spend Tracking: Attribute costs to keys, users, teams, or organizations; track spend across providers; use tag-based tracking.
• Budgets & Rate Limits: Set usage limits and rate limits per key or team.
• LLM Fallbacks: Automatically fallback to alternate models on failure.
• Load Balancing: Distribute requests across multiple LLM instances.
• Guardrails: Apply safety and content moderation checks.
• Observability: Integrate with Langfuse, Arize Phoenix, Langsmith, and OpenTelemetry for logging.
• Open Source: Free self-hosted version with all core features; enterprise tier for advanced management and support.

Use Cases

• Platform teams that need to manage LLM access across multiple teams and projects.
• Enterprises requiring centralized cost control and observability for LLM usage.
• Developers who want a consistent interface to experiment with different models without provider-specific code.

Key Benefits

• Supports 100+ LLM providers in a unified API
• OpenAI-compatible, reducing integration effort
• Built-in spend tracking and budgets for cost management
• Free open source version with enterprise upgrade option