# LLM Gateway > LLM Gateway is an open-source, OpenAI-compatible API gateway that routes, manages, and analyzes LLM requests across 40+ providers (OpenAI, Anthropic, Google, and more) through a single unified API. Switch providers without changing code, manage API keys centrally, track usage and cost, add caching and guardrails, and self-host or use the managed cloud. ## Key facts - One OpenAI-compatible API for 40+ providers and 200+ models. - Migrate by changing only the base URL (`https://api.llmgateway.io/v1`) and your API key — no code rewrites. - Open source (AGPLv3 core) with a managed cloud option; self-hosting supported via Docker. - Built-in usage analytics, per-model/provider cost breakdowns, automatic routing, fallbacks, caching, and guardrails. - API base URL: `https://api.llmgateway.io/v1` · Docs: https://docs.llmgateway.io · Site: https://llmgateway.io ## Product pages - [Home](https://llmgateway.io): Unified API for multiple LLM providers. - [Models](https://llmgateway.io/models): Browse 200+ supported models with pricing and capabilities. - [Providers](https://llmgateway.io/providers): All supported LLM providers. - [Pricing](https://llmgateway.io/pricing): Plans and pricing. - [Enterprise](https://llmgateway.io/enterprise): Self-hosting, SSO, and team features. - [Token Cost Calculator](https://llmgateway.io/token-cost-calculator): Estimate and compare LLM costs across models. - [LLM Gateway vs LiteLLM](https://llmgateway.io/compare/litellm) - [LLM Gateway vs OpenRouter](https://llmgateway.io/compare/open-router) - [LLM Gateway vs Portkey](https://llmgateway.io/compare/portkey) ## Getting Started - [Introduction to LLM Gateway](https://docs.llmgateway.io/): LLM Gateway is an open-source API gateway for Large Language Models. Route requests to multiple providers, manage API keys, track usage, and optimize costs. - [Overview](https://docs.llmgateway.io/overview): Introduction to LLM Gateway, an open-source API gateway for LLMs. - [Quickstart](https://docs.llmgateway.io/quick-start): Fastest way to start using LLM Gateway in any language or framework. - [Image Generation with the AI SDK](https://docs.llmgateway.io/developers/ai-sdk-images): Generate images with generateImage or stream image output through chat using the AI SDK - [Using the AI SDK](https://docs.llmgateway.io/developers/ai-sdk): Generate text, stream responses, and call tools with the Vercel AI SDK and LLM Gateway - [Overview](https://docs.llmgateway.io/developers): Build AI applications on LLM Gateway with the AI SDK - [AWS](https://docs.llmgateway.io/self-host/aws): Deploy LLM Gateway on AWS with EKS, RDS for Postgres, ElastiCache, and Secrets Manager. - [Azure](https://docs.llmgateway.io/self-host/azure): Deploy LLM Gateway on Azure with AKS, Azure Database for PostgreSQL, Azure Cache for Redis, and Key Vault. - [Docker Compose](https://docs.llmgateway.io/self-host/docker-compose): Run LLM Gateway with Docker Compose — each service in its own container for more control. - [Docker](https://docs.llmgateway.io/self-host/docker): Run LLM Gateway as a single Docker container — the fastest way to a working instance. - [Google Cloud](https://docs.llmgateway.io/self-host/gcp): Deploy LLM Gateway on Google Cloud with GKE, Cloud SQL, Memorystore, and Secret Manager. - [Self Host LLM Gateway](https://docs.llmgateway.io/self-host): Deploy LLM Gateway on your own infrastructure with Docker, Docker Compose, or Kubernetes on AWS, GCP, and Azure. - [Kubernetes](https://docs.llmgateway.io/self-host/kubernetes): Deploy LLM Gateway to any Kubernetes cluster with the official Helm chart. ## Features - [Anthropic API Compatibility](https://docs.llmgateway.io/features/anthropic-endpoint): Use the Anthropic-compatible endpoint to access any LLM model through the familiar Anthropic API format. - [API Keys & IAM Rules](https://docs.llmgateway.io/features/api-keys): Comprehensive guide to API key management and Identity Access Management (IAM) rules for fine-grained access control - [Audit Logs](https://docs.llmgateway.io/features/audit-logs): Track all organization activity with comprehensive audit logs - [Coding Agents](https://docs.llmgateway.io/features/coding-agents): Supported coding agents for DevPass plans and how to configure your tool. - [Compliance](https://docs.llmgateway.io/features/compliance): Restrict routing to providers that meet your compliance requirements (SOC 2, ISO 27001, GDPR, no prompt training, no prompt logging) - [Cost Breakdown](https://docs.llmgateway.io/features/cost-breakdown): Get real-time cost information for each API request directly in the response. - [Custom Providers](https://docs.llmgateway.io/features/custom-providers): Learn how to integrate custom OpenAI-compatible providers with LLMGateway for enhanced flexibility and control. - [Data Retention](https://docs.llmgateway.io/features/data-retention): Store and access your full request and response data for debugging, analytics, and compliance. - [Document Reading](https://docs.llmgateway.io/features/documents): Learn how to send PDFs and other document data to document-capable models. - [Embeddable Payments](https://docs.llmgateway.io/features/embeddable-payments): Embed end-user payments and sessions into your own site with the Payments SDK. Your end-users get their own wallet, buy credits, and pay per request — billed through LLM Gateway, with your markup as margin. - [Embeddings](https://docs.llmgateway.io/features/embeddings): Generate vector embeddings using OpenAI-compatible embedding models. - [Guardrails](https://docs.llmgateway.io/features/guardrails): Protect your LLM usage with content guardrails that detect and block harmful content - [Image Generation](https://docs.llmgateway.io/features/image-generation): Generate images using AI models through the OpenAI-compatible images API or chat completions API - [Master Keys](https://docs.llmgateway.io/features/master-keys): Provision projects and gateway API keys programmatically with org-scoped bearer tokens (Enterprise only) - [Metadata](https://docs.llmgateway.io/features/metadata): Send additional context and metadata to LLM Gateway using custom headers. - [Moderations](https://docs.llmgateway.io/features/moderations): Classify unsafe text and image inputs with the OpenAI-compatible moderations API - [OCR](https://docs.llmgateway.io/features/ocr): Extract text and structure from documents and images as markdown with the OCR API - [Realtime API](https://docs.llmgateway.io/features/realtime): Low-latency speech-to-speech conversations over WebSockets through the OpenAI-compatible realtime API - [Reasoning](https://docs.llmgateway.io/features/reasoning): Learn how to use reasoning-capable models that show their step-by-step thought process. - [Rerank](https://docs.llmgateway.io/features/rerank): Reorder candidate documents by relevance to a query with the rerank API - [Response Healing](https://docs.llmgateway.io/features/response-healing): Automatically repair malformed JSON responses from AI models. - [Routing](https://docs.llmgateway.io/features/routing): Learn how LLMGateway intelligently routes your requests to the best available models and providers. - [Service Tiers](https://docs.llmgateway.io/features/service-tiers): Trade latency against cost on supported OpenAI and Google models with Flex and Priority processing tiers. - [Sessions](https://docs.llmgateway.io/features/sessions): Group related requests into a session to keep provider routing sticky and to observe a conversation end-to-end in the activity log. - [Source Attribution](https://docs.llmgateway.io/features/source): Use the X-Source header to identify your domain for public usage statistics. - [Speech Generation](https://docs.llmgateway.io/features/speech-generation): Generate speech (text-to-speech) using ElevenLabs, Gemini, OpenAI, and Qwen models through the OpenAI-compatible audio API - [Request Timeouts](https://docs.llmgateway.io/features/timeouts): Per-request time limits on the gateway, how they interact with streaming and agentic workloads, and how to change them. - [Transcription](https://docs.llmgateway.io/features/transcription): Transcribe audio into text (speech-to-text) with word-level timestamps through the audio transcriptions API - [Video Generation](https://docs.llmgateway.io/features/video-generation): Generate videos with an OpenAI-compatible async API and signed completion callbacks - [Vision Support](https://docs.llmgateway.io/features/vision): Learn how to send images to vision-enabled models using URLs or inline base64 data. - [Native Web Search](https://docs.llmgateway.io/features/web-search): Enable real-time web search capabilities to get up-to-date information from the internet. - [Gateway Caching](https://docs.llmgateway.io/features/caching/gateway-caching): Serve byte-identical requests entirely from LLM Gateway at $0 cost. - [Caching](https://docs.llmgateway.io/features/caching): Overview of the two types of caching available in LLM Gateway. - [Provider Cache Control](https://docs.llmgateway.io/features/caching/provider-cache-control): Use provider-side prompt caching to reduce the cost of long, reused prompts in chat and coding tools. - [Microsoft Entra ID](https://docs.llmgateway.io/features/sso/entra): Set up SAML single sign-on and SCIM provisioning for your organization with Microsoft Entra ID (formerly Azure AD). - [Google Workspace](https://docs.llmgateway.io/features/sso/google): Let members of your Google Workspace domain auto-join your organization when they sign in with Google — no SAML app or SCIM required. - [SSO](https://docs.llmgateway.io/features/sso): Per-organization SAML single sign-on and SCIM directory provisioning for enterprise teams (Enterprise only). - [Okta](https://docs.llmgateway.io/features/sso/okta): Set up SAML single sign-on and SCIM provisioning for your organization with Okta. ## Guides & AI Tooling - [Agent Skills](https://docs.llmgateway.io/guides/agent-skills): Packaged instructions and guidelines for AI coding agents - [Autohand Code Integration](https://docs.llmgateway.io/guides/autohand): Use GPT-5, Claude, Gemini, or any model with Autohand Code's autonomous coding agent. Simple config, full cost tracking. - [Claude Code Integration](https://docs.llmgateway.io/guides/claude-code): Use GPT-5, Gemini, or any model with Claude Code. Three environment variables, full cost tracking. - [LLM Gateway CLI](https://docs.llmgateway.io/guides/cli): Command-line tool for launching coding agents, scaffolding projects, and managing keys, budgets, and usage - [Cline Integration](https://docs.llmgateway.io/guides/cline): Use LLM Gateway with Cline for AI-powered coding assistance in VS Code - [Codex CLI Integration](https://docs.llmgateway.io/guides/codex-cli): Use any model with OpenAI's Codex CLI through LLM Gateway. One config file, full cost tracking. - [Continue CLI Integration](https://docs.llmgateway.io/guides/continue): Use any model with Continue CLI through LLM Gateway. One config file, 200+ models, full cost tracking. - [Cursor Integration](https://docs.llmgateway.io/guides/cursor): Use LLM Gateway with Cursor IDE — plan mode and agent mode both route through the gateway. Tab autocomplete and inline edit stay on Cursor's backend. - [DevPass Code](https://docs.llmgateway.io/guides/devpass-code): DevPass Code is a terminal coding agent that talks only to LLM Gateway. One-click browser login, ~190 models, and DevPass subscription billing handled automatically. - [Hermes Agent Integration](https://docs.llmgateway.io/guides/hermes-agent): Use any model with Hermes Agent through LLM Gateway. One config change, full cost tracking, 200+ models. - [Kilo Code Integration](https://docs.llmgateway.io/guides/kilo-code): Use LLM Gateway with Kilo Code in VS Code. Built-in provider — just search, connect your API key, and start coding. - [Kimi Code Integration](https://docs.llmgateway.io/guides/kimi-code): Use GPT-5, Claude, Gemini, or any model with Kimi Code CLI. Custom provider configuration, full cost tracking. - [Model Context Protocol (MCP)](https://docs.llmgateway.io/guides/mcp): Use LLM Gateway as an MCP server for Claude Code, Cursor, and other MCP-compatible clients - [MiMo Code Integration](https://docs.llmgateway.io/guides/mimocode): Use GPT-5.5, Claude, Gemini, or any model with MiMo Code. Custom provider configuration, full cost tracking. - [N8n Integration](https://docs.llmgateway.io/guides/n8n): Connect n8n workflow automation to LLM Gateway for AI-powered workflows - [OpenClaw Integration](https://docs.llmgateway.io/guides/openclaw): Use GPT-5.4, Claude Opus, Gemini, or any model with OpenClaw across Discord, WhatsApp, Telegram, and more - [OpenCode Desktop Integration](https://docs.llmgateway.io/guides/opencode-desktop): Connect OpenCode Desktop to 200+ models through LLM Gateway. No config files — just open Settings, connect, and start building. - [OpenCode Integration](https://docs.llmgateway.io/guides/opencode): Connect OpenCode to 200+ models through LLM Gateway's built-in provider. No config files needed — just select, authenticate, and code. - [Pi Integration](https://docs.llmgateway.io/guides/pi): Use any model with Pi coding agent through LLM Gateway — GPT-5.5, Gemini, Claude, DeepSeek, and 200+ others in your terminal. ## Provider Integrations - [AWS Bedrock Integration](https://docs.llmgateway.io/integrations/aws-bedrock): Connect AWS Bedrock to LLM Gateway for access to foundation models - [Azure Integration](https://docs.llmgateway.io/integrations/azure): Connect Azure to LLM Gateway for enterprise-grade OpenAI models - [Vertex AI Anthropic Integration](https://docs.llmgateway.io/integrations/vertex-anthropic): Connect Google Cloud Vertex AI to LLM Gateway to run Claude models on your own GCP project ## Platform & Dashboard - [Activity](https://docs.llmgateway.io/learn/activity): View and inspect every API request made through LLM Gateway - [Agents](https://docs.llmgateway.io/learn/agents): Monitor your AI coding agents and their activity - [Analytics](https://docs.llmgateway.io/learn/analytics): Break down cost, requests, and tokens by model for any project - [API Keys](https://docs.llmgateway.io/learn/api-keys): Create, limit, and control API keys for authenticating with LLM Gateway - [Audit Logs](https://docs.llmgateway.io/learn/audit-logs): Track every action taken within your organization - [Billing](https://docs.llmgateway.io/learn/billing): Manage your credits, subscription plan, and payment methods - [Lounge Memberships](https://docs.llmgateway.io/learn/chat-plans): Monthly membership plans for Lounge, the LLM Gateway chat app - [Dashboard](https://docs.llmgateway.io/learn/dashboard): Your central hub for monitoring LLM usage, costs, and performance - [Guardrails](https://docs.llmgateway.io/learn/guardrails): Configure content safety rules to protect your LLM usage - [Introduction](https://docs.llmgateway.io/learn): Learn how to navigate and use the LLM Gateway dashboard - [Master Keys](https://docs.llmgateway.io/learn/master-keys): Provision projects and API keys programmatically with org-scoped master keys - [Member Analytics](https://docs.llmgateway.io/learn/member-analytics): See organization usage and cost broken down per team member - [Model Categories & Fair Use](https://docs.llmgateway.io/learn/model-categories): How models are categorized and how premium fair-use caps work - [Model Usage](https://docs.llmgateway.io/learn/model-usage): Track usage breakdown by individual model - [Organization Analytics](https://docs.llmgateway.io/learn/org-analytics): Roll cost, requests, and tokens up across every project in your organization, broken down by model, project, or API key - [Org Preferences](https://docs.llmgateway.io/learn/org-preferences): Manage your organization's name, billing email, and billing details - [Audio Studio](https://docs.llmgateway.io/learn/playground-audio): Generate speech from text using AI voices - [Group Chat](https://docs.llmgateway.io/learn/playground-group): Watch multiple AI models discuss and collaborate on your prompt - [Image Studio](https://docs.llmgateway.io/learn/playground-image): Generate and edit images using AI models - [Voice Calls](https://docs.llmgateway.io/learn/playground-realtime): Have a live speech-to-speech conversation with a model in the browser - [Video Studio](https://docs.llmgateway.io/learn/playground-video): Generate videos using AI models - [Lounge Chat](https://docs.llmgateway.io/learn/playground): Chat with any LLM interactively in Lounge, the LLM Gateway chat app - [Policies](https://docs.llmgateway.io/learn/policies): Configure data retention and other organization policies - [Preferences](https://docs.llmgateway.io/learn/preferences): Configure project-level settings including caching and project mode - [Provider Keys](https://docs.llmgateway.io/learn/provider-keys): Bring your own provider API keys to use without additional fees - [Referrals](https://docs.llmgateway.io/learn/referrals): Earn credits by referring other users to LLM Gateway - [Reset Passes](https://docs.llmgateway.io/learn/reset-passes): Instantly restore your DevPass weekly premium allowance - [Payments SDK](https://docs.llmgateway.io/learn/sdk-settings): Configure end-user sessions and platform secret keys for the Payments SDK (Embeddable Payments) - [Security Events](https://docs.llmgateway.io/learn/security-events): Monitor guardrail violations and content policy events - [Team](https://docs.llmgateway.io/learn/team): Manage team members and their roles within your organization - [Transactions](https://docs.llmgateway.io/learn/transactions): View your complete payment and credit history - [Usage & Metrics](https://docs.llmgateway.io/learn/usage-metrics): Detailed analytics for requests, models, errors, caching, and costs ## Migration Guides - [Migrate from LiteLLM](https://docs.llmgateway.io/migrations/litellm): Switch from self-hosted LiteLLM to managed LLM Gateway. Same API format, zero infrastructure to maintain. - [Migrate from OpenRouter](https://docs.llmgateway.io/migrations/openrouter): Switch to LLM Gateway for built-in analytics, self-hosting options, and simpler API. Two-line code change. - [Migrate from Vercel AI Gateway](https://docs.llmgateway.io/migrations/vercel-ai-gateway): Keep your Vercel AI SDK code, add response caching, detailed analytics, and smart routing. One provider for all models. ## Resources - [Error Handling](https://docs.llmgateway.io/resources/error-handling): How LLMGateway returns errors in an OpenAI-compatible format on the OpenAI-compatible endpoints. - [Rate Limits](https://docs.llmgateway.io/resources/rate-limits): Understanding rate limits for free and paid models on LLMGateway. ## API Reference - [Health check](https://docs.llmgateway.io/health): Health check endpoint. - [Create speech](https://docs.llmgateway.io/v1_audio_speech): >- - [Create transcription](https://docs.llmgateway.io/v1_audio_transcriptions): >- - [Chat Completions](https://docs.llmgateway.io/v1_chat_completions): Create a completion for the chat conversation - [Embeddings](https://docs.llmgateway.io/v1_embeddings): >- - [Edit image](https://docs.llmgateway.io/v1_images_edits): Creates an edited image from one or more source images and a prompt. - [Create image](https://docs.llmgateway.io/v1_images_generations): >- - [Retrieve key status](https://docs.llmgateway.io/v1_key_retrieve): >- - [Anthropic Messages](https://docs.llmgateway.io/v1_messages): Create a message using Anthropic's API format - [Models](https://docs.llmgateway.io/v1_models): List all available models - [Moderations](https://docs.llmgateway.io/v1_moderations): Classify text or multimodal inputs with OpenAI moderation. - [OCR](https://docs.llmgateway.io/v1_ocr): >- - [Rerank](https://docs.llmgateway.io/v1_rerank): >- - [Video content](https://docs.llmgateway.io/v1_videos_content): >- - [Create video](https://docs.llmgateway.io/v1_videos_create): >- - [Video log content](https://docs.llmgateway.io/v1_videos_log_content): >- - [Retrieve video](https://docs.llmgateway.io/v1_videos_retrieve): Retrieves the current state of a video generation job.