In 2024, "LLM gateway" was a niche concept. By 2026, it's standard infrastructure for any serious AI application. If you're evaluating your options, here's an honest guide to the four leading tools.
What Is an LLM Gateway?
An LLM gateway sits between your app and LLM providers like OpenAI, Anthropic, and Groq. It gives you one OpenAI-compatible endpoint that handles routing, failover, cost tracking, and authentication. Your app talks to one URL; the gateway handles provider complexity behind the scenes.
The Four We're Comparing
- DeltaLLM — open-source, Python/FastAPI, self-hosted, MCP gateway, Prompt Registry, full RBAC
- LiteLLM — open-source, Python, 100+ providers, managed cloud option
- Portkey — open-source gateway + managed cloud, strong dashboard and prompt tooling
- Helicone — observability-first gateway with routing, prompt tooling, and self-hosting
Feature Comparison
| Feature | DeltaLLM | LiteLLM | Portkey | Helicone |
|---|---|---|---|---|
| OpenAI-compatible API | ✓ | ✓ | ✓ | ✓ |
| Self-hostable | ✓ | ✓ | ✓ (gateway) | ✓ |
| Managed cloud | — | ✓ (paid) | ✓ (paid) | ✓ (paid) |
| Provider count | 100+ | 100+ | Many | 100+ |
| Failover / routing | ✓ | ✓ | ✓ | ✓ |
| Virtual keys | ✓ | ✓ | ✓ | API keys + provider keys |
| Spend tracking | ✓ | ✓ | ✓ | ✓ |
| Dashboard | Self-hosted | Self-hosted + cloud | Cloud | Core focus |
| Guardrails | ✓ | ✓ | ✓ | — |
| MCP gateway | ✓ | No first-party docs | ✓ | — |
| Prompt management | ✓ (Prompt Registry) | — | ✓ | ✓ |
| Primary language | Python | Python | TypeScript | TypeScript |
DeltaLLM
DeltaLLM is a Python gateway focused on being a complete self-hosted control plane. It combines MCP gateway support, a full Prompt Registry, granular RBAC (platform → org → team → user), guardrails, and a clean admin dashboard in one package. Every request stays on infrastructure you control.
Best for: Teams with compliance or data residency needs, teams building AI agents with MCP, and organizations that want a full admin dashboard without a cloud dependency.
Honest limitation: Newer project with a smaller community. If ecosystem maturity matters more than self-hosted features, that's worth weighing.
LiteLLM
LiteLLM is the most mature and widely-used open-source LLM proxy. It supports 100+ providers, has a large active community, and BerriAI offers a managed cloud product for hands-off infrastructure.
Best for: Python teams needing the broadest provider coverage and a large ecosystem of community resources.
Honest limitation: Configuration can get complex with many providers. Less control-plane focused than DeltaLLM's admin UI approach.
Portkey
Portkey offers both an open-source gateway and a polished managed cloud. The cloud includes a rich dashboard, prompt versioning, guardrails, and MCP tooling. The best experience is on the managed path.
Best for: Teams that want a managed solution with minimal infrastructure work and strong prompt management.
Honest limitation: Cloud-first (and paid). If you need full open-source self-hosting with strong governance, DeltaLLM is the cleaner fit.
Helicone
Helicone started as observability but now covers routing, failover, prompt tooling, and key management. Still strongest on the analytics side.
Best for: Teams that care most about LLM analytics and want a gateway with strong visibility built in.
Honest limitation: Less opinionated about self-hosted governance compared to DeltaLLM's RBAC and admin UI approach.
How to Choose
| Your situation | Best fit |
|---|---|
| Need the broadest provider coverage and largest ecosystem | LiteLLM |
| Need self-hosted MCP gateway + prompt registry + RBAC | DeltaLLM |
| Want managed SaaS, don't want to run infrastructure | Portkey |
| Need observability-first gateway | Helicone |
| Need granular RBAC (org/team/user) | DeltaLLM |
| Prefer the largest community and most docs | LiteLLM |
| Strict data residency or compliance requirements | DeltaLLM (self-hosted) |
