Connect every application to one OpenAI-compatible gateway, then control who gets access, how model capacity is shared, which policies apply, and how batch workloads run.
Built for multi-provider products, internal AI teams, batch-heavy workflows, and self-hosted operations.
Keep the familiar multi-provider gateway experience, then add the controls your product needs for access, capacity sharing, batch work, and day-to-day operation without growing your application surface area.
Give multiple apps and teams one API, one access model, and one place to manage routing, capacity, and spend.
Keep one OpenAI-compatible integration while switching or combining providers, regions, credentials, and deployment capacity.
Run embedding, indexing, enrichment, and evaluation jobs with queueing, progress, retries, and cost attribution.
Keep the gateway, provider credentials, prompts, policies, logs, and spend data on infrastructure you manage.
A gateway connects applications to models. DeltaLLM also gives you control over who can use them, how capacity is shared, and how work is operated at scale.
Give each organization a reusable model package with allowed models, limits, pricing, and shared capacity instead of configuring every key by hand.
Put multiple provider, region, or deployment instances behind one model target with weighted routing, failover, and rate-limit-aware policies.
Queue embedding and offline workloads with progress, retries, downloadable artifacts, and cost attribution without building a second AI service.
Apply scoped access, guardrails, prompt versions, MCP tool policies, budgets, and spend attribution at the same boundary as model traffic.
DeltaLLM is the gateway for teams whose requirements have grown beyond provider abstraction: access, capacity, queued work, reliability, and accountable spend.
Manage the parts teams usually end up rebuilding around provider APIs: organization access, capacity policy, batch work, governance, spend, and observability.
Package models, limits, pricing, and shared capacity into reusable policies for every organization or customer plan.
Share model capacity across providers, regions, credentials, and deployments with centrally managed routing policies.
Queue embedding and offline work with progress, retries, result artifacts, and cost attribution—providing provider-agnostic batching at the gateway layer when native support is absent.
Issue scoped keys with budgets, rate limits, and model restrictions. Control access without sharing provider credentials.
Multiple routing strategies with automatic failover and retries. Never lose a request to a provider outage.
Built-in PII detection and prompt injection protection. Keep your data safe at the gateway level.
Per-key, per-team, and per-model cost attribution. Know exactly where every dollar goes.
Response caching with memory, Redis, or S3 backends. Reduce latency and costs with intelligent caching.
Role-based access control with SSO support. Microsoft Entra, Google, Okta, and custom OIDC providers.
Hierarchical limits at organization, team, user, and key levels. Prevent abuse and control consumption.
Prometheus metrics, request logging, and spend analytics. Full visibility into your LLM usage.
Control tool visibility, per-tool policy, approvals, and auditability through the same gateway boundary.
Choose the fastest local path with Docker Compose or install the released Helm chart for Kubernetes, then point your OpenAI client at one /v1 endpoint.
git clone https://github.com/deltawi/deltallm.git cd deltallm cp config.example.yaml config.yaml
# Generate and paste into .env
python3 -c 'import secrets; print("DELTALLM_MASTER_KEY=sk-" + secrets.token_hex(20) + "A1")'
python3 -c 'import secrets; print("DELTALLM_SALT_KEY=" + secrets.token_hex(32))'
# Also add:
OPENAI_API_KEY=sk-your-keydocker compose --profile single up -d --build # DeltaLLM is now at http://localhost:4002 curl http://localhost:4002/health/liveliness
Sign in to the Admin UI with your bootstrap admin account at http://localhost:4002. Use $DELTALLM_MASTER_KEY for gateway calls.
1from openai import OpenAI23client = OpenAI(4 base_url="http://localhost:4000/v1",5 api_key="YOUR_VIRTUAL_KEY", # created in the Admin UI6)78response = client.chat.completions.create(9 model="gpt-4o-mini",10 messages=[{"role": "user", "content": "Hello!"}],11)12print(response.choices[0].message.content)Short answers to the product and deployment questions that come up before evaluation.
An LLM gateway is a layer between your application and providers like OpenAI, Anthropic, and Gemini. DeltaLLM gives you one OpenAI-compatible API for routing, failover, authentication, capacity sharing, spend tracking, guardrails, access control, and batch workloads.
Yes. DeltaLLM is open source under the MIT license and is designed to be self-hosted.
DeltaLLM supports 100+ providers through an OpenAI-compatible API, including OpenAI, Anthropic, Groq, Google Gemini, Azure OpenAI, and more.
DeltaLLM provides the same familiar multi-provider, OpenAI-compatible gateway foundation, with built-in controls for service tiers, capacity sharing across instances, scoped access, batch workloads, and production operations.
Yes. DeltaLLM can run embedding batch jobs with queueing, per-item progress, retries, downloadable artifacts, and cost attribution so asynchronous work does not compete with interactive traffic.
Yes. DeltaLLM includes a built-in MCP gateway with scoped tool visibility, per-tool policies, approval flows, and audit logging.