MCP Gateway: Connecting AI Agents to Any LLM Provider

The Model Context Protocol (MCP) gives AI agents a standard way to use external tools — databases, search engines, code executors, anything you can wrap in an MCP server. But raw MCP connections lack the controls you need in production: who can access which tools, rate limits, audit trails, and failover when the LLM provider goes down.

DeltaLLM's MCP gateway adds that control plane on top of MCP, so your agents get tool access with enterprise-grade guardrails.

What MCP Is (Simply)

MCP defines a standard protocol for AI models to use external tools. Instead of every developer building their own function-calling integration, MCP provides a common format:

• MCP Servers — services that expose tools (a search API, a database, a code runner)
• MCP Clients — AI agents that call those tools
• MCP Gateway — a proxy that manages access, authentication, and policies for MCP traffic

A tool built once as an MCP server can be used with any MCP-compatible model or framework. That's the portability benefit.

What DeltaLLM Adds on Top

Raw MCP servers expose tools to anyone who can reach them. DeltaLLM adds the controls you need in production:

• Server registry — register MCP servers with health checks and capability refresh.
• Scoped visibility — control which organizations, teams, or API keys can see which tools.
• Per-tool policies — enable/disable tools, set rate limits, require manual approvals, cache results.
• Audit logging — every tool call is logged with the caller, tool, timing, and result.
• Namespaced tools — tools are exposed as <code className='font-mono text-sm'>server_key.tool_name</code>, so names never collide.

Getting Started: Your First MCP Tool Call

Here's the full workflow to get an MCP tool working through DeltaLLM. It takes about five minutes.

1. Register an MCP server

curl -X POST http://localhost:4002/ui/api/mcp-servers \
  -H "Authorization: Bearer $DELTALLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "server_key": "docs",
    "name": "Docs Search",
    "transport": "streamable_http",
    "base_url": "http://host.docker.internal:8899/mcp",
    "auth_mode": "none",
    "enabled": true
  }'

2. Refresh capabilities and check health

curl -X POST $BASE/ui/api/mcp-servers/$SERVER_ID/refresh-capabilities \
  -H "Authorization: Bearer $DELTALLM_MASTER_KEY"

curl -X POST $BASE/ui/api/mcp-servers/$SERVER_ID/health-check \
  -H "Authorization: Bearer $DELTALLM_MASTER_KEY"

3. Make it visible to your app

MCP tools aren't globally visible by default. Bind the server to an organization, team, or API key:

curl -X POST http://localhost:4002/ui/api/mcp-bindings \
  -H "Authorization: Bearer $DELTALLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "server_id": "'"$SERVER_ID"'",
    "scope_type": "organization",
    "scope_id": "org_123",
    "enabled": true
  }'

4. Set a tool policy

curl -X POST http://localhost:4002/ui/api/mcp-tool-policies \
  -H "Authorization: Bearer $DELTALLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "server_id": "'"$SERVER_ID"'",
    "tool_name": "search",
    "scope_type": "organization",
    "scope_id": "org_123",
    "enabled": true,
    "require_approval": "never"
  }'

5. Call the tool

Use the MCP gateway endpoint directly:

curl http://localhost:4002/mcp \
  -H "Authorization: Bearer $YOUR_SCOPED_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc": "2.0",
    "id": 1,
    "method": "tools/call",
    "params": {
      "name": "docs.search",
      "arguments": {"query": "DeltaLLM"}
    }
  }'

Using MCP Tools from Chat

The real power is letting LLMs call MCP tools automatically through the regular chat API. DeltaLLM translates MCP tools into OpenAI-style function calls:

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Search the docs for DeltaLLM."}],
    tools=[{
        "type": "mcp",
        "server": "docs",
        "allowed_tools": ["search"],
        "require_approval": "never"
    }],
    tool_choice="required"
)

DeltaLLM resolves the visible MCP tools, converts them to OpenAI function-call format, executes any tool calls the model makes, and feeds results back. Your app doesn't need to know about MCP at all.

Why This Matters for Agents

AI agents doing multi-step tasks are especially sensitive to provider failures. A mid-task outage can leave the agent in a broken state. With DeltaLLM's failover, if the primary LLM fails mid-session, the request retries on a secondary provider — without the agent handling the error.

Combined with scoped tool visibility (so each team's agents only see the tools they should), per-tool rate limits, and audit logging for every call, you get an agent infrastructure that's ready for production — not just demos.

Quick Troubleshooting