If you need a single gateway that supports HTTP, MCP, and LLM APIs simultaneously, this guide shows you how to install agentgateway on Ubuntu, wire up a real LLM route, expose an MCP backend, and lock everything down with TLS and JWT auth.

Agentgateway is an open-source proxy built in Rust that handles regular HTTP traffic plus LLM, MCP, and agent-to-agent (A2A) traffic, all through one gateway, so you don’t need separate tools for normal and AI traffic. In this guide, you’ll install agentgateway on Ubuntu, set up listeners, connect it to an LLM provider, add an MCP backend, turn on TLS and authentication, and test it all with curl.

Why Install agentgateway on Ubuntu

Choosing to install agentgateway on Ubuntu gives you a stable LTS kernel, systemd for process supervision, and full root access to tune networking and TLS certificates for AI-native traffic.

Most API gateways only handle simple, one-off request/response calls. Agentgateway is different; it can keep track of ongoing JSON-RPC sessions, handle live SSE push updates from MCP servers, and dynamically manage tools, things regular reverse proxies just aren’t built for.

Because of that, it’s best run on its own dedicated Ubuntu server, and a reliable Linux VPS works well as that dedicated gateway node sitting in front of your private AI and MCP services.

Prerequisites

Before you install agentgateway on Ubuntu, confirm the server meets these requirements:

• Ubuntu 22.04 LTS or 24.04 LTS with a non-root sudo user and root access.
• At least 2 vCPUs, 2 GB RAM, and 10 GB free disk, more if you route heavy LLM or MCP traffic.
• curl, openssl, and Node.js/npx installed. npx is just needed to run a sample MCP server for testing.
• Firewall open for outbound traffic, so you can install packages, and inbound access on ports 80/443 or whichever port you choose for the gateway.
• An API key from your LLM provider. This guide uses OpenAI, but Anthropic, Gemini, Bedrock, Azure, and OpenAI-compatible endpoints like Ollama or vLLM work the same way.

Before you proceed to install agentgateway on Ubuntu, update the system and install the required packages:

sudo apt update && sudo apt upgrade -y
sudo apt install curl openssl ca-certificates nodejs npm -y

nodejs and npm give you npx, which agentgateway uses to launch the sample MCP test server. If Node isn’t available on your Ubuntu repo version, install it via NodeSource; agentgateway itself doesn’t depend on Node, only the demo MCP server does.

Allow firewall rules with:

sudo ufw allow 443/tcp
sudo ufw allow 80/tcp
sudo ufw enable
sudo ufw status

Then, create a dedicated system user, which is recommended for production:

sudo useradd -r -m -d /home/agentgateway -s /bin/bash agentgateway
sudo su - agentgateway

Running the gateway under its own limited-permission user keeps it separate from everything else on the server, which matters once this VPS is handling private AI and MCP traffic.

Step 1. Install agentgateway on Ubuntu

As the agentgateway user, install the binary package with:

curl -sL https://agentgateway.dev/install | bash

This downloads the prebuilt binary matching your CPU architecture and installs it locally.

Verify the installation:

agentgateway --version

If the binary landed in your home directory instead of your PATH, move it system-wide:

sudo mv ./agentgateway /usr/local/bin/agentgateway
sudo chmod +x /usr/local/bin/agentgateway

Step 2. Create the Working Directory and Config File

Create a dedicated directory to keep the gateway configuration and TLS assets organized:

mkdir -p /home/agentgateway/config
mkdir -p /home/agentgateway/certs
cd /home/agentgateway/config

You’ll create one file here, which is config.yaml. This is the single file that drives every route, listener, LLM backend, MCP backend, TLS setting, and auth policy.

Create the file with your desired text editor:

nano /home/agentgateway/config/config.yaml

Step 3. Configure the LLM Route

Paste this into config.yaml.This is the official simplified LLM format, which serves an OpenAI-compatible chat endpoint on port 4000 by default:

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
llm:
  models:
    - name: "gpt-4.1-nano"
      provider: openAI
      params:
        model: gpt-4.1-nano
        apiKey: "$OPENAI_API_KEY"

Save and exit the file. Create your .env file to hold the real key:

nano /home/agentgateway/.env

Add this line and replace it with your real key from OpenAI:

OPENAI_API_KEY=sk-your-real-key-here

Save the file, set safe permissions, and load it into your current shell to test:

chmod 600 /home/agentgateway/.env
export $(cat /home/agentgateway/.env | xargs)

Step 4. Test the LLM Route

Start agentgateway manually first to confirm everything works before wiring it into systemd:

agentgateway -f /home/agentgateway/config/config.yaml

In a second terminal, verify the LLM route with curl:

curl http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4.1-nano","messages":[{"role":"user","content":"Hello!"}]}'

A working response returns a standard OpenAI-style JSON body with a choices array. Stop the process with Ctrl+C once confirmed.

Step 5. Add the MCP Route

Now expand config.yaml to add MCP support alongside the LLM route. Reopen the file:

nano /home/agentgateway/config/config.yaml

Update it to this combined format, which adds an mcp block on its own port using the official simplified syntax:

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
llm:
  port: 4000
  models:
    - name: "gpt-4.1-nano"
      provider: openAI
      params:
        model: gpt-4.1-nano
        apiKey: "$OPENAI_API_KEY"

mcp:
  port: 3000
  policies:
    cors:
      allowOrigins: ["*"]
      allowHeaders: ["mcp-protocol-version", "content-type", "cache-control"]
      exposeHeaders: ["Mcp-Session-Id"]
  targets:
    - name: everything
      stdio:
        cmd: npx
        args: ["@modelcontextprotocol/server-everything"]

This single file, once you install agentgateway on Ubuntu with it, serves the LLM API on port 4000 and the MCP server on port 3000 at the same time.

Step 6. Test the MCP Route

Start the gateway again:

agentgateway -f /home/agentgateway/config/config.yaml

Initialize an MCP session:

curl -s -i http://localhost:3000/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -d '{"jsonrpc":"2.0","method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}},"id":1}'

Copy the Mcp-Session-Id header value from the response, then list the available tools:

curl -s http://localhost:3000/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -H "Mcp-Session-Id: YOUR_SESSION_ID" \
  -d '{"jsonrpc":"2.0","method":"tools/list","id":2}'

You should see tools like echo, add, and longRunningOperation returned. You can also open http://your-server-ip:15000/ui/ in a browser and use the built-in Playground to connect and test tools visually.

Step 7. Add TLS Certificates

Create self-signed certs for staging and swap for real Let’s Encrypt files in production:

cd /home/agentgateway/certs
openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes -subj "/CN=your-domain.com"

Add a tls block under the llm section in config.yaml to serve it over HTTPS:

llm:
  port: 8443
  tls:
    cert: /home/agentgateway/certs/cert.pem
    key: /home/agentgateway/certs/key.pem
  models:
    - name: "gpt-4.1-nano"
      provider: openAI
      params:
        model: gpt-4.1-nano
        apiKey: "$OPENAI_API_KEY"

mcp:
  port: 3000
  targets:
    - name: everything
      stdio:
        cmd: npx
        args: ["@modelcontextprotocol/server-everything"]

If you prefer not to manage certificates by hand, you can put Traefik in front of agentgateway instead and let it handle TLS termination and automatic renewal. You can check this guide on Setting Up Traefik with Docker Compose for Self-Hosted Apps and Automatic HTTPS.

Step 8. Add JWT Authentication

For production, you can add JWT validation and per-tool authorization rules directly in config.yaml:

mcp:
  port: 3000
  policies:
    mcpAuthentication:
      issuer: http://localhost:9000
      audiences:
        - http://localhost:3000/mcp
      jwks:
        url: http://localhost:9000/.well-known/jwks.json
    mcpAuthorization:
      rules:
        - 'mcp.tool.name == "echo"'
        - 'jwt.sub == "test-user" && mcp.tool.name == "add"'
  targets:
    - name: everything
      stdio:
        cmd: npx
        args: ["@modelcontextprotocol/server-everything"]

This example lets any authenticated user call echo, but restricts add to a specific user via their JWT subject claim.

Step 9. Run agentgateway as a systemd Service

Exit back to your sudo user and create the service file:

exit
sudo nano /etc/systemd/system/agentgateway.service

Add:

[Unit]
Description=agentgateway
After=network.target

[Service]
Type=simple
User=agentgateway
WorkingDirectory=/home/agentgateway/config
ExecStart=/usr/local/bin/agentgateway -f /home/agentgateway/config/config.yaml
EnvironmentFile=/home/agentgateway/.env
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Enable and start the service:

sudo systemctl daemon-reload
sudo systemctl enable --now agentgateway
sudo systemctl status agentgateway

Check live logs at any point:

sudo journalctl -u agentgateway -f

Step 10. Final Verification

Run both routes end-to-end to confirm the full install agentgateway on Ubuntu setup works together:

curl -k https://your-domain.com:8443/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4.1-nano","messages":[{"role":"user","content":"ping"}]}'

curl -s http://localhost:3000/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -d '{"jsonrpc":"2.0","method":"tools/list","id":3}'

If both return valid JSON, your gateway is correctly routing LLM and MCP traffic.

Conclusion

You’ve now installed agentgateway on Ubuntu, configured one LLM route and one MCP route, secured both with TLS and JWT authentication, and verified everything with curl. Running it under systemd on a dedicated VPS keeps the service strong, and pairing it with Traefik gives you automatic certificate renewal for real domains.

We hope you enjoy this guide. For more information, check the official agentgateway GitHub repository.

FAQs