Install agentgateway on Ubuntu for MCP, LLM API, and Agent-to-Agent Traffic

If you need a single gateway that supports HTTP, MCP, and LLM APIs simultaneously, this guide shows you how to install agentgateway on Ubuntu. It sets up a real LLM route, exposes an MCP backend, and locks everything down with TLS and JWT auth.

Agentgateway is an open-source proxy built in Rust that handles regular HTTP traffic plus LLM, MCP, and agent-to-agent (A2A) traffic, all through one gateway, so you don't need separate tools for normal and AI traffic.

Choosing to install agentgateway on Ubuntu gives you a stable LTS kernel, systemd for process supervision, and full root access to tune networking and TLS certificates for AI-native traffic.

Most API gateways only handle simple, one-off request/response calls. Agentgateway is different; it can keep track of ongoing JSON-RPC sessions, handle live SSE push updates from MCP servers, and dynamically manage tools, things regular reverse proxies just aren't built for.

Because of that, it's best run on its own dedicated Ubuntu server, and a reliable Linux VPS works well as that dedicated gateway node sitting in front of your private AI and MCP services.

Before you install agentgateway on Ubuntu, confirm the server meets these requirements:

• Ubuntu 22.04 LTS or 24.04 LTS with a non-root sudo user and root access.
• At least 2 vCPUs, 2 GB RAM, and 10 GB free disk, more if you route heavy LLM or MCP traffic.
• curl, openssl, and Node.js/npx installed. npx is just needed to run a sample MCP server for testing.
• Firewall open for outbound traffic, so you can install packages, and inbound access on ports 80/443 or whichever port you choose for the gateway.
• An API key from your LLM provider. This guide uses OpenAI, but Anthropic, Gemini, Bedrock, Azure, and OpenAI-compatible endpoints like Ollama or vLLM work the same way.

Before you proceed to install agentgateway on Ubuntu, update the system and install the required packages:

nodejs and npm give you npx, which agentgateway uses to launch the sample MCP test server. If Node isn't available on your Ubuntu repo version, install it via NodeSource; agentgateway itself doesn't depend on Node, only the demo MCP server does.

Allow firewall rules with:

Then, create a dedicated system user, which is recommended for production:

Running the gateway under its own limited-permission user keeps it separate from everything else on the server, which matters once this VPS is handling private AI and MCP traffic.

As the agentgateway user, install the binary package with:

This downloads the prebuilt binary matching your CPU architecture and installs it locally.

Verify the installation:

If the binary landed in your home directory instead of your PATH, move it system-wide:

Create a dedicated directory to keep the gateway configuration and TLS assets organized:

You'll create one file here, which is config.yaml. This is the single file that drives every route, listener, LLM backend, MCP backend, TLS setting, and auth policy.

Create the file with your desired text editor:

Paste this into config.yaml.This is the official simplified LLM format, which serves an OpenAI-compatible chat endpoint on port 4000 by default:

Save and exit the file. Create your .env file to hold the real key:

Add this line and replace it with your real key from OpenAI:

Save the file, set safe permissions, and load it into your current shell to test:

Start agentgateway manually first to confirm everything works before wiring it into systemd:

In a second terminal, verify the LLM route with curl:

A working response returns a standard OpenAI-style JSON body with a choices array. Stop the process with Ctrl+C once confirmed.

Now expand config.yaml to add MCP support alongside the LLM route. Reopen the file:

Update it to this combined format, which adds an mcp block on its own port using the official simplified syntax:

This single file, once you install agentgateway on Ubuntu with it, serves the LLM API on port 4000 and the MCP server on port 3000 at the same time.

Start the gateway again:

Initialize an MCP session:

Copy the Mcp-Session-Id header value from the response, then list the available tools:

You should see tools like echo, add, and longRunningOperation returned. You can also open http://your-server-ip:15000/ui/ in a browser and use the built-in Playground to connect and test tools visually.

Create self-signed certs for staging and swap for real Let's Encrypt files in production:

Add a tls block under the llm section in config.yaml to serve it over HTTPS:

If you prefer not to manage certificates by hand, you can put Traefik in front of agentgateway instead and let it handle TLS termination and automatic renewal. You can check this guide on Setting Up Traefik with Docker Compose for Self-Hosted Apps and Automatic HTTPS.

For production, you can add JWT validation and per-tool authorization rules directly in config.yaml:

This example lets any authenticated user call echo, but restricts add to a specific user via their JWT subject claim.

Exit back to your sudo user and create the service file:

Add:

Enable and start the service:

Check live logs at any point:

Run both routes end-to-end to confirm the full install agentgateway on Ubuntu setup works together:

If both return valid JSON, your gateway is correctly routing LLM and MCP traffic.

You've now installed agentgateway on Ubuntu, configured one LLM route and one MCP route, secured both with TLS and JWT authentication, and verified everything with curl. Running it under systemd on a dedicated VPS keeps the service strong, and pairing it with Traefik gives you automatic certificate renewal for real domains.

We hope you enjoy this guide. For more information, check the official agentgateway GitHub repository.