Running Vector Databases with Milvus on Dedicated VPS

Milvus is an open-source vector database designed to store, search, and manage large amounts of embedding data from AI models like text, images, or audio. You can run Milvus Vector Database on a VPS to:

• Host your own AI search or recommendation system.
• Keep data private and under your control.
• Ensure fast similarity searches without relying on third-party APIs.

We will show you how to set up Milvus with both Docker Compose and the Native DEB package on an Ubuntu VPS. In this guide, we use PerLod’s Linux VPS Hosting, which provides reliable Linux servers with fast SSD storage and easy SSH access.

Requirements to Run Milvus Vector Database on a VPS

You must prepare your Ubuntu VPS with essential system updates and security measures to ensure your server is secure and ready for deployment.

Run the system update and install the required packages with the commands below:

sudo apt update && sudo apt upgrade -y
sudo apt install curl ufw wget ca-certificates gnupg lsb-release -y

Set the correct timezone on your server. For example:

sudo timedatectl set-timezone Europe/Amsterdam

Milvus vector database requires Docker 19.03+ and Compose 1.25+. It is recommended to use Compose v2. Install Docker and Compose from the official Docker repo:

curl -fsSL https://get.docker.com | sudo bash

Start and enable the Docker service with:

sudo systemctl enable --now docker

Check for Docker and Compose version:

docker --version
docker compose version

Also, you must create the data folders and set the correct ownership by running the commands below:

sudo mkdir -p /opt/milvus/{volumes,backups,logs}
sudo chown -R $USER:$USER /opt/milvus

Set up the proper UFW firewall rules:

# allow SSH (adjust if custom port)
sudo ufw allow OpenSSH

# expose Milvus to a trusted IP only (replace 203.0.113.10 with your app server IP)
sudo ufw allow from 203.0.113.10 to any port 19530 proto tcp
sudo ufw allow from 203.0.113.10 to any port 9091  proto tcp

# enable firewall
sudo ufw enable
sudo ufw status numbered

Once you are done with these requirements, proceed to the following steps, choose your installation method, and set up Milvus.

Method 1. Deploy Milvus Standalone via Docker Compose

The first method is to use the official Docker Compose configuration to deploy Milvus, which includes etcd for metadata storage and MinIO for object storage.

Navigate to the Milvus directory and download the official Milvus Docker compose file with the commands below:

cd /opt/milvus
sudo wget https://github.com/milvus-io/milvus/releases/download/v2.3.21/milvus-standalone-docker-compose.yml -O docker-compose.yml

Once your download is completed, bring up the Docker compose:

sudo docker compose up -d

Check for containers with the command below. You should see containers for etcd, minio, and milvus with 19530 and 9091 mapped.

sudo docker compose ps

You can check for Milvus logs with the command below:

sudo docker logs -f milvus-standalone

When you run Milvus, it saves its data in a folder called volume. You should store this folder on a fast SSD or NVMe drive so Milvus runs smoothly, especially because the etcd part needs good disk speed.

You can change Milvus settings by editing a file called user.yaml. After changing that file, remember to restart Milvus to apply the updates.

If you use a message queue for handling data messages:

• Set mq.type to RocksMQ or NATSMQ for single-server setups.
• For cluster setups, Milvus uses Pulsar or Kafka; the simpler ones don’t work in clusters.

To stop and remove the Milvus data, you can use the commands below:

sudo docker compose down
sudo rm -rf /opt/milvus/volumes

Method 2. Install Milvus as a Native System Service (Deb Package)

For environments where containers aren’t preferred, Milvus can be installed natively as a systemd service. Navigate to the Milvus directory and install the latest Milvus deb package with the commands below:

cd /opt/milvus
sudo apt install -y ./milvus_2.6.0-1_amd64.deb

Start and check the Milvus service with the commands below:

sudo systemctl start milvus
sudo systemctl status milvus

Note: The official Milvus docs describe Milvus Standalone as a systemd service named milvus, with config at /etc/milvus/configs/milvus.yaml.

Configure Firewall and Network Access for Milvus Database Vector

You can secure your Milvus database vector by implementing strict firewall rules that display only necessary ports to trusted IP addresses. This reduces your attack surface while ensuring your applications can communicate with the vector database securely.

Milvus uses two main ports:

• 19530 (TCP): For gRPC connections used by gRPC SDKs.
• 9091 (TCP): For REST/HTTP connections.

Note: If your program only talks to Milvus using gRPC, you don’t need to open port 9091.

sudo ufw delete allow 9091/tcp
sudo ufw status numbered

Test Milvus with Python Client Code

At this point, you can run a complete end-to-end test with the Python client to validate your Milvus installation.

This setup shows core vector database operations, including collection creation, data insertion, indexing, and similarity search, which confirms everything is working correctly before moving to production:

# pip install -U pymilvus
from pymilvus import MilvusClient

client = MilvusClient(
    uri="http://YOUR_VPS_PUBLIC_OR_PRIVATE_IP:19530",
    # if you configured auth: token="root:Milvus"
)

COLL = "demo_collection"
if client.has_collection(collection_name=COLL):
    client.drop_collection(collection_name=COLL)

# Create a 768-dim vector collection (change to your model dim)
client.create_collection(
    collection_name=COLL,
    dimension=768,  # vector field 'vector' defaults to this dimension
)

# Insert 3 fake 768-dim vectors just to test end-to-end
import random
vectors = [[random.uniform(-1, 1) for _ in range(768)] for _ in range(3)]
data = [
    {"id": i, "vector": vectors[i], "text": f"row {i}", "tag": "demo"}
    for i in range(3)
]
client.insert(collection_name=COLL, data=data)

# Build an index and load (optional but recommended for performance)
client.create_index(collection_name=COLL, index_params={"metric_type": "COSINE"})
client.load_collection(collection_name=COLL)

# Query: search top-2 for a random query vector
query_vec = [[random.uniform(-1, 1) for _ in range(768)]]
res = client.search(
    collection_name=COLL,
    data=query_vec,
    limit=2,
    output_fields=["text", "tag"],
)
print(res)

Milvus Production Hardening and Optimization

For reliable vector database operations in production, you can consider these best practices, including storage optimization, message queue selection, monitoring setup, and backup strategies.

1. Storage and IOPS for etcd:

• Store Milvus data on fast local NVMe drives.
• The etcd part is sensitive to disk speed: aim for >500 IOPS and <10 ms fsync latency.
• You can test disk speed with fio as shown.

mkdir -p /opt/milvus/test-data
sudo apt -y install fio
fio --rw=write --ioengine=sync --fdatasync=1 --directory=/opt/milvus/test-data \
    --size=2200m --bs=2300 --name=mytest

2. Choosing your message queue (MQ):

• Standalone Milvus: use RocksMQ (default) or NATSMQ, which is lightweight and simple.
• Cluster Milvus: must use Pulsar or Kafka. RocksMQ/NATS don’t work in clusters.

3. Security hardening:

• Bind Milvus to private networks; only allow your app’s IP.
• Use a reverse proxy like nginx or haproxy for HTTPS if you need it on port 9091.
• Rotate MinIO credentials and avoid exposing MinIO publicly.
• Use a private VPC or VLAN if your provider allows it.

4. Backups: Use the Milvus Backup tool or take offline snapshots of the data folder.

5. Monitoring: Milvus works with Prometheus; add Prometheus and scrape Milvus metrics.

For production use, consider using a VPS provider like PerLod Hosting that offers backups, stable uptime, and good support, especially if you plan to integrate Milvus with your website or other services.

Tip: To keep Milvus running smoothly, it’s important to monitor your VPS resources. For a full guide on setting up automated server monitoring, alerts, and dashboards, you can check this guide on setting up an AIOps stack for server monitoring.

That’s it, you are done with running the Milvus database vector on a VPS.

FAQs

Conclusion

Running the Milvus database vector on a VPS has a straightforward process with both Docker Compose and a native DEB install. Remember to secure your ports, use fast storage, and monitor resources. Milvus makes vector search easy, reliable, and scalable, even on a single VPS.

If your dataset grows, you will need a multi-node Milvus cluster. Our guide on VPS Horizontal Scaling Strategies explains how to scale multiple VPS servers efficiently.

We hope you enjoy this guide. Subscribe to our X and Facebook channels to get the latest articles and updates.