Event-Driven Autoscaling on K3s with KEDA, RabbitMQ, and Prometheus

Updated on Sep 15, 2026
Mila H
10 MINS READ
Table of Contents
Install KEDA on K3s and Autoscale Workers

This KEDA K3s autoscaling tutorial shows you how to scale two real workloads. One worker that reacts to RabbitMQ queue depth, and one worker that reacts to a Prometheus metric. You will learn to create both ScaledObjects, generate real load, and check the scaling decisions step by step.

What Is KEDA and Why Use It on K3s

KEDA (Kubernetes Event-Driven Autoscaling) adds extra triggers to the normal Kubernetes Horizontal Pod Autoscaler (HPA). Instead of scaling only on CPU or memory, KEDA can scale on a queue length, a message rate, or any Prometheus query. Also, it can scale a deployment down to zero pods when there is no work, which the normal HPA cannot do.

K3s is a lightweight Kubernetes distribution. It runs easily on a single VPS or a small home lab. Since K3s is still full Kubernetes, every standard Helm chart and every KEDA object works on it with no changes.

For real workloads, it is recommended to use a reliable dedicated server. KEDA needs to check signals and start pods fast, so steady CPU and network performance matter for smooth autoscaling.

Prerequisites for KEDA K3s Autoscaling

Before you start this KEDA K3s autoscaling tutorial, make sure you have:

  • A Linux server running Ubuntu 22.04 or newer with at least 4 GB RAM and 2 CPU cores, root or sudo access.
  • An open outbound internet connection to download K3s, Helm charts, and container images.
  • Basic comfort with the terminal and kubectl.

Step 1: Install K3s

First, install the latest stable K3s release with the official install script:

Bash
curl -sfL https://get.k3s.io | sh -

Wait about 30 to 60 seconds, then check that the node is ready:

Bash
sudo k3s kubectl get nodes

You should see one node with status Ready. Now set up kubectl so you don't need sudo k3s kubectl every time:

Bash
mkdir -p ~/.kubesudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/configsudo chown $(id -u):$(id -g) ~/.kube/configexport KUBECONFIG=~/.kube/configecho 'export KUBECONFIG=~/.kube/config' >> ~/.bashrc

Confirm it works correctly by checking its version:

Bash
kubectl version --short

Step 2: Install Helm

Helm is the package manager we will use for KEDA and Prometheus. Install the latest stable Helm 3 with the official script:

Bash
curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3chmod +x get_helm.sh./get_helm.shhelm version

Step 3: Install KEDA on K3s

This is the core step of the KEDA K3s autoscaling tutorial. You must add the official KEDA Helm repository and install the latest stable chart, which deploys KEDA 2.20.2:

Bash
helm repo add kedacore https://kedacore.github.io/chartshelm repo updatehelm install keda kedacore/keda --namespace keda --create-namespace

Check that KEDA is running:

Bash
kubectl get pods -n kedakubectl get crd | grep keda.sh

You should see three pods, including keda-operator, keda-operator-metrics-apiserver, and keda-admission-webhooks all in Running state. And CRDs such as scaledobjects.keda.sh and triggerauthentications.keda.sh.

This confirms KEDA is installed correctly before we connect any real signal.

Step 4: Deploy RabbitMQ on K3s

We will run RabbitMQ with the management plugin enabled, using a plain Deployment so you can see every setting. First, create a namespace and a secret:

Bash
kubectl create namespace rabbitmqkubectl create secret generic rabbitmq-auth \  --namespace rabbitmq \  --from-literal=username=admin \  --from-literal=password='ChangeThisPassword123'

Then, create the RabbitMQ manifest file:

Bash
cat > rabbitmq.yaml <<'EOF'apiVersion: apps/v1kind: Deploymentmetadata:  name: rabbitmq  namespace: rabbitmqspec:  replicas: 1  selector:    matchLabels:      app: rabbitmq  template:    metadata:      labels:        app: rabbitmq    spec:      containers:        - name: rabbitmq          image: rabbitmq:4.3-management          ports:            - containerPort: 5672            - containerPort: 15672          env:            - name: RABBITMQ_DEFAULT_USER              valueFrom:                secretKeyRef:                  name: rabbitmq-auth                  key: username            - name: RABBITMQ_DEFAULT_PASS              valueFrom:                secretKeyRef:                  name: rabbitmq-auth                  key: password---apiVersion: v1kind: Servicemetadata:  name: rabbitmq  namespace: rabbitmqspec:  selector:    app: rabbitmq  ports:    - name: amqp      port: 5672      targetPort: 5672    - name: management      port: 15672      targetPort: 15672EOF

Apply the file and check the status:

Bash
kubectl apply -f rabbitmq.yamlkubectl -n rabbitmq rollout status deployment/rabbitmq

Note: If a queue ever gets stuck or messages will not deliver, our RabbitMQ error-fixing guide covers the most common problems.

Step 5: Deploy the RabbitMQ Worker and Create the Queue

The worker is the deployment that KEDA will scale. Here we use a small busybox loop as a stand-in worker so you can focus on the scaling behavior:

Bash
cat > rabbitmq-worker.yaml <<'EOF'apiVersion: apps/v1kind: Deploymentmetadata:  name: rabbitmq-worker  namespace: rabbitmqspec:  replicas: 0  selector:    matchLabels:      app: rabbitmq-worker  template:    metadata:      labels:        app: rabbitmq-worker    spec:      containers:        - name: worker          image: busybox:1.36          command: ["sh", "-c", "while true; do echo processing message; sleep 5; done"]EOF

Apply it with the command below:

Bash
kubectl apply -f rabbitmq-worker.yaml

Then, you should create the queue named orders. Exec into the RabbitMQ pod and use rabbitmqadmin, the command-line tool that ships with the management plugin:

Bash
RABBIT_POD=$(kubectl -n rabbitmq get pod -l app=rabbitmq -o jsonpath='{.items[0].metadata.name}') kubectl -n rabbitmq exec -it "$RABBIT_POD" -- bash -c \  "curl -s -o /rabbitmqadmin -u admin:ChangeThisPassword123 http://localhost:15672/cli/rabbitmqadmin && \   chmod +x /rabbitmqadmin && \   /rabbitmqadmin -u admin -p ChangeThisPassword123 declare queue name=orders durable=true"

Step 6: Create the RabbitMQ ScaledObject

At this point, you should create a TriggerAuthentication so KEDA can log in to RabbitMQ using the secret you already made:

Bash
cat > rabbitmq-trigger-auth.yaml <<'EOF'apiVersion: keda.sh/v1alpha1kind: TriggerAuthenticationmetadata:  name: rabbitmq-trigger-auth  namespace: rabbitmqspec:  secretTargetRef:    - parameter: username      name: rabbitmq-auth      key: username    - parameter: password      name: rabbitmq-auth      key: passwordEOF

Apply the file with:

Bash
kubectl apply -f rabbitmq-trigger-auth.yaml

Now you can create the ScaledObject. It watches the orders queue and scales the worker between 0 and 15 replicas, targeting 5 messages per pod:

Bash
cat > rabbitmq-scaledobject.yaml <<'EOF'apiVersion: keda.sh/v1alpha1kind: ScaledObjectmetadata:  name: rabbitmq-worker-scaler  namespace: rabbitmqspec:  scaleTargetRef:    name: rabbitmq-worker  pollingInterval: 10  cooldownPeriod: 60  minReplicaCount: 0  maxReplicaCount: 15  triggers:    - type: rabbitmq      authenticationRef:        name: rabbitmq-trigger-auth      metadata:        protocol: amqp        host: amqp://rabbitmq.rabbitmq.svc.cluster.local:5672/        queueName: orders        mode: QueueLength        value: "5"EOF

pollingInterval: 10 means KEDA checks the queue every 10 seconds.

cooldownPeriod: 60 means the worker only scales down to zero after the queue stays empty for 60 straight seconds. This stops short quiet moments from shutting the worker down too soon.

Apply and check the RabbitMQ autoscaler:

Bash
kubectl apply -f rabbitmq-scaledobject.yamlkubectl -n rabbitmq get scaledobject

Step 7: Install Prometheus with kube-prometheus-stack Chart

For the second worker, we need a real Prometheus server to query. Install the community kube-prometheus-stack, which bundles Prometheus, the Prometheus Operator, and Grafana:

Bash
helm repo add prometheus-community https://prometheus-community.github.io/helm-chartshelm repo updatehelm install kube-prometheus-stack prometheus-community/kube-prometheus-stack \  --namespace monitoring --create-namespace

Wait for the pods to become ready, then confirm the Prometheus service name:

Bash
kubectl -n monitoring get podskubectl -n monitoring get svc | grep prometheus

You should see a service called kube-prometheus-stack-prometheus listening on port 9090. KEDA's Prometheus scaler will query this service directly.

Step 8: Deploy HTTP Worker That Exposes Prometheus Metrics

Now you can create a small HTTP app that counts incoming requests and exposes that count on a /metrics endpoint, which Prometheus will scrape. We store the app code in a ConfigMap.

Bash
kubectl create namespace http-worker
Bash
cat > http-worker.yaml <<'EOF'apiVersion: v1kind: ConfigMapmetadata:  name: http-worker-code  namespace: http-workerdata:  app.py: |    from flask import Flask    from prometheus_client import Counter, generate_latest    app = Flask(__name__)    REQUEST_COUNT = Counter("http_requests_total", "Total HTTP requests")     @app.route("/")    def index():        REQUEST_COUNT.inc()        return "ok"     @app.route("/metrics")    def metrics():        return generate_latest()     if __name__ == "__main__":        app.run(host="0.0.0.0", port=8080)---apiVersion: apps/v1kind: Deploymentmetadata:  name: http-worker  namespace: http-workerspec:  replicas: 1  selector:    matchLabels:      app: http-worker  template:    metadata:      labels:        app: http-worker    spec:      containers:        - name: http-worker          image: python:3.12-slim          command: ["sh", "-c", "pip install --no-cache-dir flask prometheus_client && python /app/app.py"]          ports:            - containerPort: 8080          volumeMounts:            - name: code              mountPath: /app      volumes:        - name: code          configMap:            name: http-worker-code---apiVersion: v1kind: Servicemetadata:  name: http-worker  namespace: http-worker  labels:    app: http-workerspec:  selector:    app: http-worker  ports:    - name: http      port: 8080      targetPort: 8080EOF

Apply and check the HTTP worker status:

Bash
kubectl apply -f http-worker.yamlkubectl -n http-worker rollout status deployment/http-worker

Next, create a ServiceMonitor so Prometheus scrapes this service.

The label release: kube-prometheus-stack must match your Helm release name. That's how the chart's Prometheus knows to pick up this ServiceMonitor by default.

Bash
cat > http-worker-servicemonitor.yaml <<'EOF'apiVersion: monitoring.coreos.com/v1kind: ServiceMonitormetadata:  name: http-worker  namespace: http-worker  labels:    release: kube-prometheus-stackspec:  selector:    matchLabels:      app: http-worker  endpoints:    - port: http      path: /metrics      interval: 10sEOF
Bash
kubectl apply -f http-worker-servicemonitor.yaml

Step 9: Create the Prometheus-Driven ScaledObject

This part of the KEDA K3s autoscaling tutorial connects KEDA directly to the Prometheus query language instead of a queue. The ScaledObject below scales the HTTP worker between 1 and 10 replicas based on the request rate:

Bash
cat > http-worker-scaledobject.yaml <<'EOF'apiVersion: keda.sh/v1alpha1kind: ScaledObjectmetadata:  name: http-worker-scaler  namespace: http-workerspec:  scaleTargetRef:    name: http-worker  pollingInterval: 15  cooldownPeriod: 90  minReplicaCount: 1  maxReplicaCount: 10  triggers:    - type: prometheus      metadata:        serverAddress: http://kube-prometheus-stack-prometheus.monitoring.svc.cluster.local:9090        query: sum(rate(http_requests_total[2m]))        threshold: "5"        activationThreshold: "1"EOF

threshold: "5" means KEDA aims for about 5 requests per second per pod.

If the rate goes above that, KEDA adds more pods. If it drops, KEDA removes pods, down to the minimum of minReplicaCount: 1.

Apply and check with:

Bash
kubectl apply -f http-worker-scaledobject.yamlkubectl -n http-worker get scaledobject

Step 10: Test the Autoscaler with Real Load

Now it's time to see KEDA in action. We'll send real traffic to both workers: messages to the RabbitMQ queue, and requests to the HTTP worker. This gives KEDA a signal to react to, so we can watch it scale the pods up.

Load the RabbitMQ queue: Exec back into the RabbitMQ pod and publish 300 test messages to the orders queue:

Bash
kubectl -n rabbitmq exec -it "$RABBIT_POD" -- bash -c \  'for i in $(seq 1 300); do /rabbitmqadmin -u admin -p ChangeThisPassword123 publish exchange=amq.default routing_key=orders payload="order-$i"; done'

Load the HTTP worker: Run a temporary pod that hits the HTTP worker in a loop to raise the request rate:

Bash
kubectl -n http-worker run load-generator --rm -it --restart=Never --image=busybox:1.36 -- \  sh -c "while true; do wget -q -O- http://http-worker:8080/; done"

Let it run for a minute or two, then press Ctrl+C to stop.

Step 11: Verify the Scaling Decisions

This final step of the KEDA K3s autoscaling tutorial confirms both workers actually reacted. Check the KEDA-managed HPA objects, which KEDA creates automatically for every ScaledObject:

Bash
kubectl -n rabbitmq get hpakubectl -n http-worker get hpa

Look at the current replica counts directly:

Bash
kubectl -n rabbitmq get deployment rabbitmq-workerkubectl -n http-worker get deployment http-worker

Check KEDA's own view of each trigger, including whether it currently sees the trigger as active:

Bash
kubectl -n rabbitmq describe scaledobject rabbitmq-worker-scalerkubectl -n http-worker describe scaledobject http-worker-scaler

In the describe output, look for the Conditions section. Active: True means KEDA detected load and told the HPA to scale up. Active: False after the cooldown period means it will scale the RabbitMQ worker back toward zero. Also, you can watch it live:

Bash
watch kubectl -n rabbitmq get pods,hpa,scaledobject
  • When the queue is empty for 60 seconds, the RabbitMQ worker scales back down to 0 pods.
  • When the request rate drops for 90 seconds, the HTTP worker scales back down to 1 pod, its set minimum.

Conclusion

At this point, you have a working KEDA K3s autoscaling tutorial with two real signals, not just a CPU demo. One trigger scales on RabbitMQ queue depth, the other on a Prometheus query. Both scale real deployments up and down automatically.

This setup works the same way on any K3s cluster, lab or production. You don't need a public cloud provider's autoscaling service to make it work.

We hope you enjoy this guide. For more detailed information, you can check the KEDA official documentation.

Prev PostKubernetes Disaster Recovery Drill: Restore Test With Velero/Longhorn
Category:Performance & Scaling
Tag:

We use cookies to run PerLod and to understand how the site is used. You can accept everything, or open Manage to turn individual categories off. Cookie policy.