The Complete Longhorn Troubleshooting Guide for Kubernetes Storage

Longhorn is a popular storage system for Kubernetes. It keeps your data safe by copying it across many nodes. But sometimes things break. A volume gets stuck. A replica fails. A node runs out of space. This guide shows you how to fix Longhorn degraded volume problems step by step.

Longhorn volumes always show one of a few states in the UI and in the volumes.longhorn.io custom resource. Knowing these states makes every fix easier.

• Healthy: All replicas are working and in sync.
• Degraded: The volume works, but it has fewer healthy replicas than it should.
• Faulted: The volume cannot attach because no replica is usable.
• Attaching / Detaching: The volume is changing state and sometimes gets stuck here.
• Unknown: Longhorn cannot report the volume's state because a node is down.

You can check any volume's state and robustness directly from the command line:

STATE tells you if the volume is attached, detached, attaching, or detaching. ROBUSTNESS tells you if it is healthy, degraded, or faulted.

This guide assumes you already have Longhorn installed on a Kubernetes or K3s cluster. If you are setting up Longhorn on K3s with NVMe disks for the first time, you can follow our Longhorn on K3s with NVMe setup guide.

The current stable release line is Longhorn v1.12, with v1.12.1 as the latest patch release. If you need to install or upgrade, you can use Helm:

Confirm all pods are running before you troubleshoot anything:

Every pod should show Running and 1/1 or 2/2 ready. If pods are crashing here, you must fix that first. Because a broken master node will cause every volume problem below.

You don't need any extra tools to do this. Just kubectl is enough. But it helps to remember these three commands, since you will use them a lot in this guide:

A degraded volume means one or more replicas failed or are missing. The most common reason people search to fix Longhorn degraded volume issues is a missing or failed replica, usually caused by a node restart, a disk error, or a network timeout.

First, list all replicas for the volume and check their state:

Look for any replica showing error, failed, or stopped when the others show running. Also, you can open the Longhorn UI, click the Volumes tab, click the degraded volume's name, and scroll to the Replicas section. Failed replicas are marked in red.

Before deleting anything, read the logs. This step matters because if you skip it, you may just recreate the same failure:

Common causes are disk I/O errors, a node running out of memory, or a lost network connection between nodes.

If the replica is dead and the node is healthy, delete it so Longhorn rebuilds a fresh copy:

Longhorn's scheduler automatically creates a new replica on a healthy node and starts copying data from a good replica. Watch the rebuild with:

When the new replica shows running and the volume's ROBUSTNESS changes back to healthy, the problem is solved.

Sometimes Longhorn keeps rebuilding the replica on the same bad node or disk, and it fails again and again. This is called a rebuild loop. To break it:

• Open the Longhorn UI and go to the Nodes tab.
• Click Edit node and disks on the problem node.
• Turn off Scheduling for that node, or just for the failing disk.
• Save, then delete the failed replica again using the command above.

With scheduling disabled, Longhorn is forced to place the new replica on a different and healthy node. Once the rebuild finishes and the volume is healthy again, you can re-enable scheduling on the original node after checking its disk health.

A faulted volume cannot attach, because every replica is broken. A stuck attach/detach loop is different. The volume keeps switching between attaching and detaching and never mounts. These can look similar to a degraded volume, so always check the real state first.

Check a stuck attachment with:

Also, check the matching VolumeAttachment object:

If the pod using the volume is gone but Longhorn still thinks it is attached, force a detach request:

Wait a few seconds, then reattach by restarting the pod that uses the PVC.

If the patch command does not help, use the Longhorn UI. Open the volume, click the menu, and choose Detach with the force option. You can also do this through the API:

If the volume is fully Faulted, open the Volume tab, filter by status Faulted, click on the volume, and use the Salvage button if it has more than one replica. Longhorn will pick the least damaged replica and rebuild the rest from it.

If salvage does not work and only one replica is left, you can manually mark that replica as not failed in its CRD:

Find the failedAt field and clear its value, then save. This tells Longhorn to try mounting that replica again. Only do this as a last resort, and always keep a backup first if one exists.

Disk pressure is a leading cause of degraded volumes, and clearing it helps you fix Longhorn degraded volume replicas. A full disk cannot accept new replica data. 

To fix this issue, follow the steps below.

First, check if the node or disk is the real cause of the problem:

Look at the diskStatus section of the output:

If you see status: "False" with reason: DiskPressure, the disk cannot accept new data. If you see Schedulable: false, no replicas can be placed there at all.

• Delete old, unused snapshots and backups you no longer need through the UI's Volume > Snapshots tab.
• Add a new disk or a new node with more free space.
• If a physical disk is failing, check the hardware directly:

If a disk is dying, you must move its data off before it fails completely:

This tells Longhorn to stop placing new replicas there and to move existing ones to healthy nodes.

The engine image is the software version Longhorn uses to run each volume. Problems appear after an upgrade, when old volumes still use an old engine image that no longer matches the installed Longhorn manager.

Check engine image status:

If an image shows incompatible: true or stays stuck in deploying, the volume's engine needs to be upgraded.

In the Longhorn UI, go to the volume, click the menu, and choose Engine Upgrade, then pick the current default engine image. For a batch upgrade after a Longhorn version update, use the UI's Volume list, select all volumes on the old engine, and apply Engine Upgrade together. 

Always do this only after confirming the volume is healthy, not degraded, since upgrading an already broken volume can make recovery harder.

Backup targets connect Longhorn to an S3 bucket, NFS share, or similar storage for offsite backups. Errors here usually come from a wrong URL or a permissions problem, not from the volume itself.

S3 backup target checklist: Your backup target URL must end with a forward slash:

Missing the trailing slash is the most common S3 backup error. Make sure the AWS region is correct, and that the credential secret referenced in Settings > Backup Target actually exists:

NFS backup target checklist: Longhorn only supports NFS versions 4.0, 4.1, and 4.2. Check what your server supports:

Your backup target URL should look like this:

If backups fail with permission errors, your NFS export probably uses root_squash. Fix this by changing it to no_root_squash in /etc/exports on the NFS server, or by running chmod o+w on the exported folder.

After any change to a backup target, open the Backup tab in the Longhorn UI. If it shows an error, the target is misconfigured. If it loads empty with no error, the target is fine; you just have no backups yet.

When a node goes down completely, its replicas become unavailable, and any volume that only had a replica there turns degraded or faulted.

Before you fix anything, make sure the node is actually down and not just slow:

If the node shows NotReady, Longhorn will detect this and mark its replicas as failed after a short timeout.

As long as the volume still has at least one healthy replica on another node, Longhorn rebuilds the missing replica automatically on a working node. You do not need to do anything except monitor progress:

If the failed node returns and its disks are healthy, Longhorn may reuse the old replica data instead of rebuilding from scratch, which saves time. If the node is permanently gone, remove it from Longhorn so the scheduler stops trying to use it:

Only do this after confirming through kubectl get pods -A -o wide that no workload still expects that node to exist.

These best practices stop you from having to fix Longhorn degraded volume errors over and over:

• Keep at least three replicas per important volume, spread across at least three nodes.
• Set up disk pressure alerts before disks actually fill up, not after.
• Test your backup target every week by running a small manual backup.
• Upgrade Longhorn engine images soon after every Longhorn version upgrade.
• Use fast and reliable local NVMe storage instead of slow network-attached disks.

If you keep needing to fix Longhorn degraded volume issues because of slow disks, network timeouts, or shared CPU load, the real problem is your hardware, not Longhorn. Moving your stateful workloads to high-performance dedicated servers gives you dedicated NVMe storage and stable latency, so replicas rebuild faster and stay healthy.

Now you know how to fix Longhorn degraded volume problems safely, from replica failures and stuck attachments to disk pressure, engine image issues, and backup errors. Always check the volume's state first, read the logs before deleting anything, and isolate a bad node or disk before letting Longhorn rebuild. Follow these steps in order, and most Longhorn problems become quick fixes.

We hope you enjoy this guide. For more detailed information on Longhorn error states, log locations, and CRDs, check the Longhorn Official Troubleshooting Knowledge Base.