Zero-Downtime Hotpatching Linux kernel

Live patching or hotpatching is about applying a kernel bug or security fix to a running kernel without rebooting the machine. It is commonly used for critical and high-priority fixes without downtime. Multiple vendors provide supported services like Canonical Livepatch and Red Hat kpatch. This step-by-step guide will cover a zero-downtime Hotpatching Linux kernel.

This guide is provided by PerLod Hosting, which provides reliable Linux hosting and server solutions for businesses of all sizes.

How Live Patching or Hotpatching Works?

Live patch or hotpatch means a system loads a special kernel module called a patch module, which replaces certain kernel functions with fixed versions. The system will redirect function calls to the new code instead of directly changing the code. This means hotpatching can fix security issues while the system is running without needing a reboot.

It only works for small and functional-level fixes. For big changes, you will still need a full reboot. Vendors will let you know when a reboot is required.

Tools or vendor options for live patching include:

• Canonical Livepatch (Ubuntu)
• kpatch (Red Hat)
• Ksplice (Oracle)

You need to choose the tool supported by your distribution.

Prerequisites for Zero-Downtime Hotpatching Linux kernel

Before you start, you must back up sensitive data and have a rollback plan. Livepatching reduces reboot frequency but does not replace the normal patch lifecycle.

You need to verify your Linux kernel version and distro compatibility. Also, you must have root privileges for live patching.

Note: Some fixes cannot be applied by livepatch, and you will need a reboot. The live patching tools will report this. And it is recommended to test live patching in a non-production server at first.

Now let’s dive into live patching kernels on Linux distros.

Kernel Live patching in Ubuntu with Canonical Livepatch

Using Canonical Livepatch is the easiest supported option for Ubuntu LTS systems. First, run the system update and install the snapd package with the following commands:

sudo apt update
sudo apt install snapd -y

Then, install the canonical-livepatch package by using the snapd:

sudo snap install canonical-livepatch

Then, you must visit the Ubuntu LivePatch, sign in to your Ubuntu account, and get a Free for personal use token or an Ubuntu Pro token.

Then, enable the service with the token:

sudo pro attach machine-token

This command registers the host to Canonical’s livepatch service.

Next, you can check the status and applied patches by using the following command:

sudo canonical-livepatch status --verbose

Or, you can use:

sudo canonical-livepatch status

All applicable livepatch modules will be inserted. If a patch cannot be applied, the status will show that a kernel upgrade is required.

You can also use the following command to ask the client to recheck the server to get the latest patches:

sudo canonical-livepatch refresh

Note: Keep in mind that Livepatch will not cover all kernel fixes; you must keep regular upgrades.

If you plan to disable and remove Canonical Livepatch, you can run the following commands:

sudo canonical-livepatch disable
sudo snap remove canonical-livepatch

Linux Kernel Live patching in RHEL with kpatch

The kpatch tool is the only kernel live patching utility that is supported by Red Hat with the RPM modules provided by Red Hat repositories. It is supported on RHEL, CentOS, Fedora, and dnf-based distros.

With kpatch, you can subscribe to the currently installed kernels to the live patching stream or automatically subscribe any future kernel to the live patching stream.

To use auto patches, you need to install kpatch with the command below:

sudo dnf install kpatch kpatch-dnf -y

The kpatch-dnf is the plugin to auto-install kpatch patches from vendor repositories.

Then, you can easily enable automatic subscription with the following command:

sudo dnf kpatch auto

It installs available kpatch-patch packages for the running kernel and keeps future kernels subscribed.

Verify that all installed kernels are patched with the command below:

sudo kpatch list

If you prefer manual patching, proceed with these steps.

Check current kernel version:

uname -r

Then, search for a live patching package that matches the kernel version:

sudo dnf search $(uname -r)

Next, install the live patching package with the following command:

sudo dnf install "kpatch-patch = $(uname -r)"

This command installs and applies the latest live patches for the current kernel only.

Verify that all installed kernels are patched with the command below:

sudo kpatch list

To unload and remove a patch, you can run the following commands:

sudo kpatch unload module-name
sudo dnf remove kpatch-patch-package

If you want to stop auto-loading of patches at boot, run:

sudo systemctl disable --now kpatch

Linux Kernel Live patching in Oracle uptrack (ksplice)

Ksplice (Uptrack) is a tool provided by Oracle that lets you apply security patches to the Linux kernel without rebooting. It usually needs an Oracle support subscription unless you are running certain free versions of Oracle Linux.

Download the installer script and configure it with the following commands:

sudo wget -N https://ksplice.oracle.com/uptrack/install-uptrack-oc
sudo sh install-uptrack-oc -autoinstall

Then, list which patches are applied or available with the command below:

sudo uptrack-show

Finally, apply updates with the command below:

sudo uptrack-upgrade

This command installs the latest live patches immediately, without rebooting.

Best Practices for Linux Kernel Live Patching

Here are the best practices you should consider for Zero-Downtime Hotpatching Linux kernel:

1. Use vendor-supported livepatch tools: Each Linux system has its own live patching system. Use the tool officially supported by your distribution.

• Ubuntu → Canonical Livepatch
• RHEL (Red Hat Enterprise Linux, CentOS, AlmaLinux, Rocky Linux) → kpatch
• Oracle Linux → Ksplice

2. Keep backups and plan for restarts: Always keep tested backups so you can roll back if something goes wrong. And schedule a maintenance window for eventual reboots, so users are not surprised when downtime is needed.

3. Monitor system health after patching: After applying the live patch, check kernel logs for errors and warnings using journalctl -k and dmseg. Always check app performance and availability.

4. Use a test environment first: Apply new live patches on test servers before production. This helps confirm stability with your applications before rolling out to critical systems.

5. Track kernel and livepatch versions: It’s important to audit what patches are currently applied in your Linux system. Keep a record of this information for compliance, audits, or troubleshooting.

FAQs

Conclusion

With zero-downtime hotpatching Linux kernel with supported tools like Canonical Livepatch, kpatch, and Ksplice, administrators can apply kernel fixes seamlessly, reduce downtime, and improve security posture. However, hotpatching is not a complete replacement for traditional patching; some updates still require planned reboots.

We hope this guide is useful for you. Subscribe to us on Facebook and X to get the latest updates and articles.

For further reading:

Explore Horizontal vs Vertical Scaling in Dedicated Servers

Secure OpenSearch Clusters on Ubuntu

Configure DNSSEC and WAF in the VPS Server