How to Split and Share GPUs in Kubernetes Using NVIDIA GPU Operator

A single GPU usually sits idle. One pod waits, one team blocks another, and expensive hardware runs at only a small part of its real power. This guide shows you how to share NVIDIA GPUs in Kubernetes using the NVIDIA GPU Operator, so many pods can use the same physical card safely. We will cover both time-slicing (software sharing) and MIG (hardware partitioning).

Most AI and inference jobs don't need a whole GPU at all times. A small model, a dev notebook, or a batch job use only 10 to 20% of a GPU's power. If you give the whole GPU to just one pod, the rest goes to waste.

When you share NVIDIA GPUs in Kubernetes, the scheduler can split one physical card into several slots for different pods, teams, or namespaces, instead of buying one GPU per workload.

The NVIDIA GPU Operator supports two ways to do this:

• Time-slicing: The GPU switches between workloads fast, like a CPU switching between apps. No memory isolation, but it works on almost any NVIDIA GPU.
• MIG (Multi-Instance GPU): The GPU is split into smaller and separate mini-GPUs, each with its own memory and power. Only works on MIG-capable cards like the A100, A30, H100, H200, and newer B-series GPUs.

Knowing both is the key skill for sharing GPUs well. Your choice depends on your hardware and how much isolation your workloads need.

This guide assumes the NVIDIA GPU Operator is already installed and healthy on your cluster. If you have not installed it yet, follow our step-by-step K3s bare-metal install guide, then come back here.

You will need:

• A working Kubernetes or K3s cluster with kubectl access.
• NVIDIA GPU Operator in the latest stable version already running, with the gpu-operator namespace healthy.
• At least one node with an NVIDIA GPU, confirmed with nvidia-smi on the host.
• helm installed on your workstation, since we patch the Operator's ClusterPolicy with Helm-managed ConfigMaps.
• For the MIG section only: a MIG-capable GPU such as an A100, A30, H100, or H200. Time-slicing works on almost any modern NVIDIA GPU, including consumer and older data center cards.

Quick check that the Operator is healthy before you continue:

Both commands should show Running/Completed pods and a ready ClusterPolicy.

Note: If you are testing on a single rented card, a noisy shared box can mess up your results. A dedicated GPU server gives you full root access to the physical GPU, so no other tenant affects your time-slicing or MIG tests.

Time-slicing is the easiest way to share NVIDIA GPUs in Kubernetes. The GPU Operator makes one physical GPU appear as several nvidia.com/gpu resources, and Kubernetes distributes these to different pods. NVIDIA's driver switches the GPU between the pods' processes many times per second, so each pod gets a turn.

Note: There is no memory isolation between replicas. If one pod uses too much GPU memory, it can affect other pods on the same card. Keep that in mind when picking which workloads to share.

First, create a file called time-slicing-config.yaml:

Paste this content into the file:

This tells the Operator to split every GPU into 4 virtual copies. So a node with one physical GPU now shows 4 nvidia.com/gpu resources. A node with two GPUs shows 8, and so on.

Apply it to the cluster with:

The ConfigMap alone does nothing until the device plugin knows to use it. You should patch the ClusterPolicy so it references your new config:

This is how the Operator shares NVIDIA GPUs in Kubernetes. The devicePlugin.config field tells the device plugin which sharing rules to use. The "default": "any" part applies the any rules from your ConfigMap to every node.

Wait about 30 seconds, then confirm the device plugin and GPU feature discovery pods restarted:

Use the command below to check the node's advertised GPU count:

Look for a block like this near Capacity and Allocatable:

One physical GPU now shows up as 4 allocatable units. The -SHARED label tells you the Operator has time-sliced this GPU, so it's not dedicated to just one pod.

To verify the GPUs are being shared, you can create a deployment that requests more GPU replicas than you have physical GPUs. Create a test file:

Add this content:

Apply it and check that all pods start, even on a single-GPU node:

All 4 pods should reach Running, sharing the same physical GPU. Check the logs to confirm each one really runs CUDA work:

You should see Test PASSED and Done repeated several times. This proves time-slicing is working, that four separate pods are sharing one GPU card. Finally, clean up when done:

If your cluster has different GPU models, you can define multiple profiles in one ConfigMap and apply them per node instead of cluster-wide. Save the following content as time-slicing-config-fine.yaml:

Apply it. Don't set a default value, so each node needs its own label to pick a config:

Now label each node with the profile it should use:

This way, different node groups can share NVIDIA GPUs in Kubernetes with different replica counts, matching each GPU model's real capacity.

MIG (Multi-Instance GPU) is a hardware feature on cards like the A100, A30, H100, and H200. Instead of switching time between pods, MIG splits one GPU into several smaller GPUs. Each one gets its own memory and compute cores. This gives real isolation: a crash or memory overrun in one slice can't touch another slice.

Tips: Use MIG when you need strong isolation between tenants, teams, or untrusted workloads, and your GPU supports it. Use time-slicing when your GPU can't do MIG, or when soft sharing is good enough.

The GPU Operator supports two MIG strategies:

• single: Every MIG instance is exposed as the same nvidia.com/gpu resource. Simple to use, but pods cannot request a specific MIG size.
• mixed: Each MIG profile is exposed as its own resource type, like nvidia.com/mig-1g.5gb. Pods can request exact GPU slice sizes.

If your Operator was installed with default settings, patch the existing ClusterPolicy to change the strategy without reinstalling:

You can use single in the above command instead of mixed if you prefer the simpler mode.

Check if the GPU Operator sees MIG support on your node:

Look for "nvidia.com/mig.capable": "true" in the output. If it's missing or says false, your GPU doesn't support MIG. You can use time-slicing instead to share NVIDIA GPUs in Kubernetes on that node.

The GPU Operator has built-in MIG profiles. Just label the node to use one. For an A100 40GB card split into three equal 3g.20gb slices:

For a mixed split, if you want a few different slice sizes on the same card, you can use:

The MIG Manager daemonset watches for this label. It drains GPU pods from the node, reconfigures the physical GPU, then reports back when done. No reboot needed on GPU Operator v24.9 and later.

Give it a minute, then check the reconfiguration status:

You should see "nvidia.com/mig.config.state": "success" along with slice counts like "nvidia.com/gpu.slices.gi": "3". Also, you can SSH into the node and run:

This lists each MIG instance as a separate device, which proves the physical GPU is now split into isolated units.

If you set the strategy to mixed, you should request the exact slice type in your pod spec. Save the following content as mig-test-pod.yaml:

Apply it and check the logs:

A Test PASSED and Done message confirms the pod ran real CUDA work on its own isolated MIG slice. You can clean up with kubectl delete -f mig-test-pod.yaml.

If you used the single strategy instead, every pod just requests nvidia.com/gpu: 1 like normal, and the scheduler places it on whichever MIG slice is free.

You can add time-slicing on top of MIG slices for better sharing. This helps when you already split a GPU into MIG instances but still have more workloads than slices. In your time-slicing ConfigMap, use the MIG resource name instead of the GPU name:

Apply and patch the ClusterPolicy the same way in the time-slicing example. Now each MIG slice is shared by 2 pods through time-slicing, giving you two layers of sharing on one card. 

This is a more advanced way to share NVIDIA GPUs in Kubernetes when requests are higher than even your hardware-partitioned slices can handle.

Setting up sharing is only half the job. You also need to check it works correctly under load. Run these checks:

GPU utilization per pod: Check DCGM Exporter metrics (the Operator installs this by default) with curl -s http://<dcgm-exporter-ip>:9400/metrics | grep DCGM_FI_DEV_GPU_UTIL. Compare the numbers across pods on the same GPU.

Memory isolation on MIG: Run nvidia-smi -q -d MEMORY in two pods on different MIG slices at the same time. Each pod should only see its own slice's memory, not the full card.

Time-slicing fairness: Run the same CUDA test in several time-sliced pods at once and compare finish times. Times that are close to each other mean the GPU is sharing time fairly.

Node resource counts: kubectl describe node <node-name> should always match your expected replica or MIG slice count. If it doesn't match, the ConfigMap or label didn't apply.

Keep these rules in mind whenever you share NVIDIA GPUs in Kubernetes in a real cluster:

• Set failRequestsGreaterThanOne: true in time-slicing configs. This stops a pod from wrongly getting extra compute by asking for more than one replica.
• Use ResourceQuota and LimitRange per namespace, so one team can't grab all the GPU slots and starve others.
• Never mix untrusted and multi-tenant workloads on time-sliced GPUs. Use MIG instead, since time-slicing gives no memory isolation.
• Watch DCGM Exporter alerts for memory pressure or ECC errors. A runaway process on a shared GPU can hurt every other pod on it, especially with time-slicing.
• Write down which nodes use time-slicing, MIG, or dedicated GPUs. Mixed modes across a cluster are easy to forget and hard to debug later.
• Restart the device plugin daemonset after editing a config map, with kubectl rollout restart -n gpu-operator daemonset/nvidia-device-plugin-daemonset. The Operator does not auto-detect ConfigMap changes.

At this point, you have learned to share NVIDIA GPUs in Kubernetes with time-slicing and MIG, using GPU Operator. You created ConfigMaps, labeled nodes, ran test pods, and checked isolation with DCGM metrics.

• Time-slicing gives fast, flexible sharing on almost any GPU.
• MIG gives hardware-level isolation on cards like the A100 and H100. 

Pick the mode that fits your workload, use the checklist above, and recheck node resource counts after every change.

We hope you enjoy this guide. For more detailed information about Time-Slicing, check the official NVIDIA GPU Operator Docs.