How to Set Up JupyterHub on a GPU Server

Updated on Apr 17, 2026
Joey SH
9 MINS READ
Table of Contents
JupyterHub GPU Server Setup using Docker

JupyterHub lets multiple users run Jupyter notebooks on one server at the same time. Instead of each team member buying their own GPU hardware, everyone can share one GPU server and run their notebooks from any web browser. This guide will show you JupyterHub GPU Server Setup using Docker.

You will learn how to install the NVIDIA drivers, set up Docker with GPU support, and configure JupyterHub so each user gets their own isolated notebook environment with full GPU access.

Hosting option: PerLod Hosting is ideal for JupyterHub setup because it offers bare-metal performance, full root access, and 24/7 support.

Requirements for JupyterHub GPU Server Setup

Before you begin the setup, make sure to have these requirements:

  • Operating System: Ubuntu 22.04 or Ubuntu 24.04 LTS.
  • GPU: NVIDIA GPU (RTX 4090, A5000, A6000, or similar).
  • Docker: Docker 19.03 or newer.
  • NVIDIA Drivers: Installed and working. Run the nvidia-smi command to verify them.
  • Root Access: SSH access with sudo or root permissions.

If you need a GPU server, PerLod Hosting offers dedicated GPU servers with RTX 4090, A5000, and A6000 GPUs. These come pre-installed with Ubuntu and offer full root access, making them ideal for JupyterHub setups.​

Connect to your server via SSH and check if the NVIDIA GPU is detected with the commands below:

Bash
ssh root@YOUR_SERVER_IPnvidia-smi

In your output, you should see a table showing your GPU name, driver version, and CUDA version. If this command fails, you need to install the NVIDIA drivers first.

Run the system update and upgrade to ensure having the latest security patches and software versions:

Bash
sudo apt update -ysudo apt upgrade -y

Step 1. Install Docker for JupyterHub GPU Setup

As we said, we want to set up JupyterHub with Docker, which lets you run JupyterHub and user notebooks in isolated containers.

If you have old versions of Docker installed on your server, remove them with the command below:

Bash
for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do     sudo apt-get remove $pkgdone

Then, add Docker's official repository with the following commands:

Bash
sudo apt install ca-certificates curl -ysudo install -m 0755 -d /etc/apt/keyringssudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.ascsudo chmod a+r /etc/apt/keyrings/docker.asc echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Install Docker and the plugins with the commands below:

Bash
sudo apt updatesudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y

Verify Docker is installed by checking its version:

Bash
docker --version

To run Docker commands without sudo, add your user to the Docker group:

Bash
sudo usermod -aG docker $USER

Log out and log in again to apply the changes.

Step 2. Install NVIDIA Container Toolkit for JupyterHub GPU Setup

To let Docker containers use your GPU, you must install Docker. Without this, your notebooks cannot access GPU power for training models.

Add the NVIDIA repository with the commands below:

Bash
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

Install the toolkit by using the commands below:

Bash
sudo apt updatesudo apt install nvidia-container-toolkit -y

Configure Docker to use the NVIDIA runtime with the command below:

Bash
sudo nvidia-ctk runtime configure --runtime=docker

This command changes /etc/docker/daemon.json, so Docker knows how to talk to your GPU.

Restart Docker to apply the changes:

Bash
sudo systemctl restart docker

Test GPU access in Docker with the following command:

Bash
docker run --rm --gpus all nvidia/cuda:12.5.0-base-ubuntu22.04 nvidia-smi

You should see the same GPU table as before. This means Docker can now use your GPU.

Step 3. Set up the JupyterHub Project Folder

You must create a project directory for your JupyterHub setup and navigate to it with the commands below:

Bash
sudo mkdir ~/jupyterhub-gpucd ~/jupyterhub-gpu

In your project directory, create a Docker network for JupyterHub with the command below:

Bash
docker network create jupyterhub-network

Also, create a volume for storing JupyterHub data:

Bash
docker volume create jupyterhub-data

These prepare the foundation for your multi-user notebook server.

Step 4. Create JupyterHub Configuration File

In this step, you must create a configuration file which tells JupyterHub how to behave.

Create the JupyterHub config file with your desired text editor:

Bash
sudo nano jupyterhub_config.py

Add the following content to the file:

Python
from dockerspawner import DockerSpawnerimport os # Use Docker to spawn user notebooksc.JupyterHub.spawner_class = 'dockerspawner.DockerSpawner' # GPU-enabled PyTorch image from NVIDIAc.DockerSpawner.image = 'nvcr.io/nvidia/pytorch:24.04-py3' # Connect to the JupyterHub networkc.DockerSpawner.network_name = 'jupyterhub-network' # Enable GPU access for all containersc.DockerSpawner.extra_host_config = {    'runtime': 'nvidia',    'device_requests': [        {            'Driver': 'nvidia',            'Count': -1,  # Use all GPUs            'Capabilities': [['gpu']]        }    ]} # Environment variables for GPU accessc.DockerSpawner.environment = {    'NVIDIA_VISIBLE_DEVICES': 'all',    'NVIDIA_DRIVER_CAPABILITIES': 'compute,utility'} # Remove containers when users log outc.DockerSpawner.remove = True # User storage - each user gets their own folderc.DockerSpawner.volumes = {    'jupyterhub-user-{username}': '/home/jovyan/work'} # Hub settingsc.JupyterHub.hub_ip = '0.0.0.0'c.JupyterHub.hub_connect_ip = 'jupyterhub' # Allow anyone to sign up (change this for production)c.JupyterHub.authenticator_class = 'nativeauthenticator.NativeAuthenticator'c.NativeAuthenticator.open_signup = True

This configuration file does several important things, including:

  • Uses DockerSpawner to create a container for each user.
  • Uses the NVIDIA PyTorch image, which already has CUDA and PyTorch installed.
  • Enables GPU access through the NVIDIA runtime.
  • Gives each user their own storage volume.
  • Allows users to sign up with a username and password.

Tip: If you want to run inference servers alongside JupyterHub, you can check this guide on AI Serving with Docker on a Dedicated Server to learn how to deploy vLLM or Triton on the same GPU host for a complete AI workload platform.

Step 5. Set up JupyterHub Docker Compose File

With Docker Compose, you can easily start JupyterHub with all the right settings.

Create the Docker compose YAML file with your desired text editor:

Bash
sudo nano docker-compose.yml

Add the following content to the file:

YAML
version: '3.8' services:  jupyterhub:    build: .    container_name: jupyterhub    ports:      - "8000:8000"    volumes:      - /var/run/docker.sock:/var/run/docker.sock      - ./jupyterhub_config.py:/srv/jupyterhub/jupyterhub_config.py      - jupyterhub-data:/srv/jupyterhub/data    environment:      - DOCKER_NETWORK_NAME=jupyterhub-network    networks:      - jupyterhub-network    restart: unless-stopped volumes:  jupyterhub-data:    external: true networks:  jupyterhub-network:    external: true

This will tell Docker how to run JupyterHub and connect it to the network and storage you created earlier.

Step 6. Create a Dockerfile for JupyterHub

At this point, you must create the Dockerfile, which builds the JupyterHub image with all needed packages by using the command below:

Bash
sudo nano Dockerfile

Add the following content to the Dockerfile:

Docker
FROM jupyterhub/jupyterhub:4.1 # Install Docker spawner and native authenticatorRUN pip install dockerspawner jupyterhub-nativeauthenticator # Copy configurationCOPY jupyterhub_config.py /srv/jupyterhub/jupyterhub_config.py WORKDIR /srv/jupyterhub CMD ["jupyterhub", "-f", "/srv/jupyterhub/jupyterhub_config.py"]

Step 7. Start and Access JupyterHub

Now that you have configured everything, you can build the Docker image and start JupyterHub with the commands below:

Bash
docker-compose builddocker-compose up -d

To check if it is running, use the command below:

Bash
docker-compose ps

You should see the JupyterHub container in a "running" state.

To access JupyterHub, open your web browser and go to:

Bash
http://YOUR_SERVER_IP:8000

You will see the JupyterHub login page. Since you enabled open signup, you can create a new account by clicking the signup link.

After logging in, JupyterHub will start a new Docker container for you with GPU access. This might take a minute the first time because Docker needs to download the NVIDIA PyTorch image.

Step 8. Test GPU Access through JupyterLab

Once you are inside JupyterLab, open a new notebook and run this code to check GPU access:

Python
import torch print(f"CUDA available: {torch.cuda.is_available()}")print(f"GPU Name: {torch.cuda.get_device_name(0)}")print(f"GPU Memory: {torch.cuda.get_device_properties(0).total_memory / 1024**3:.1f} GB")

You should see your GPU name and memory. If CUDA shows as "True", everything is working correctly.

You can also run nvidia-smi in a terminal cell:

Bash
!nvidia-smi

This shows the full GPU status, including temperature, memory usage, and running processes.

How to Enable HTTPS for JupyterHub Security?

You should not run JupyterHub without HTTPS on a public network. HTTPS protects passwords and data as they travel between browsers and your server.

Option 1: Let's Encrypt (Free SSL)

If you have a domain name pointing to your server, you can use Let's Encrypt for free SSL certificates.

Add these lines with your domain name to your jupyterhub_config.py file:

Python
c.JupyterHub.ssl_key = '/etc/letsencrypt/live/yourdomain.com/privkey.pem'c.JupyterHub.ssl_cert = '/etc/letsencrypt/live/yourdomain.com/fullchain.pem'

Install Certbot and get a certificate by using the commands below:

Bash
sudo apt install certbot -ysudo certbot certonly --standalone -d yourdomain.com

Option 2: Self-Signed Certificate

For internal use or testing, you can create a self-signed certificate with the command below:

Bash
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \    -keyout jupyterhub.key -out jupyterhub.crt \    -subj "/CN=localhost"

Add these to your configuration:

Python
c.JupyterHub.ssl_key = '/path/to/jupyterhub.key'c.JupyterHub.ssl_cert = '/path/to/jupyterhub.crt'

For a complete security checklist for your GPU server, including container hardening and NVIDIA MIG isolation, you can check this guide on Best Practices for GPU Hosting Environments Security.

GPU Time-Slicing for More Users (Optional)

If you have many users but limited GPUs, you can use GPU time-slicing, which lets multiple users share one GPU by taking turns.

With time-slicing, if you have one GPU and set four time slices, four users can work on that GPU at the same time. Each user gets a fraction of the GPU's power.

This is useful when users run light tasks like inference or small training jobs. For heavy training, each user should get a dedicated GPU.

Troubleshooting Common Issues in JupyterHub GPU Setup

Here are the most common errors in JupyterHub GPU server setup and their solutions:

GPU Not Visible in Container: If nvidia-smi works on the host but not in the container, check these things:

  • Make sure the NVIDIA Container Toolkit is installed.
  • Check that Docker is configured with the NVIDIA runtime.
  • Verify the extra_host_config has the correct runtime settings.

Container Fails to Start: If user containers fail to start:

  • Check if the Docker image exists with the docker images command.
  • Look at logs with the docker-compose logs jupyterhub command.
  • Make sure the network exists with the docker network ls command.

Connection Refused Error: If you cannot connect to port 8000:

  • Check if the container is running with docker-compose ps.
  • Check firewall settings with sudo ufw status.
  • Allow port 8000 with sudo ufw allow 8000.

Why Use PerLod Hosting for JupyterHub GPU Setup?

PerLod Hosting provides dedicated GPU servers that work great for JupyterHub deployments. Here is why PerLod is a good choice:

  • Full Root Access
  • Bare-Metal Performance
  • Ubuntu Pre-installed
  • NVMe Storage
  • 24/7 Support
  • Predictable Pricing

PerLod offers GPU servers with RTX 4090 (24GB), A5000 (24GB), and A6000 (48GB) GPUs, which are perfect for machine learning teams who need reliable GPU access.

Conclusion

JupyterHub GPU Server Setup gives your team a powerful platform for machine learning and data science work. With Docker and the NVIDIA Container Toolkit, each user gets their own isolated environment with full GPU access.

For teams that need reliable GPU servers, PerLod Hosting offers dedicated GPU servers with full root access, NVMe storage, and 24/7 support. This lets you focus on building AI applications instead of managing infrastructure.

We hope you enjoy this guide. Subscribe to our X and Facebook channels to get the latest updates and articles on GPU and AI hosting.

JupyterHub is a multi-user server that manages Jupyter notebook servers for many users. Each user gets their own notebook environment while sharing the same GPU hardware.

Docker isolates each user's notebook in a separate container, which prevents users from interfering with each other.

For small teams doing inference or fine-tuning, an RTX 4090 (24GB) works well. For larger models or more users, consider an A6000 (48GB) or multiple GPUs.

Prev PostBuild High-Performance OpenAI Compatible Hosting
Next PostHow to Build Distributed Training Setup with Horovod
Category:AI & GPU Infrastructure
Tag:

We use cookies to run PerLod and to understand how the site is used. You can accept everything, or open Manage to turn individual categories off. Cookie policy.