Tutorial, Ubuntu 24.04
Install the NVIDIA driver and CUDA on Ubuntu. Check first, install only what’s missing.
Most guides start by removing and reinstalling drivers. Usually you need less than that. Here’s the check-first way, on Ubuntu 24.04.
NVIDIA L4, 24 GB
Forge L4 = one 2 GB slice, 8% of the card’s memory, yours alone. Processing is shared in turns.
NVIDIA L40S, 48 GB
Forge L40S = one 8 GB slice, 17% of the card’s memory, yours alone. Processing is shared in turns.
The short answer
Run nvidia-smi first. If it shows your GPU, the driver is working and you can skip it. If not, install the server driver with sudo ubuntu-drivers install --gpgpu and reboot. Install the CUDA toolkit from NVIDIA’s repository only if you compile CUDA code yourself; it’s separate from the driver.
The decoder below turns each nvidia-smi error into the fix.
The driver lets Linux talk to the GPU. The CUDA toolkit is for building CUDA programs. People mix the two up, install both twice, and end up with a driver that won’t load.
So start with one command and install only what it tells you is missing.
In this guide you’ll learn
- Check with nvidia-smi
- An error decoder for every nvidia-smi message
- Install the driver the way Ubuntu supports
- Install the CUDA toolkit, only if you need it
Chapter 1
What does nvidia-smi say?
Run nvidia-smi and pick what you see.
What to do
Install the server driver.
The driver isn’t installed. Let Ubuntu pick the right server driver:
sudo ubuntu-drivers list --gpgpu
sudo ubuntu-drivers install --gpgpu
sudo reboot
Chapter 2
Install the driver the Ubuntu way
Ubuntu’s own tool picks a driver that matches your GPU and kernel, and prefers signed builds that work with Secure Boot.
See what it recommends
sudo ubuntu-drivers list --gpgpu
Install it
sudo ubuntu-drivers install --gpgpu sudo reboot
Check
nvidia-smi
Chapter 3
Install the CUDA toolkit, if you need it
NVIDIA’s own guide says it plainly: the cuda-toolkit package installs the toolkit only and doesn’t install a driver.
Add NVIDIA’s repository
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb sudo dpkg -i cuda-keyring_1.1-1_all.deb sudo apt update
Install the toolkit
sudo apt install -y cuda-toolkit
Put it on your PATH
echo 'export PATH=/usr/local/cuda/bin:$PATH' >> ~/.bashrc source ~/.bashrc nvcc --version
MythThe CUDA version in nvidia-smi is the one installed.
That number is the newest CUDA the driver supports. The installed toolkit is what nvcc --version reports.
MythYou need the toolkit to run AI models.
You need a working driver. The toolkit is for compiling CUDA code; check your framework’s install guide before adding it.
Chapter 4
On a LiteKeep Forge
A Forge is a virtual server with a slice of an NVIDIA GPU, and the driver and CUDA support come with it.
Run nvidia-smi: it shows your slice and its memory, about 2 GB on a Forge L4 and about 8 GB on a Forge L40S. Skip the driver install. If nvidia-smi fails, open a support ticket from your server’s page instead of reinstalling, so the GPU stays set up the way the host expects.
Next: serve a model with vLLM or generate images with ComfyUI.
Now it’s your move
Check with nvidia-smi, install the driver only if it’s missing, and the toolkit only if you build CUDA code.
Skip all of it on a Forge: the driver is already there.