CUDA Setup on Ubuntu 24.04

Install and verify CUDA on Ubuntu 24.04 with an NVIDIA GPU and a working driver. The driver alone provides nvidia-smi and the CUDA runtime, but the Toolkit (nvcc and development tools) is a separate install. This page covers both and confirms GPU access from Python.

Verify the driver and CUDA runtime

Confirm the GPU driver is working. The CUDA Version field shows the maximum runtime version supported by the driver, not proof that the CUDA Toolkit is installed.

nvidia-smi

Expected output includes a line similar to:

Driver Version: 590.xx  CUDA Version: 13.1  GPU: NVIDIA GeForce RTX 3060

Check if the CUDA compiler exists

If nvcc is missing, the CUDA Toolkit is not yet installed and the next steps apply.

nvcc --version

When the Toolkit is missing, the output is:

Command 'nvcc' not found

Add the NVIDIA CUDA repository

On Ubuntu 24.04, CUDA packages require the NVIDIA CUDA apt repository. Add the keyring and refresh the package index.

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update

Install the CUDA Toolkit

Install the Toolkit. This brings in nvcc and supporting development tools.

sudo apt install cuda-toolkit-13-1

Configure environment variables

Append the CUDA bin and lib64 directories to PATH and LD_LIBRARY_PATH. This makes nvcc available on the command line and lets the dynamic linker find the CUDA shared libraries.

echo 'export PATH=/usr/local/cuda-13.1/bin:$PATH' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda-13.1/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc

Reload the shell configuration and clear the command hash table so the new nvcc path resolves immediately.

source ~/.bashrc
hash -r

Confirm the install.

nvcc --version

Expected output:

Cuda compilation tools, release 13.1

Verify CUDA from Python

Activate the project virtual environment and run a quick check that PyTorch sees the GPU.

source venv/bin/activate

import torch

print("torch version:", torch.__version__)
print("cuda available:", torch.cuda.is_available())
print("cuda device count:", torch.cuda.device_count())

if torch.cuda.is_available():
    print("gpu name:", torch.cuda.get_device_name(0))
    print("cuda capability:", torch.cuda.get_device_capability(0))

python3 cuda_test.py

Expected output:

torch version: 2.10.0+cu126
cuda available: True
cuda device count: 1
gpu name: NVIDIA ...
cuda capability: (8, 6)