GPU Monitoring
GPU monitoring from the terminal. nvidia-smi gives a snapshot or a refreshed view via watch. nvtop is an interactive monitor that handles multiple GPUs and shows utilization graphs over time.
One-shot nvidia-smi
Section titled “One-shot nvidia-smi”Print a single snapshot of GPU utilization, memory used and free, temperature, and running processes.
nvidia-smiLive refresh with watch
Section titled “Live refresh with watch”Run nvidia-smi on a fixed interval and refresh the display in place. Useful for watching utilization and memory during training runs.
watch -n 1 nvidia-smiAdjust the refresh interval as needed:
watch -n 2 nvidia-smi # refresh every 2 secondswatch -n 0.5 nvidia-smi # refresh every 500msExit with Ctrl+c.
Interactive monitor with nvtop
Section titled “Interactive monitor with nvtop”nvtop is a interactive monitor for GPUs. It shows per-GPU utilization graphs over time, memory usage, temperature, and the processes using each GPU. Better than watch + nvidia-smi when monitoring multiple GPUs or watching utilization patterns over time.
Install on Ubuntu and Debian:
sudo apt install nvtopInstall on Arch:
sudo pacman -S nvtopRun:
nvtopExit with q.