# Why nvidia-smi is missing or cannot talk to the driver, and how to fix it

> Source: https://netray.co/troubleshoot/nvidia-smi-not-found
> Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems.
> Section: On-Prem AI Troubleshooting

## Short answer

nvidia-smi not found or unable to communicate almost always means the NVIDIA kernel module never loaded, and the two most common reasons are that the driver was never installed, or the driver is installed but Secure Boot is blocking the unsigned kernel module from loading. WSL2 users hit a different variant: the driver must be installed on the Windows host, never inside the Linux guest.

## Error Title

nvidia-smi command not found or fails to communicate with the driver

## Error Signature

bash: nvidia-smi: command not found

## Alternate Signatures

- NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running
- /dev/nvidia0: No such file or directory

## Category

installation

## Stack

- nvidia-driver
- cuda

## Affects

Fresh Linux installs, cloud GPU instances before first boot setup, WSL2 GPU environments, and any machine with Secure Boot enabled in the BIOS.

## Quick Fix

### Fastest path to a working nvidia-smi

**Steps:**

- Check whether the module even attempted to load: run dmesg | grep -i nvidia.
- Check Secure Boot state with mokutil --sb-state; if enabled, either disable it in the BIOS or enroll the MOK key generated during driver install.
- Check for a lingering nouveau driver with lsmod | grep nouveau; if present, blacklist nouveau and rebuild the initramfs.
- On WSL2, confirm you installed the GPU driver on the Windows host only, never a Linux driver inside the WSL distro.
- Reinstall or reconfigure the NVIDIA driver package for your distro, then reboot and rerun nvidia-smi.

## Symptoms

- Shell reports nvidia-smi: command not found
- nvidia-smi runs but prints a driver communication failure instead of GPU output
- /dev/nvidia0 or /dev/nvidiactl device files are missing under /dev
- dmesg shows the nvidia module failed to load or was tainted/rejected

## Causes

### The NVIDIA driver was never installed

**Likelihood:** most-common

**Explanation:** Base OS images and fresh cloud instances rarely ship a GPU driver by default. Without the driver package, there is no nvidia-smi binary and no kernel module to expose the GPU, so any CUDA-aware tool including PyTorch fails immediately at import or first GPU call.

**Fix:** Install the driver through your distro's package manager (apt install nvidia-driver-XXX on Ubuntu, or the vendor's .run installer), matching a version that supports your GPU generation, then reboot.

**Commands:**

- ubuntu-drivers devices
- sudo apt install nvidia-driver-550
- sudo reboot

### Secure Boot is blocking the unsigned kernel module from loading

**Likelihood:** common

**Explanation:** Modern Ubuntu and other distros enable UEFI Secure Boot by default, which refuses to load any kernel module that is not signed with a key enrolled in the machine's firmware. The NVIDIA driver installer generates its own signing key (MOK) during install, but if that key is never enrolled through mokutil, the module load is silently rejected.

**Fix:** Enroll the MOK key the driver installer generated by running mokutil --import and completing the enrollment on next reboot, or disable Secure Boot in the BIOS if your security policy allows it.

**Commands:**

- mokutil --sb-state
- sudo mokutil --import /var/lib/shim-signed/mok/MOK.der

### The open-source nouveau driver is still bound to the GPU

**Likelihood:** common

**Explanation:** Linux kernels ship nouveau, an open-source NVIDIA driver, enabled by default. If it loads before or alongside the proprietary driver, it claims the GPU device first and the NVIDIA module refuses to attach, so nvidia-smi cannot find a compatible device even though hardware is present.

**Fix:** Blacklist nouveau in a modprobe.d config file, regenerate the initramfs, and reboot so only the proprietary driver claims the device.

**Commands:**

- echo 'blacklist nouveau' | sudo tee /etc/modprobe.d/blacklist-nouveau.conf
- sudo update-initramfs -u
- sudo reboot

### WSL2 GPU driver installed inside the Linux guest instead of the Windows host

**Likelihood:** occasional

**Explanation:** WSL2 GPU passthrough works by exposing the host's driver into the Linux environment automatically. Installing a Linux NVIDIA driver package inside WSL conflicts with this passthrough mechanism and typically breaks nvidia-smi entirely, since WSL expects to use the Windows-side driver, not its own.

**Fix:** Uninstall any driver package installed inside the WSL distro, install only the current NVIDIA driver on the Windows host, and verify with wsl --update followed by nvidia-smi inside the distro.

### Kernel was upgraded but the DKMS module was never rebuilt for it

**Likelihood:** rare

**Explanation:** When the kernel updates via unattended-upgrades or a manual apt upgrade, DKMS is supposed to automatically rebuild the NVIDIA kernel module for the new kernel headers. If the headers package is missing or DKMS fails silently, the old module no longer matches the running kernel and fails to load.

**Fix:** Install the matching linux-headers package for the running kernel, then force a DKMS rebuild with dkms autoinstall, and reboot.

**Commands:**

- sudo apt install linux-headers-$(uname -r)
- sudo dkms autoinstall
- sudo reboot

## Diagnostics

### Check if the kernel even attempted to load the nvidia module

**Command:** dmesg | grep -i nvidia

**Interpretation:** Lines mentioning 'module verification failed' point to Secure Boot rejecting the module. Silence entirely usually means the driver package is not installed at all.

### Check Secure Boot state

**Command:** mokutil --sb-state

**Interpretation:** SecureBoot enabled combined with a missing MOK enrollment is the single most common cause on fresh Ubuntu desktops and workstations with default BIOS settings.

### Check for a conflicting nouveau driver

**Command:** lsmod | grep nouveau

**Interpretation:** Any output means nouveau is loaded and likely holding the GPU device, which must be blacklisted before the proprietary driver can attach.

### Confirm the device nodes exist after a successful driver load

**Command:** ls -la /dev/nvidia*

**Interpretation:** Missing device files after a fresh install and reboot indicate the module still failed to load; re-check dmesg rather than reinstalling the same package again.

## Prevention

- Standardize on a golden image with the driver, Secure Boot MOK enrollment, and nouveau blacklist already baked in for new GPU nodes.
- Pin kernel and driver versions together and test kernel upgrades on a canary node before fleet-wide rollout.
- Document your organization's Secure Boot policy explicitly so new hires do not waste hours rediscovering the MOK enrollment step.
- For WSL2 developer machines, keep driver installation instructions restricted to the Windows host in onboarding docs.

## When To Escalate

If this recurs across every new node in a rollout, or your organization mandates Secure Boot for compliance reasons that make unsigned module workarounds unacceptable, treat it as an imaging and provisioning problem to solve once at the fleet level rather than a per-machine troubleshooting task.

## Faq

### Why does nvidia-smi work in Windows but not inside WSL2?

**Answer:** WSL2 GPU support relies on the Windows host driver being passed through automatically; you should never install a separate Linux NVIDIA driver inside the WSL distro. If nvidia-smi fails inside WSL, first confirm the Windows host driver is current and that wsl --update has been run, rather than trying to install a driver in the Linux guest.

### Do I have to disable Secure Boot entirely to fix this?

**Answer:** No. Enrolling the MOK (Machine Owner Key) that the NVIDIA driver installer generates lets Secure Boot keep working while allowing the signed module to load. Disabling Secure Boot entirely is simpler but weakens the machine's security posture, which matters more in regulated environments.

### How do I know if nouveau is the actual problem?

**Answer:** Run lsmod | grep nouveau. Any output confirms it is loaded. Combined with dmesg showing the NVIDIA module failing to bind to the device, this is a strong signal that blacklisting nouveau and rebuilding the initramfs will resolve the issue.

### I reinstalled the driver and nvidia-smi still fails. What now?

**Answer:** Check dmesg immediately after the reboot rather than reinstalling again. Repeated reinstalls without checking the kernel log rarely help; the log will tell you whether Secure Boot, nouveau, or a DKMS build failure is the actual blocker.

## Search terms

nvidia-smi command not found, nvidia-smi has failed, nvidia driver secure boot, nouveau conflict nvidia driver, wsl2 nvidia-smi not working, nvidia kernel module not loading

---
Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json