# Why your GPU is not visible inside a Docker container, and how to fix it > Source: https://netray.co/troubleshoot/docker-gpu-not-available > Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems. > Section: On-Prem AI Troubleshooting ## Short answer Docker containers cannot see a host GPU unless the NVIDIA Container Toolkit is installed and the nvidia runtime is registered with the daemon, since containers are isolated from host devices by default. The fix is almost always to install nvidia-container-toolkit, run nvidia-ctk runtime configure, restart Docker, and launch with --gpus all. If nvidia-smi already fails on the host itself, the problem is the driver, not Docker. ## Error Title GPU not visible inside a Docker container ## Error Signature docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]] ## Alternate Signatures - RuntimeError: No CUDA GPUs are available - nvidia-smi: command not found (inside container) - nvidia-docker: command not found - docker: Error response from daemon: could not select device driver "nvidia.com/gpu" with capabilities: [[gpu]] ## Category containers ## Stack - docker - nvidia-driver - cuda ## Affects Docker Engine 20.10 and later on Linux hosts with an NVIDIA GPU, any CUDA-based image ## Quick Fix ### Fix it in 60 seconds **Steps:** - Confirm the host driver works: run nvidia-smi on the host, not in a container. - Install the toolkit: sudo apt-get install -y nvidia-container-toolkit. - Register the runtime: sudo nvidia-ctk runtime configure --runtime=docker. - Restart Docker: sudo systemctl restart docker. - Test it: docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi. - If that works but your app still can't see the GPU, check your app's own --gpus flag or Compose device reservation. ## Symptoms - docker run reports 'could not select device driver "" with capabilities: [[gpu]]' - nvidia-smi works on the host but not inside any container - torch.cuda.is_available() returns False only when the code runs in a container - the same image works on one host but not another ## Causes ### Using the retired nvidia-docker command, which is no longer installed **Likelihood:** common **Explanation:** The old nvidia-docker wrapper was replaced by the NVIDIA Container Toolkit, so 'nvidia-docker: command not found' on a current system is expected. Plain docker with the --gpus flag does the same job once the toolkit is installed. **Fix:** Do not install nvidia-docker. Install nvidia-container-toolkit, register the runtime, restart Docker, and replace any nvidia-docker run call with docker run --gpus all. **Commands:** - sudo apt-get install -y nvidia-container-toolkit - sudo nvidia-ctk runtime configure --runtime=docker - sudo systemctl restart docker - docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi ### Missing --gpus all flag or NVIDIA Container Toolkit not installed **Likelihood:** most-common **Explanation:** Docker does not pass through GPUs by default; without the --gpus flag on the run command and the NVIDIA Container Toolkit installed on the host, the container gets a normal Linux namespace with no device nodes or driver libraries mounted in, so any CUDA call fails immediately even though the host GPU is healthy. **Fix:** Install nvidia-container-toolkit on the host, restart Docker, then run with --gpus all (or --gpus device=0 for a specific GPU). Confirm with a plain nvidia-smi test container. **Commands:** - sudo apt-get install -y nvidia-container-toolkit - sudo nvidia-ctk runtime configure --runtime=docker - sudo systemctl restart docker - docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi ### NVIDIA runtime not registered in daemon.json **Likelihood:** common **Explanation:** The toolkit can be installed but Docker's daemon.json never got updated to register the nvidia runtime, or a manual edit was reverted by a package update, so docker run --gpus all silently falls back to the default runc runtime which has no GPU support. **Fix:** Run nvidia-ctk runtime configure to regenerate /etc/docker/daemon.json with the nvidia runtime entry, restart the daemon, and verify with docker info. **Commands:** - cat /etc/docker/daemon.json - sudo nvidia-ctk runtime configure --runtime=docker --set-as-default - sudo systemctl restart docker - docker info | grep -i runtime ### Driver and container toolkit version mismatch **Likelihood:** occasional **Explanation:** The host NVIDIA driver and the CUDA version baked into the container image must be compatible; an old driver paired with a newer CUDA base image reports the GPU as present but every CUDA API call fails with a driver library version mismatch, which looks identical to no GPU from inside the container. **Fix:** Check the driver version on the host against the CUDA version in the image tag, and either upgrade the host driver or pull an older CUDA base image that matches it. **Commands:** - nvidia-smi --query-gpu=driver_version --format=csv - docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi ### Docker rootless mode without the runtime configured for the user namespace **Likelihood:** rare **Explanation:** Rootless Docker runs the daemon in a separate user namespace, and the default NVIDIA Container Toolkit setup only registers the runtime for the system daemon, so a rootless install needs its own CDI configuration or GPU access never gets negotiated for that namespace. **Fix:** Follow the rootless-specific setup (CDI mode is simplest) and regenerate the CDI spec with nvidia-ctk cdi generate, then reference it explicitly in the run command. **Commands:** - nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml - docker run --rm --device nvidia.com/gpu=all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi ## Diagnostics ### Confirm the host sees the GPU **Command:** nvidia-smi **Interpretation:** If this fails on the host itself, the problem is the driver, not Docker; fix the host driver before touching containers. ### Test the toolkit directly **Command:** docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi **Interpretation:** If this fails with the device driver error, the toolkit or daemon.json is the problem; if it succeeds but your app still can't see the GPU, the issue is inside your image or entrypoint. ### Check which runtime Docker is using **Command:** docker info | grep -i runtime **Interpretation:** You should see nvidia listed among the runtimes; if it is absent, nvidia-ctk runtime configure was never run or was reverted. ### Check installed toolkit version **Command:** dpkg -l | grep nvidia-container **Interpretation:** Confirms whether nvidia-container-toolkit is actually installed versus only the driver; a missing package means the whole toolkit needs installing, not just configuring. ## Prevention - Pin the NVIDIA Container Toolkit version in your base image build docs alongside the host driver version - Bake a nvidia-smi smoke test into your CI image build so a broken runtime fails the build, not production - Document the exact docker run or Compose gpus stanza in your deployment runbook so it is never typed from memory - Standardize on one CUDA base image tag across the fleet to avoid per-host driver mismatches ## When To Escalate If every host in the fleet needs the same toolkit fix and you are about to repeat these steps across dozens of nodes, or you are moving from single Docker hosts to Kubernetes with a GPU Operator, this becomes a platform decision worth designing once rather than debugging per node. ## Faq ### Do I need the NVIDIA Container Toolkit if I already have the NVIDIA driver installed? **Answer:** Yes. The host driver lets the host itself use the GPU, but Docker containers are isolated by default and have no way to see host devices or driver libraries. The NVIDIA Container Toolkit is a separate component that registers a Docker runtime capable of mounting the GPU device nodes and driver libraries into a container at launch time. Installing only the driver leaves containers with no GPU access at all. ### Why does nvidia-smi work on the host but not inside my container? **Answer:** This is the classic signature of a missing or misconfigured NVIDIA Container Toolkit. The host driver is healthy, but Docker has no runtime configured to pass GPU devices into containers, so every container gets a plain Linux namespace with none of the device files or shared libraries the driver needs. Installing the toolkit and adding --gpus all to the run command resolves it in most cases. ### What is the difference between --gpus all and --runtime=nvidia? **Answer:** --gpus all is the modern Docker flag that requests GPU devices for a specific container and works with recent Docker Engine versions once the toolkit is installed. --runtime=nvidia is the older mechanism that forces the entire container to use the nvidia OCI runtime. Current NVIDIA documentation recommends --gpus for new deployments; --runtime is mainly seen in legacy Compose files. ### Does Docker Compose need special configuration for GPU access? **Answer:** Yes. Compose does not read --gpus from the CLI, so you need a deploy.resources.reservations.devices block in the service definition specifying the nvidia driver and GPU count or all. Without this block, a Compose-launched container behaves exactly like one started without --gpus, and CUDA calls fail even though a plain docker run with --gpus all works fine. ## Search terms docker gpu not available, could not select device driver with capabilities gpu, docker nvidia-smi not found in container, nvidia container toolkit gpus all, docker cuda not available, no cuda gpus are available docker, pytorch gpu not detected in container --- Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json