# Why llama.cpp fails to build, and how to fix common CMake and CUDA errors > Source: https://netray.co/troubleshoot/llama-cpp-build-errors > Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems. > Section: On-Prem AI Troubleshooting ## Short answer Most llama.cpp build failures come from missing or mismatched CUDA toolkit installs, wrong cmake backend flags (forgetting to enable GGML_CUDA or GGML_METAL), or a host compiler version newer than the CUDA toolkit supports. Confirm the toolkit is installed and on PATH, pass the correct backend flag for your hardware, and match your compiler version to what your CUDA version officially supports. ## Error Title llama.cpp fails to build from source (CMake, CUDA, Metal) ## Error Signature CMake Error: CUDA_TOOLKIT_ROOT_DIR not found ## Alternate Signatures - nvcc fatal : Unsupported gpu architecture - error: 'CUDA_R_16F' was not declared in this scope - make: *** [Makefile] Error 1 ## Category local-runtime ## Stack - llama-cpp - cuda - nvidia-driver ## Affects llama.cpp built from source on Linux, macOS, and Windows, with CUDA, Metal, or CPU-only backends ## Quick Fix ### Fastest path to a successful build **Steps:** - Confirm the CUDA toolkit is installed and visible: nvcc --version should print a version, not a command-not-found error. - Clean any previous build directory to avoid stale cmake cache conflicts: rm -rf build. - Reconfigure with the correct backend flag for your hardware, for example cmake -B build -DGGML_CUDA=ON for NVIDIA GPUs or -DGGML_METAL=ON on Apple Silicon. - Build with cmake --build build --config Release -j. - If the compiler is rejected, install or switch to a gcc/clang version that your CUDA toolkit's release notes list as supported. ## Symptoms - cmake configuration fails before compilation even starts, citing a missing CUDA toolkit path - nvcc reports an unsupported GPU architecture during compilation - The build succeeds but the resulting binary runs on CPU only, with no GPU acceleration - A compiler error references a CUDA header symbol that seems to not exist ## Causes ### CUDA toolkit not installed or not on PATH **Likelihood:** most-common **Explanation:** Having an NVIDIA driver installed, which nvidia-smi confirms, is not the same as having the CUDA toolkit and its compiler nvcc installed; cmake's CUDA-enabled configuration step specifically needs nvcc and the toolkit's headers and libraries, and fails immediately if it cannot locate them, which is a different and more basic requirement than just having a working GPU driver. **Fix:** Install the CUDA toolkit matching your driver's supported CUDA version, and ensure nvcc is on PATH before running cmake. **Commands:** - nvcc --version - export PATH=/usr/local/cuda/bin:$PATH ### Missing or wrong backend flag passed to cmake **Likelihood:** common **Explanation:** llama.cpp supports multiple compute backends (CUDA, Metal, Vulkan, CPU-only), and each requires an explicit cmake flag to enable; building without the flag for your hardware silently produces a CPU-only binary or, in stricter configurations, a configuration error because required libraries were never searched for. **Fix:** Pass the correct backend flag explicitly: GGML_CUDA=ON for NVIDIA GPUs, GGML_METAL=ON for Apple Silicon, and confirm after building that the binary reports GPU offload capability at runtime. **Commands:** - cmake -B build -DGGML_CUDA=ON - cmake -B build -DGGML_METAL=ON ### GPU compute capability not included in the CMAKE_CUDA_ARCHITECTURES list **Likelihood:** common **Explanation:** CUDA code is compiled for specific compute capability targets, and if your GPU's architecture is not in the list cmake was configured with, either because it defaulted to an older list or you set it manually and missed one, nvcc reports an unsupported architecture error for that GPU at build or runtime. **Fix:** Set CMAKE_CUDA_ARCHITECTURES to match your GPU's actual compute capability, which you can look up from nvidia-smi's compute_cap query or NVIDIA's published list for your GPU model. **Commands:** - nvidia-smi --query-gpu=compute_cap --format=csv - cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=89 ### Host compiler version incompatible with the installed CUDA toolkit **Likelihood:** occasional **Explanation:** Each CUDA toolkit release officially supports a specific range of gcc, g++, or MSVC versions as the host compiler; a system with a newer default compiler than that range, common after an OS upgrade, causes nvcc to reject headers or produce cryptic template and declaration errors during compilation. **Fix:** Install an older, supported gcc/g++ version alongside the system default and point CUDA at it explicitly via the CMAKE_CUDA_HOST_COMPILER cmake variable or update-alternatives, rather than fighting the default compiler. **Commands:** - sudo update-alternatives --config gcc - cmake -B build -DCMAKE_CUDA_HOST_COMPILER=/usr/bin/gcc-11 ### Stale cmake cache from a previous configuration **Likelihood:** rare **Explanation:** Reconfiguring cmake with new flags in an existing build directory sometimes leaves stale cached variables from the previous run, which can produce confusing errors that look like a fresh misconfiguration but are actually leftover state from before you fixed the toolkit or flags. **Fix:** Delete the build directory entirely and reconfigure from scratch after any change to toolchain, compiler, or major cmake flags. **Commands:** - rm -rf build - cmake -B build -DGGML_CUDA=ON ## Diagnostics ### Confirm the CUDA compiler is available **Command:** nvcc --version **Interpretation:** Command not found means the toolkit is missing or not on PATH, independent of whether the driver works; install the toolkit before touching cmake flags. ### Check your GPU's compute capability **Command:** nvidia-smi --query-gpu=compute_cap --format=csv **Interpretation:** Use this value to set CMAKE_CUDA_ARCHITECTURES explicitly if the default architecture list does not include your card. ### Check the installed compiler version **Command:** gcc --version **Interpretation:** Compare this against your CUDA toolkit's documented supported host compiler range; a mismatch is a common source of obscure template and header errors. ### Confirm the built binary actually reports GPU support **Command:** ./build/bin/llama-cli --version **Interpretation:** Modern llama.cpp builds report which backends were compiled in; if CUDA or Metal is absent from that output, the build succeeded but without the acceleration you intended. ## Prevention - Pin a specific CUDA toolkit and compiler version pair known to work together, and document it in your build scripts rather than relying on whatever the OS currently defaults to. - Always do a clean build (rm -rf build) after changing backend flags, toolkit versions, or compiler versions. - Keep a short reference in your deployment repo that reproduces the exact cmake invocation for each target platform (CUDA Linux, Metal macOS, CPU-only). - Prefer prebuilt Ollama binaries or official Docker images when you do not specifically need a custom llama.cpp build, since they ship pre-validated toolchains. ## When To Escalate If you are maintaining custom llama.cpp builds across many different GPU generations and driver versions, standardizing on a small number of validated Docker build images, rather than building from source on every target machine, is worth the upfront investment and removes an entire class of toolchain drift problems. ## Faq ### Do I need the full CUDA toolkit, or is the NVIDIA driver enough to build llama.cpp with GPU support? **Answer:** You need the full CUDA toolkit, which includes nvcc and CUDA headers and libraries, in addition to the driver. The driver alone lets a prebuilt binary run on the GPU; building CUDA code from source requires the toolkit's compiler. ### Why does my llama.cpp build succeed but still run on CPU only? **Answer:** Most likely the backend flag (GGML_CUDA=ON or GGML_METAL=ON) was not passed during cmake configuration, so the default CPU-only backend was built instead. Reconfigure explicitly with the correct flag and rebuild from a clean directory. ### Can I use a newer compiler than my CUDA toolkit officially supports? **Answer:** It sometimes works but is unsupported and can produce confusing compilation errors that are hard to diagnose. If you hit strange template or header errors during a CUDA build, installing an older, officially supported compiler version alongside the system default is the more reliable fix. ## Search terms llama.cpp build error, cmake cuda toolkit not found, nvcc unsupported gpu architecture, GGML_CUDA cmake, llama.cpp metal build, llama.cpp compile error --- Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json