# Why FlashAttention fails to install, and how to get it building successfully

> Source: https://netray.co/troubleshoot/flash-attention-install-failed
> Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems.
> Section: On-Prem AI Troubleshooting

## Short answer

FlashAttention's pip install compiles CUDA kernels from source unless an exact prebuilt wheel exists for your torch, CUDA, Python, and C++ ABI combination, and that compilation is extremely RAM-hungry per parallel job. The build gets silently OOM-killed on machines without enough memory unless you limit MAX_JOBS, and separately fails if your CUDA toolkit does not match the version torch itself was built against.

## Error Title

FlashAttention install fails during compilation or gets killed

## Error Signature

ninja: build stopped: subcommand failed

## Alternate Signatures

- RuntimeError: Error building extension 'flash_attn_2_cuda'
- Killed
- c++: fatal error: Killed signal terminated program cc1plus

## Category

installation

## Stack

- pytorch
- cuda

## Affects

flash-attn package installs via pip on any machine, especially those with less than 32GB of system RAM or a CUDA toolkit version that does not match the installed torch build.

## Quick Fix

### Fastest path to a successful build or install

**Steps:**

- First try to find a matching prebuilt wheel for your exact torch, CUDA, and Python version from the flash-attn GitHub releases page, which skips compilation entirely.
- If you must build from source, limit parallel compile jobs to control memory use: MAX_JOBS=4 pip install flash-attn --no-build-isolation.
- Confirm you have enough free RAM (roughly 4 to 8GB per job) with free -h before starting the build.
- Confirm nvcc --version matches the CUDA version torch was built against (python -c "import torch; print(torch.version.cuda)").
- If the build still gets killed, run it on a machine with more RAM, then copy the resulting wheel to the target machine.

## Symptoms

- pip install flash-attn runs for a long time then prints Killed with no further explanation
- ninja build error referencing a failed subcommand with no clear root cause
- Build succeeds on a workstation with lots of RAM but fails on a smaller cloud instance
- Error building extension flash_attn_2_cuda during setup.py execution

## Causes

### Compiling from source runs out of RAM because MAX_JOBS was not limited

**Likelihood:** most-common

**Explanation:** FlashAttention's setup.py compiles multiple large CUDA kernel files in parallel using all available CPU cores by default. Each parallel nvcc compile job can use several gigabytes of RAM, so on machines with limited memory (common on cloud GPU instances that are GPU-heavy but RAM-light), the OS OOM killer terminates the compiler mid-build, producing an unhelpful 'Killed' message.

**Fix:** Set the MAX_JOBS environment variable to a small number like 2 or 4 before running pip install flash-attn, which reduces peak memory use at the cost of a longer build time.

**Commands:**

- MAX_JOBS=4 pip install flash-attn --no-build-isolation

### CUDA toolkit version does not match what torch was compiled against

**Likelihood:** common

**Explanation:** FlashAttention's build system compiles against the CUDA toolkit visible on PATH via nvcc, but the resulting kernels must be binary compatible with the torch build actually running the process. A toolkit that diverges from torch's own bundled CUDA version produces subtle compile-time or load-time failures.

**Fix:** Check python -c "import torch; print(torch.version.cuda)" and install a matching CUDA toolkit version, or better, use a prebuilt flash-attn wheel that already matches your torch build.

### No prebuilt wheel exists for your exact torch, CUDA, Python, and ABI combination

**Likelihood:** common

**Explanation:** FlashAttention publishes prebuilt wheels for common combinations of torch version, CUDA version, Python version, and the C++11 ABI flag. Any deviation, such as a very recent torch release or an uncommon Python version, means pip falls back to a from-source build automatically, which is where most of these failures originate.

**Fix:** Check the flash-attn GitHub releases page for a wheel matching your exact environment before attempting to build; if none exists, consider using a slightly older torch version that does have a matching wheel.

### gcc or g++ version outside the range the CUDA toolkit supports

**Likelihood:** occasional

**Explanation:** Compiling CUDA extensions requires nvcc to invoke a host C++ compiler, and each CUDA toolkit release only supports a specific range of gcc/g++ major versions. A system gcc that is too new (common on recently released Linux distributions) causes nvcc to reject the compiler outright before any FlashAttention-specific code even compiles.

**Fix:** Install an older, supported gcc/g++ version alongside the system default and point CC and CXX environment variables at it for the duration of the build.

### Insufficient disk space in /tmp during the build

**Likelihood:** rare

**Explanation:** The compilation process generates large intermediate object files in a temporary build directory. On machines with a small root partition or a /tmp mounted on limited space, the build fails partway through with a disk-full error that can be mistaken for a memory or compiler problem.

**Fix:** Check available space with df -h /tmp, and if low, point the build's temporary directory (TMPDIR environment variable) to a partition with more free space before retrying.

## Diagnostics

### Check available system RAM before starting a build

**Command:** free -h

**Interpretation:** Less than roughly 16GB of free memory strongly suggests you should limit MAX_JOBS to 2 or fewer, or move the build to a larger machine.

### Check whether the installed CUDA toolkit matches torch's bundled version

**Command:** nvcc --version

**Interpretation:** Compare this output against python -c "import torch; print(torch.version.cuda)"; a mismatch is a common cause of both build failures and runtime crashes after a successful build.

### Confirm the compiler is within the CUDA toolkit's supported range

**Command:** gcc --version

**Interpretation:** Check this against your CUDA toolkit's documented supported host compiler versions; a gcc that is too new is a frequent, easy-to-miss cause of compile failures.

### Watch memory usage live during the build to confirm an OOM condition

**Command:** watch -n 1 free -h

**Interpretation:** If available memory drops to near zero right before the process is Killed, that confirms an OOM kill and validates lowering MAX_JOBS as the fix.

## Prevention

- Always check for a prebuilt wheel matching your exact environment before attempting a source build.
- Default to setting MAX_JOBS explicitly in your build scripts and Dockerfiles rather than relying on the compiler's automatic parallelism.
- Build FlashAttention once on a machine with ample RAM and bake the resulting wheel into your container image, rather than rebuilding on every deploy.
- Pin flash-attn, torch, and CUDA toolkit versions together in your environment lockfile so a routine upgrade does not silently break the build again.

## When To Escalate

If you need FlashAttention across a heterogeneous fleet of GPU nodes with varying RAM and CUDA toolkit versions, or in an air-gapped environment where source builds are the only option, standardize on a single build machine that produces wheels for the whole fleet rather than troubleshooting each node individually.

## Faq

### What does MAX_JOBS actually control in a FlashAttention build?

**Answer:** MAX_JOBS limits how many CUDA kernel files the compiler builds in parallel. Each parallel job can consume several gigabytes of RAM, so on memory-constrained machines, an unlimited MAX_JOBS setting causes the operating system to kill the build process partway through. Setting MAX_JOBS to 2 or 4 trades build speed for a much lower peak memory footprint.

### Why does the build just say 'Killed' with no error message?

**Answer:** 'Killed' with no further detail is the classic signature of the Linux OOM killer terminating a process that exceeded available memory. It is not a bug in FlashAttention itself; it means the compilation step used more RAM than the machine had available, and the fix is to reduce parallelism or use a machine with more memory.

### Should I always try to avoid building FlashAttention from source?

**Answer:** Yes, whenever possible. Prebuilt wheels published for common torch, CUDA, and Python combinations skip compilation entirely and avoid all of these failure modes. Only build from source when no matching wheel exists for your specific environment, and expect it to take significantly longer and require more careful RAM and compiler version management.

### Do I need the CUDA toolkit installed if I only use prebuilt wheels?

**Answer:** No. If you install a prebuilt FlashAttention wheel that matches your torch and CUDA version exactly, you do not need a separately installed CUDA toolkit or nvcc; the necessary CUDA runtime pieces are already handled by your torch installation. The toolkit is only required for compiling from source.

## Search terms

flash attention install failed, flash-attn build killed, MAX_JOBS flash attention, ninja build stopped subcommand failed, flash attn cuda compile error, flash-attn no prebuilt wheel

---
Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json