mirror of
https://github.com/mudler/LocalAI
synced 2026-05-24 09:28:23 +00:00
73aacad2f9
The pinned flash-attn 2.8.3+cu12torch2.7 wheel breaks at import time
once vllm 0.19.1 upgrades torch to its hard-pinned 2.10.0:
ImportError: .../flash_attn_2_cuda...so: undefined symbol:
_ZN3c104cuda29c10_cuda_check_implementationEiPKcS2_ib
That C10 CUDA symbol is libtorch-version-specific. Dao-AILab has not yet
published flash-attn wheels for torch 2.10 -- the latest release (2.8.3)
tops out at torch 2.8 -- so any wheel pinned here is silently ABI-broken
the moment vllm completes its install.
vllm 0.19.1 lists flashinfer-python==0.6.6 as a hard dep, which already
covers the attention path. The only other use of flash-attn in vllm is
the rotary apply_rotary import in
vllm/model_executor/layers/rotary_embedding/common.py, which is guarded
by find_spec("flash_attn") and falls back cleanly when absent.
Also unpin torch in requirements-cublas12.txt: the 2.7.0 pin only
existed to give the flash-attn wheel a matching torch to link against.
With flash-attn gone, vllm's own torch==2.10.0 dep is the binding
constraint regardless of what we put here.
Assisted-by: Claude:claude-opus-4-7 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>
|
||
|---|---|---|
| .. | ||
| backend.py | ||
| install.sh | ||
| Makefile | ||
| package.sh | ||
| README.md | ||
| requirements-after.txt | ||
| requirements-cpu-after.txt | ||
| requirements-cpu.txt | ||
| requirements-cublas12-after.txt | ||
| requirements-cublas12.txt | ||
| requirements-cublas13-after.txt | ||
| requirements-cublas13.txt | ||
| requirements-hipblas-after.txt | ||
| requirements-hipblas.txt | ||
| requirements-install.txt | ||
| requirements-intel-after.txt | ||
| requirements-intel.txt | ||
| requirements-l4t13-after.txt | ||
| requirements-l4t13.txt | ||
| requirements.txt | ||
| run.sh | ||
| test.py | ||
| test.sh | ||
Creating a separate environment for the vllm project
make vllm