fix: use absolute path for CUDA directory detection
The capability detection was using a relative path 'usr/local/cuda-13'
which doesn't work when LocalAI is run from a different working directory.
This caused whisper (and other backends) to fail on CUDA-13 containers
because the system incorrectly detected 'nvidia' capability instead of
'nvidia-cuda-13', leading to wrong backend selection (cuda12-whisper
instead of cuda13-whisper).
Fixes: https://github.com/mudler/LocalAI/issues/8033
Co-authored-by: localai-bot <localai-bot@users.noreply.github.com>
Closes#8527.
This PR fixes the excessive logging issue in capability detection by applying the existing capabilityLogged guard to the forced capability run file case.
## Changes
- Apply capabilityLogged flag to forced capability detection logging
- Prevents repeated log messages during backend discovery and gallery operations
Co-authored-by: localai-bot <localai-bot@users.noreply.github.com>
* Unload model prior to deletion
Signed-off-by: Andres Smith <andressmithdev@pm.me>
* Fix LFM model in gallery
Signed-off-by: Andres Smith <andressmithdev@pm.me>
* Remove mistakenly added files
Signed-off-by: Andres Smith <andressmithdev@pm.me>
---------
Signed-off-by: Andres Smith <andressmithdev@pm.me>
* Initial plan
* Add backend gallery filtering based on system capabilities
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
* Refactor L4T backend check to come before NVIDIA check
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
* Refactor: move capabilities business logic to capabilities.go and use constants
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
* feat: display system capability in webui and refactor tests
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
* chore: rename System/Capability
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* refactor: use getSystemCapabilities in IsBackendCompatible for consistency
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
* refactor: keep unused constants private in capabilities.go
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
* fix: skip AMD/ROCm and Intel/SYCL tests on darwin
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* chore(ci): add cuda13 jobs
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Add to pipelines and to capabilities. Start to work on the gallery
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* gallery
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* capabilities: try to detect by looking at /usr/local
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* neutts
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* backends.yaml
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* add cuda13 l4t requirements.txt
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* add cuda13 requirements.txt
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Fixups
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Fixups
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Pin vllm
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Not all backends are compatible
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* add vllm to requirements
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* vllm is not pre-compiled for cuda 13
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
- Add a system backend path
- Refactor and consolidate system information in system state
- Use system state in all the components to figure out the system paths
to used whenever needed
- Refactor BackendConfig -> ModelConfig. This was otherway misleading as
now we do have a backend configuration which is not the model config.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* chore(capability): improve messages
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* chore: isolate to constants, do not detect from the first gpu
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* migrate core/system to pkg/system - it has no dependencies FROM core, and IS USED in pkg
Signed-off-by: Dave Lee <dave@gray101.com>
* move pkg/templates up to core/templates -- nothing in pkg references it, but it does reference core.
Signed-off-by: Dave Lee <dave@gray101.com>
* remove extra check, len of nil is 0
Signed-off-by: Dave Lee <dave@gray101.com>
* move pkg/startup to core/startup -- it does have important and unfixable dependencies on core
Signed-off-by: Dave Lee <dave@gray101.com>
---------
Signed-off-by: Dave Lee <dave@gray101.com>
2025-07-24 15:03:41 +02:00
Renamed from core/system/capabilities.go (Browse further)
