Elgato_dark/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI synced 2026-05-24 09:28:23 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 8
6355 commits 101 branches 150 tags 134 MiB
Commit graph

530 commits

Author SHA1 Message Date
dependabot[bot]
5881c82413
chore(deps): bump torch from 2.4.1 to 2.7.1+xpu in /backend/python/bark in the pip group across 1 directory (#8175) 
chore(deps): bump torch

Bumps the pip group with 1 update in the /backend/python/bark directory: torch.


Updates `torch` from 2.4.1 to 2.7.1+xpu

---
updated-dependencies:
- dependency-name: torch
  dependency-version: 2.7.1+xpu
  dependency-type: direct:production
  dependency-group: pip
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-01-23 15:32:15 +00:00
Ettore Di Giacinto
923ebbb344
feat(qwen-tts): add Qwen-tts backend (#8163) 
* feat(qwen-tts): add Qwen-tts backend

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Update intel deps

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Drop flash-attn for cuda13

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2026-01-23 15:18:41 +01:00
Ettore Di Giacinto
0fa0ac4797
fix(videogen): drop incomplete endpoint, add GGUF support for LTX-2 (#8160) 
* Debug

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Drop openai video endpoint (is not complete)

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add download button

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2026-01-22 14:09:20 +01:00
Ettore Di Giacinto
22c0eb5421
chore(diffusers): add 'av' to requirements.txt (#8155) 
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2026-01-21 22:35:00 +01:00
Ettore Di Giacinto
d16722ee13
Revert "chore(deps): bump torch from 2.3.1+cxx11.abi to 2.8.0 in /backend/python/rerankers in the pip group across 1 directory" (#8072) 
Revert "chore(deps): bump torch from 2.3.1+cxx11.abi to 2.8.0 in /backend/pyt…"

This reverts commit 1f10ab39a9.
 2026-01-16 20:50:33 +01:00
dependabot[bot]
1f10ab39a9
chore(deps): bump torch from 2.3.1+cxx11.abi to 2.8.0 in /backend/python/rerankers in the pip group across 1 directory (#8066) 
chore(deps): bump torch

Bumps the pip group with 1 update in the /backend/python/rerankers directory: [torch](https://github.com/pytorch/pytorch).


Updates `torch` from 2.3.1+cxx11.abi to 2.8.0
- [Release notes](https://github.com/pytorch/pytorch/releases)
- [Changelog](https://github.com/pytorch/pytorch/blob/main/RELEASE.md)
- [Commits](https://github.com/pytorch/pytorch/commits/v2.8.0)

---
updated-dependencies:
- dependency-name: torch
  dependency-version: 2.8.0
  dependency-type: direct:production
  dependency-group: pip
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-01-16 19:38:12 +00:00
Ettore Di Giacinto
b19afc9e64
feat(diffusers): add support to LTX-2 (#8019) 
* feat(diffusers): add support to LTX-2

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add to the gallery

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2026-01-14 09:07:30 +01:00
Ettore Di Giacinto
a6ff354c86
feat(tts): add pocket-tts backend (#8018) 
* feat(pocket-tts): add new backend

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add to the gallery

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fixups

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Update docs

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2026-01-13 23:35:19 +01:00
dependabot[bot]
94eecc43a3
chore(deps): bump protobuf from 6.33.2 to 6.33.4 in /backend/python/transformers (#7993) 
chore(deps): bump protobuf in /backend/python/transformers

Bumps [protobuf](https://github.com/protocolbuffers/protobuf) from 6.33.2 to 6.33.4.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Commits](https://github.com/protocolbuffers/protobuf/commits)

---
updated-dependencies:
- dependency-name: protobuf
  dependency-version: 6.33.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-01-12 23:46:32 +00:00
Ettore Di Giacinto
2de30440fe
fix(l4t-12): use pip to install python deps (#7967) 
* fix: install only torch/torchvision from jetson index

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix: use pip for l4t-12

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Revert "fix: install only torch/torchvision from jetson index"

This reverts commit 2d2b020078

* chatterbox needs wheel

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2026-01-11 00:21:32 +01:00
Ettore Di Giacinto
a4d224dd1b
Revert "chore(uv): add --index-strategy=unsafe-first-match to l4t" (#7936) 
Revert "chore(uv): add --index-strategy=unsafe-first-match to l4t (#7934)"

This reverts commit f5dee90962.
 2026-01-08 23:31:51 +01:00
Ettore Di Giacinto
f5dee90962
chore(uv): add --index-strategy=unsafe-first-match to l4t (#7934) 
This is because the main index might not contain all the dependencies
for torch

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2026-01-08 22:48:03 +01:00
Ettore Di Giacinto
383312b50e
chore(l4t-12): do not use python 3.12 (wheels are only for 3.10) (#7928) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2026-01-08 19:00:07 +01:00
Ettore Di Giacinto
b964b3d53e
feat(backends): add moonshine backend for faster transcription (#7833) 
* feat(backends): add moonshine backend for faster transcription

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add backend to CI, update AGENTS.md from this exercise

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2026-01-07 21:44:35 +01:00
Copilot
fd53978a7b
feat: package GPU libraries inside backend containers for unified base image (#7891) 
* Initial plan

* Add GPU library packaging for isolated backend environments

- Create scripts/build/package-gpu-libs.sh for packaging CUDA, ROCm, SYCL, and Vulkan libraries
- Update llama-cpp, whisper, stablediffusion-ggml package.sh to include GPU libraries
- Update Dockerfile.python to package GPU libraries into Python backends
- Update libbackend.sh to set LD_LIBRARY_PATH for GPU library loading

Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>

* Address code review feedback: fix variable consistency and quoting

Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>

* Fix code review issues: improve glob handling and remove redundant variable

Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>

* Simplify main Dockerfile and workflow to use unified base image

- Remove GPU-specific driver installation from Dockerfile (CUDA, ROCm, Vulkan, Intel)
- Simplify image.yml workflow to build single unified base image for linux/amd64 and linux/arm64
- GPU libraries are now packaged in individual backend containers

Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2026-01-07 15:48:51 +01:00
Richard Palethorpe
e6ba26c3e7
chore: Update to Ubuntu24.04 (cont #7423) (#7769) 
* ci(workflows): bump GitHub Actions images to Ubuntu 24.04

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* ci(workflows): remove CUDA 11.x support from GitHub Actions (incompatible with ubuntu:24.04)

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* ci(workflows): bump GitHub Actions CUDA support to 12.9

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* build(docker): bump base image to ubuntu:24.04 and adjust Vulkan SDK/packages

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* fix(backend): correct context paths for Python backends in workflows, Makefile and Dockerfile

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* chore(make): disable parallel backend builds to avoid race conditions

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* chore(make): export CUDA_MAJOR_VERSION and CUDA_MINOR_VERSION for override

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* build(backend): update backend Dockerfiles to Ubuntu 24.04

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* chore(backend): add ROCm env vars and default AMDGPU_TARGETS for hipBLAS builds

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* chore(chatterbox): bump ROCm PyTorch to 2.9.1+rocm6.4 and update index URL; align hipblas requirements

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* chore: add local-ai-launcher to .gitignore

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* ci(workflows): fix backends GitHub Actions workflows after rebase

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* build(docker): use build-time UBUNTU_VERSION variable

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* chore(docker): remove libquadmath0 from requirements-stage base image

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* chore(make): add backends/vllm to .NOTPARALLEL to prevent parallel builds

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* fix(docker): correct CUDA installation steps in backend Dockerfiles

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* chore(backend): update ROCm to 6.4 and align Python hipblas requirements

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* ci(workflows): switch GitHub Actions runners to Ubuntu-24.04 for CUDA on arm64 builds

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* build(docker): update base image and backend Dockerfiles for Ubuntu 24.04 compatibility on arm64

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* build(backend): increase timeout for uv installs behind slow networks on backend/Dockerfile.python

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* ci(workflows): switch GitHub Actions runners to Ubuntu-24.04 for vibevoice backend

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* ci(workflows): fix failing GitHub Actions runners

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>

* fix: Allow FROM_SOURCE to be unset, use upstream Intel images etc.

Signed-off-by: Richard Palethorpe <io@richiejp.com>

* chore(build): rm all traces of CUDA 11

Signed-off-by: Richard Palethorpe <io@richiejp.com>

* chore(build): Add Ubuntu codename as an argument

Signed-off-by: Richard Palethorpe <io@richiejp.com>

---------

Signed-off-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>
Signed-off-by: Richard Palethorpe <io@richiejp.com>
Co-authored-by: Alessandro Sturniolo <alessandro.sturniolo@gmail.com>
 2026-01-06 15:26:42 +01:00
blightbow
67baf66555
feat(mlx): add thread-safe LRU prompt cache and min_p/top_k sampling (#7556) 
* feat(mlx): add thread-safe LRU prompt cache

Port mlx-lm's LRUPromptCache to fix race condition where concurrent
requests corrupt shared KV cache state. The previous implementation
used a single prompt_cache instance shared across all requests.

Changes:
- Add backend/python/common/mlx_cache.py with ThreadSafeLRUPromptCache
- Modify backend.py to use per-request cache isolation via fetch/insert
- Add prefix matching for cache reuse across similar prompts
- Add LRU eviction (default 10 entries, configurable)
- Add concurrency and cache unit tests

The cache uses a trie-based structure for efficient prefix matching,
allowing prompts that share common prefixes to reuse cached KV states.
Thread safety is provided via threading.Lock.

New configuration options:
- max_cache_entries: Maximum LRU cache entries (default: 10)
- max_kv_size: Maximum KV cache size per entry (default: None)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Blightbow <blightbow@users.noreply.github.com>

* feat(mlx): add min_p and top_k sampler support

Add MinP field to proto (field 52) following the precedent set by
other non-OpenAI sampling parameters like TopK, TailFreeSamplingZ,
TypicalP, and Mirostat.

Changes:
- backend.proto: Add float MinP field for min-p sampling
- backend.py: Extract and pass min_p and top_k to mlx_lm sampler
  (top_k was in proto but not being passed)
- test.py: Fix test_sampling_params to use valid proto fields and
  switch to MLX-compatible model (mlx-community/Llama-3.2-1B-Instruct)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Blightbow <blightbow@users.noreply.github.com>

* refactor(mlx): move mlx_cache.py from common to mlx backend

The ThreadSafeLRUPromptCache is only used by the mlx backend. After
evaluating mlx-vlm, it was determined that the cache cannot be shared
because mlx-vlm's generate/stream_generate functions don't support
the prompt_cache parameter that mlx_lm provides.

- Move mlx_cache.py from backend/python/common/ to backend/python/mlx/
- Remove sys.path manipulation from backend.py and test.py
- Fix test assertion to expect "MLX model loaded successfully"

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Blightbow <blightbow@users.noreply.github.com>

* test(mlx): add comprehensive cache tests and document upstream behavior

Added comprehensive unit tests (test_mlx_cache.py) covering all cache
operation modes:
- Exact match
- Shorter prefix match
- Longer prefix match with trimming
- No match scenarios
- LRU eviction and access order
- Reference counting and deep copy behavior
- Multi-model namespacing
- Thread safety with data integrity verification

Documents upstream mlx_lm/server.py behavior: single-token prefixes are
deliberately not matched (uses > 0, not >= 0) to allow longer cached
sequences to be preferred for trimming. This is acceptable because real
prompts with chat templates are always many tokens.

Removed weak unit tests from test.py that only verified "no exception
thrown" rather than correctness.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Blightbow <blightbow@users.noreply.github.com>

* chore(mlx): remove unused MinP proto field

The MinP field was added to PredictOptions but is not populated by the
Go frontend/API. The MLX backend uses getattr with a default value,
so it works without the proto field.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Blightbow <blightbow@users.noreply.github.com>

---------

Signed-off-by: Blightbow <blightbow@users.noreply.github.com>
Co-authored-by: Blightbow <blightbow@users.noreply.github.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2025-12-16 11:27:46 +01:00
dependabot[bot]
dbd25885c3
chore(deps): bump sentence-transformers from 5.1.0 to 5.2.0 in /backend/python/transformers (#7594) 
chore(deps): bump sentence-transformers in /backend/python/transformers

Bumps [sentence-transformers](https://github.com/huggingface/sentence-transformers) from 5.1.0 to 5.2.0.
- [Release notes](https://github.com/huggingface/sentence-transformers/releases)
- [Commits](https://github.com/huggingface/sentence-transformers/compare/v5.1.0...v5.2.0)

---
updated-dependencies:
- dependency-name: sentence-transformers
  dependency-version: 5.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-12-16 09:12:57 +01:00
Ettore Di Giacinto
7790a24682
Revert "chore(deps): bump torch from 2.5.1+cxx11.abi to 2.7.1+cpu in /backend/python/diffusers in the pip group across 1 directory" (#7558) 
Revert "chore(deps): bump torch from 2.5.1+cxx11.abi to 2.7.1+cpu in /backend…"

This reverts commit 1b4aa6f1be.
 2025-12-13 17:04:46 +01:00
dependabot[bot]
1b4aa6f1be
chore(deps): bump torch from 2.5.1+cxx11.abi to 2.7.1+cpu in /backend/python/diffusers in the pip group across 1 directory (#7549) 
chore(deps): bump torch

Bumps the pip group with 1 update in the /backend/python/diffusers directory: torch.


Updates `torch` from 2.5.1+cxx11.abi to 2.7.1+cpu

---
updated-dependencies:
- dependency-name: torch
  dependency-version: 2.7.1+cpu
  dependency-type: direct:production
  dependency-group: pip
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-12-13 13:12:18 +00:00
Ettore Di Giacinto
504d954aea
Add chardet to requirements-l4t13.txt 
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2025-12-13 12:59:03 +01:00
Ettore Di Giacinto
6d2a535813
chore(l4t13): use pytorch index (#7546) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-12-13 10:04:57 +01:00
Ettore Di Giacinto
32dcb58e89
feat(vibevoice): add new backend (#7494) 
* feat(vibevoice): add backend

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore: add workflow and backend index

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore(gallery): add vibevoice

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Use self-hosted for intel builds

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Pin python version for l4t

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-12-10 21:14:21 +01:00
dependabot[bot]
bbce461f57
chore(deps): bump protobuf from 6.33.1 to 6.33.2 in /backend/python/transformers (#7481) 
chore(deps): bump protobuf in /backend/python/transformers

Bumps [protobuf](https://github.com/protocolbuffers/protobuf) from 6.33.1 to 6.33.2.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Commits](https://github.com/protocolbuffers/protobuf/commits)

---
updated-dependencies:
- dependency-name: protobuf
  dependency-version: 6.33.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-12-08 22:13:18 +01:00
Copilot
1abbedd732
feat(diffusers): implement dynamic pipeline loader to remove per-pipeline conditionals (#7365) 
* Initial plan

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add dynamic loader for diffusers pipelines and refactor backend.py

Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Fix pipeline discovery error handling and test mock issue

Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Address code review feedback: direct imports, better error handling, improved tests

Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Address remaining code review feedback: specific exceptions, registry access, test imports

Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add defensive fallback for DiffusionPipeline registry access

Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Actually use dynamic pipeline loading for all pipelines in backend

Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Use dynamic loader consistently for all pipelines including AutoPipelineForText2Image

Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Move dynamic loader tests into test.py for CI compatibility

Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Extend dynamic loader to discover any diffusers class type, not just DiffusionPipeline

Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add AutoPipeline classes to pipeline registry for default model loading

Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(python): set pyvenv python home

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* do pyenv update during start

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Minor changes

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2025-12-04 19:02:06 +01:00
Ettore Di Giacinto
cfd95745ed
feat: add cuda13 images (#7404) 
* chore(ci): add cuda13 jobs

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add to pipelines and to capabilities. Start to work on the gallery

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* gallery

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* capabilities: try to detect by looking at /usr/local

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* neutts

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backends.yaml

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* add cuda13 l4t requirements.txt

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* add cuda13 requirements.txt

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Fixups

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Fixups

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Pin vllm

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Not all backends are compatible

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* add vllm to requirements

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* vllm is not pre-compiled for cuda 13

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-12-02 14:24:35 +01:00
Ettore Di Giacinto
4b5977f535
chore: drop pinning of python 3.12 (#7389) 
Update install.sh

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2025-11-28 11:02:56 +01:00
Ettore Di Giacinto
0d877b1e71
Revert "chore(l4t): Update extra index URL for requirements-l4t.txt" (#7388) 
Revert "chore(l4t): Update extra index URL for requirements-l4t.txt (#7383)"

This reverts commit 0d781e6b7e.
 2025-11-28 11:02:11 +01:00
Ettore Di Giacinto
e27f1370eb
chore(diffusers): Add PY_STANDALONE_TAG for l4t Python version (#7387) 
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2025-11-28 09:34:05 +01:00
Ettore Di Giacinto
e01d821314
chore: Add Python 3.12 support for l4t build profile (#7384) 
Set Python version to 3.12 for l4t build profile.

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2025-11-27 23:00:09 +01:00
Ettore Di Giacinto
0d781e6b7e
chore(l4t): Update extra index URL for requirements-l4t.txt (#7383) 
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2025-11-27 22:02:06 +01:00
Ettore Di Giacinto
7ccc383a8b
chore(l4t/diffusers): bump nvidia l4t index for pytorch 2.9 (#7379) 
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2025-11-27 17:42:01 +01:00
Ettore Di Giacinto
2f8a2b1297
chore(deps): update diffusers dependency to use GitHub repo for l4t (#7369) 
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2025-11-27 16:02:48 +01:00
dependabot[bot]
7e01aa8faa
chore(deps): bump protobuf from 6.32.0 to 6.33.1 in /backend/python/transformers (#7340) 
chore(deps): bump protobuf in /backend/python/transformers

Bumps [protobuf](https://github.com/protocolbuffers/protobuf) from 6.32.0 to 6.33.1.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/protobuf_release.bzl)
- [Commits](https://github.com/protocolbuffers/protobuf/commits)

---
updated-dependencies:
- dependency-name: protobuf
  dependency-version: 6.33.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-11-24 20:12:17 +00:00
Ettore Di Giacinto
3a232446e0
Revert "chore(chatterbox): bump l4t index to support more recent pytorch" (#7333) 
Revert "chore(chatterbox): bump l4t index to support more recent pytorch (#7332)"

This reverts commit 55607a5aac.
 2025-11-22 10:10:27 +01:00
Ettore Di Giacinto
55607a5aac
chore(chatterbox): bump l4t index to support more recent pytorch (#7332) 
This should add support for devices like the DGX Spark

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2025-11-21 22:24:46 +01:00
Ettore Di Giacinto
ec492a4c56
fix(typo): environment variable name for max jobs 
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2025-11-21 18:37:22 +01:00
Ettore Di Giacinto
2defe98df8
fix(vllm): Update flash-attn to specific wheel URL 
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2025-11-21 18:06:46 +01:00
Ettore Di Giacinto
6261c87b1b
Add NVCC_THREADS and MAX_JOB environment variables 
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2025-11-21 16:14:13 +01:00
Ettore Di Giacinto
daf39e1efd
chore(vllm/ci): set maximum number of jobs 
Also added comments to clarify CPU usage during build.

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2025-11-20 15:53:32 +01:00
Mikhail Khludnev
01cd58a739
fix(reranker): support omitting top_n (#7199) 
* fix(reranker): support omitting top_n

Signed-off-by: Mikhail Khludnev <mkhl@apache.org>

* fix(reranker): support omitting top_n

Signed-off-by: Mikhail Khludnev <mkhl@apache.org>

* pass 0 explicitly 

Signed-off-by: Mikhail Khludnev <mkhludnev@users.noreply.github.com>

---------

Signed-off-by: Mikhail Khludnev <mkhl@apache.org>
Signed-off-by: Mikhail Khludnev <mkhludnev@users.noreply.github.com>
 2025-11-09 18:40:32 +01:00
Ettore Di Giacinto
2f2f9beee7
fix(chatterbox): pin numpy (#7198) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-11-08 16:52:22 +01:00
Mikhail Khludnev
122e4c7094
fix(reranker): reproduce ignoring top_n (#7025) 
* fix(reranker): reproduce ignoring top_n

Signed-off-by: Mikhail Khludnev <mkhl@apache.org>

* fix(reranker): ignoring top_n

Signed-off-by: Mikhail Khludnev <mkhl@apache.org>

---------

Signed-off-by: Mikhail Khludnev <mkhl@apache.org>
 2025-11-06 10:03:05 +00:00
Lukas Schaefer
d95d4992fe
feat: return complete audio for kokoro (#6842) 
Signed-off-by: Lukas Schaefer <lukas@lschaefer.xyz>
 2025-10-28 08:49:18 +01:00
dependabot[bot]
63e6721c2f
chore(deps): bump grpcio from 1.75.1 to 1.76.0 in /backend/python/diffusers (#6839) 
chore(deps): bump grpcio in /backend/python/diffusers

Bumps [grpcio](https://github.com/grpc/grpc) from 1.75.1 to 1.76.0.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.75.1...v1.76.0)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.76.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-10-27 21:45:42 +01:00
dependabot[bot]
be027b1ccd
chore(deps): bump grpcio from 1.75.1 to 1.76.0 in /backend/python/transformers (#6828) 
chore(deps): bump grpcio in /backend/python/transformers

Bumps [grpcio](https://github.com/grpc/grpc) from 1.75.1 to 1.76.0.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.75.1...v1.76.0)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.76.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-10-27 21:32:31 +01:00
dependabot[bot]
3ecadeeb93
chore(deps): bump grpcio from 1.75.1 to 1.76.0 in /backend/python/exllama2 (#6836) 
chore(deps): bump grpcio in /backend/python/exllama2

Bumps [grpcio](https://github.com/grpc/grpc) from 1.75.1 to 1.76.0.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.75.1...v1.76.0)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.76.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-10-27 21:32:13 +01:00
dependabot[bot]
4af3348f91
chore(deps): bump grpcio from 1.75.1 to 1.76.0 in /backend/python/vllm (#6827) 
Bumps [grpcio](https://github.com/grpc/grpc) from 1.75.1 to 1.76.0.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.75.1...v1.76.0)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.76.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-10-27 21:31:47 +01:00
dependabot[bot]
dde08845bf
chore(deps): bump grpcio from 1.75.1 to 1.76.0 in /backend/python/bark (#6826) 
Bumps [grpcio](https://github.com/grpc/grpc) from 1.75.1 to 1.76.0.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.75.1...v1.76.0)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.76.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-10-27 21:31:24 +01:00
dependabot[bot]
76d1ba168d
chore(deps): bump grpcio from 1.75.1 to 1.76.0 in /backend/python/common/template (#6830) 
chore(deps): bump grpcio in /backend/python/common/template

Bumps [grpcio](https://github.com/grpc/grpc) from 1.75.1 to 1.76.0.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.75.1...v1.76.0)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.76.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-10-27 21:31:03 +01:00
dependabot[bot]
80605e4f66
chore(deps): bump grpcio from 1.75.1 to 1.76.0 in /backend/python/rerankers (#6819) 
chore(deps): bump grpcio in /backend/python/rerankers

Bumps [grpcio](https://github.com/grpc/grpc) from 1.75.1 to 1.76.0.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.75.1...v1.76.0)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.76.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-10-27 21:30:46 +01:00
dependabot[bot]
53f44dac89
chore(deps): bump grpcio from 1.75.1 to 1.76.0 in /backend/python/coqui (#6822) 
Bumps [grpcio](https://github.com/grpc/grpc) from 1.75.1 to 1.76.0.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.75.1...v1.76.0)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.76.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-10-27 21:27:05 +01:00
Ettore Di Giacinto
33c48164d7
Change sleep duration in tests from 10 to 20 seconds 
Increased sleep duration in tests to 20 seconds.

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2025-10-17 14:56:52 +02:00
Ettore Di Giacinto
cb0ed55d89
feat(neutts): add backend (#6404) 
* feat(neutts): add backend

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore(ci): add images to CI

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore(gallery): add Neutts

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Make it work with quantized versions

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Fixups

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Docs

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Fixups

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Apply suggestion from @mudler

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>

* Apply suggestion from @mudler

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>

* Apply suggestion from @mudler

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2025-10-09 21:51:28 +02:00
Ettore Di Giacinto
a9c7ce7275 fix(chatterbox): add poetry 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-10-07 17:46:52 +02:00
Ettore Di Giacinto
20f1e842b3
fix(chatterbox): chunk long text (#6407) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-10-07 12:41:34 +02:00
dependabot[bot]
f41f30ad92
chore(deps): bump grpcio from 1.74.0 to 1.75.1 in /backend/python/exllama2 (#6356) 
chore(deps): bump grpcio in /backend/python/exllama2

Bumps [grpcio](https://github.com/grpc/grpc) from 1.74.0 to 1.75.1.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.74.0...v1.75.1)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.75.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-30 14:40:41 +02:00
dependabot[bot]
e77340e8a5
chore(deps): bump grpcio from 1.75.0 to 1.75.1 in /backend/python/transformers (#6362) 
chore(deps): bump grpcio in /backend/python/transformers

Bumps [grpcio](https://github.com/grpc/grpc) from 1.75.0 to 1.75.1.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.75.0...v1.75.1)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.75.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-30 14:40:29 +02:00
dependabot[bot]
d51a3090f7
chore(deps): bump grpcio from 1.74.0 to 1.75.1 in /backend/python/bark (#6359) 
Bumps [grpcio](https://github.com/grpc/grpc) from 1.74.0 to 1.75.1.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.74.0...v1.75.1)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.75.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-30 14:40:16 +02:00
dependabot[bot]
1bf3bc932c
chore(deps): bump grpcio from 1.74.0 to 1.75.1 in /backend/python/vllm (#6357) 
Bumps [grpcio](https://github.com/grpc/grpc) from 1.74.0 to 1.75.1.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.74.0...v1.75.1)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.75.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-30 14:40:02 +02:00
dependabot[bot]
564a47da4e
chore(deps): bump grpcio from 1.74.0 to 1.75.1 in /backend/python/common/template (#6358) 
chore(deps): bump grpcio in /backend/python/common/template

Bumps [grpcio](https://github.com/grpc/grpc) from 1.74.0 to 1.75.1.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.74.0...v1.75.1)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.75.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-30 08:52:36 +02:00
dependabot[bot]
c37ee93ff2
chore(deps): bump grpcio from 1.74.0 to 1.75.1 in /backend/python/rerankers (#6360) 
chore(deps): bump grpcio in /backend/python/rerankers

Bumps [grpcio](https://github.com/grpc/grpc) from 1.74.0 to 1.75.1.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.74.0...v1.75.1)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.75.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-30 08:52:25 +02:00
dependabot[bot]
f4b65db4e7
chore(deps): bump grpcio from 1.74.0 to 1.75.1 in /backend/python/diffusers (#6361) 
chore(deps): bump grpcio in /backend/python/diffusers

Bumps [grpcio](https://github.com/grpc/grpc) from 1.74.0 to 1.75.1.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.74.0...v1.75.1)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.75.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-30 08:52:11 +02:00
Ettore Di Giacinto
f5fa8e6649
Revert "chore(deps): bump transformers from 4.48.3 to 4.56.2 in /backend/python/coqui" (#6363) 
Revert "chore(deps): bump transformers from 4.48.3 to 4.56.2 in /backend/pyth…"

This reverts commit 570e39bdcf.
 2025-09-30 08:51:49 +02:00
dependabot[bot]
570e39bdcf
chore(deps): bump transformers from 4.48.3 to 4.56.2 in /backend/python/coqui (#6330) 
chore(deps): bump transformers in /backend/python/coqui

Bumps [transformers](https://github.com/huggingface/transformers) from 4.48.3 to 4.56.2.
- [Release notes](https://github.com/huggingface/transformers/releases)
- [Commits](https://github.com/huggingface/transformers/compare/v4.48.3...v4.56.2)

---
updated-dependencies:
- dependency-name: transformers
  dependency-version: 4.56.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-29 21:30:29 +00:00
dependabot[bot]
2ebe37b671
chore(deps): bump grpcio from 1.74.0 to 1.75.1 in /backend/python/coqui (#6353) 
Bumps [grpcio](https://github.com/grpc/grpc) from 1.74.0 to 1.75.1.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.74.0...v1.75.1)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.75.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-29 20:11:55 +00:00
Ettore Di Giacinto
c85d559919
feat(chatterbox): support multilingual (#6240) 
* feat(chatterbox): support multilingual

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add l4t support

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Fixups

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix: switch to fork

Until https://github.com/resemble-ai/chatterbox/pull/295 is merged

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-09-24 18:37:37 +02:00
dependabot[bot]
0ae334fc62
chore(deps): bump grpcio from 1.74.0 to 1.75.0 in /backend/python/transformers (#6332) 
chore(deps): bump grpcio in /backend/python/transformers

Bumps [grpcio](https://github.com/grpc/grpc) from 1.74.0 to 1.75.0.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.74.0...v1.75.0)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.75.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-22 19:53:35 +00:00
Ettore Di Giacinto
36c373b7c9
feat(kokoro): add support for l4t devices (#6322) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-09-22 10:33:26 +02:00
Ettore Di Giacinto
660bd45be8
fix(python): make option check uniform across backends (#6314) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-09-19 19:56:08 +02:00
Ettore Di Giacinto
c27da0a0f6
fix(diffusers): fix float detection (#6313) 
There was apparently an oversight, this fixes the float/int detection

Fixes: https://github.com/mudler/LocalAI/issues/6312

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2025-09-19 19:09:04 +02:00
Ettore Di Giacinto
ffa934b959
feat(chatterbox): add MPS, and CPU, pin version (#6242) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-09-09 17:58:07 +02:00
Ettore Di Giacinto
ead00a28b9
Add 'optimum-quanto' to requirements 
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2025-08-28 13:32:03 +02:00
Ettore Di Giacinto
9621edb4c5
feat(diffusers): add support for wan2.2 (#6153) 
* feat(diffusers): add support for wan2.2

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore(ci): use ttl.sh for PRs

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add ftfy deps

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Revert "chore(ci): use ttl.sh for PRs"

This reverts commit c9fc3ecf28.

* Simplify

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore: do not pin torch/torchvision on cuda12

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-28 10:26:42 +02:00
Ettore Di Giacinto
7ce92f0646
fix: select portable environment if detected (#6158) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-28 10:00:54 +02:00
Ettore Di Giacinto
3be8b2d8e1
chore(refactor): cli -> cmd, update docs (#6148) 
* chore(refactor): cli -> cmd

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Update README

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-26 19:07:10 +02:00
Ettore Di Giacinto
3c3f477854
feat(mlx-audio): Add mlx-audio backend (#6138) 
* feat(mlx-audio): Add mlx-audio backend

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* improve loading

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* CI tests

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix: set C_INCLUDE_PATH to point to python install

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-26 15:27:06 +02:00
Ettore Di Giacinto
960e51e527
chore(diffusers): support both src and reference_images in diffusers (#6135) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-24 22:03:08 +02:00
Ettore Di Giacinto
ff5d2dc8be
Revert "fix(rfdetr): use cpu torch for cpu builds" (#6131) 
Revert "fix(rfdetr): use cpu torch for cpu builds (#6129)"

This reverts commit fec8a36b36.
 2025-08-24 11:41:08 +02:00
Ettore Di Giacinto
fec8a36b36
fix(rfdetr): use cpu torch for cpu builds (#6129) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-24 10:17:25 +02:00
Ettore Di Giacinto
6971f71a6c
Add mlx-vlm (#6119) 
* Add mlx-vlm

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add to CI workflows

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add requirements-mps.txt

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Simplify

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-23 23:05:30 +02:00
Ettore Di Giacinto
1ba66d00f5
feat: bundle python inside backends (#6123) 
* feat(backends): bundle python

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test ci

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* vllm on self-hosted

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add clang

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Try to fix it for Mac

* Relocate links only when is portable

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Make sure to call macosPortableEnv

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Use self-hosted for vllm

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Fixups

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* CI

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-23 22:36:39 +02:00
Ettore Di Giacinto
0fd395d6ec
feat(diffusers): add MPS version (#6121) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-22 23:14:54 +02:00
Ettore Di Giacinto
1d830ce7dd
feat(mlx): add mlx backend (#6049) 
* chore: allow to install with pip

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* WIP

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Make the backend to build and actually work

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* List models from system only

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add script to build darwin python backends

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Run protogen in libbackend

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Detect if mps is available across python backends

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* CI: try to build backend

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Debug CI

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Fixups

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Fixups

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Index mlx-vlm

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Remove mlx-vlm

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Drop CI test

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-22 08:42:29 +02:00
Ettore Di Giacinto
8193d18c7c
feat(img2img): Add support to Qwen Image Edit (#6113) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-20 10:18:55 +02:00
Ettore Di Giacinto
060037bcd4
Revert "chore(deps): bump transformers from 4.48.3 to 4.55.2 in /backend/python/coqui" (#6105) 
Revert "chore(deps): bump transformers from 4.48.3 to 4.55.2 in /backend/pyth…"

This reverts commit 27ce570844.
 2025-08-19 15:00:33 +02:00
Ettore Di Giacinto
d9da4676b4
Revert "chore(deps): bump torch from 2.3.1+cxx11.abi to 2.8.0 in /backend/python/coqui" (#6104) 
Revert "chore(deps): bump torch from 2.3.1+cxx11.abi to 2.8.0 in /backend/pyt…"

This reverts commit 42c7859ab1.
 2025-08-19 15:00:11 +02:00
Ettore Di Giacinto
5ef4c2e471
feat(diffusers): add torchvision to support qwen-image-edit (#6103) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-19 12:05:48 +02:00
dependabot[bot]
27ce570844
chore(deps): bump transformers from 4.48.3 to 4.55.2 in /backend/python/coqui (#6096) 
chore(deps): bump transformers in /backend/python/coqui

Bumps [transformers](https://github.com/huggingface/transformers) from 4.48.3 to 4.55.2.
- [Release notes](https://github.com/huggingface/transformers/releases)
- [Commits](https://github.com/huggingface/transformers/compare/v4.48.3...v4.55.2)

---
updated-dependencies:
- dependency-name: transformers
  dependency-version: 4.55.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-19 09:44:01 +00:00
dependabot[bot]
42c7859ab1
chore(deps): bump torch from 2.3.1+cxx11.abi to 2.8.0 in /backend/python/coqui (#6099) 
chore(deps): bump torch in /backend/python/coqui

Bumps [torch](https://github.com/pytorch/pytorch) from 2.3.1+cxx11.abi to 2.8.0.
- [Release notes](https://github.com/pytorch/pytorch/releases)
- [Changelog](https://github.com/pytorch/pytorch/blob/main/RELEASE.md)
- [Commits](https://github.com/pytorch/pytorch/commits/v2.8.0)

---
updated-dependencies:
- dependency-name: torch
  dependency-version: 2.8.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-19 08:42:52 +00:00
Ettore Di Giacinto
e7e83d0fa6
Revert "chore(deps): bump intel-extension-for-pytorch from 2.3.110+xpu to 2.8.10+xpu in /backend/python/coqui" (#6102) 
Revert "chore(deps): bump intel-extension-for-pytorch from 2.3.110+xpu to 2.8…"

This reverts commit c6dc1d86f1.
 2025-08-19 09:29:56 +02:00
dependabot[bot]
c6dc1d86f1
chore(deps): bump intel-extension-for-pytorch from 2.3.110+xpu to 2.8.10+xpu in /backend/python/coqui (#6095) 
chore(deps): bump intel-extension-for-pytorch in /backend/python/coqui

Bumps intel-extension-for-pytorch from 2.3.110+xpu to 2.8.10+xpu.

---
updated-dependencies:
- dependency-name: intel-extension-for-pytorch
  dependency-version: 2.8.10+xpu
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-19 07:09:47 +00:00
dependabot[bot]
6fd2e1964d
chore(deps): bump grpcio from 1.71.0 to 1.74.0 in /backend/python/coqui (#6097) 
Bumps [grpcio](https://github.com/grpc/grpc) from 1.71.0 to 1.74.0.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.71.0...v1.74.0)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.74.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-19 08:11:58 +02:00
dependabot[bot]
2b6be10b6b
chore(deps): bump protobuf from 6.31.0 to 6.32.0 in /backend/python/transformers (#6100) 
chore(deps): bump protobuf in /backend/python/transformers

Bumps [protobuf](https://github.com/protocolbuffers/protobuf) from 6.31.0 to 6.32.0.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/protobuf_release.bzl)
- [Commits](https://github.com/protocolbuffers/protobuf/commits)

---
updated-dependencies:
- dependency-name: protobuf
  dependency-version: 6.32.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-19 05:09:17 +00:00
dependabot[bot]
9d9c853541
chore(deps): bump grpcio from 1.71.0 to 1.74.0 in /backend/python/transformers (#6013) 
chore(deps): bump grpcio in /backend/python/transformers

Bumps [grpcio](https://github.com/grpc/grpc) from 1.71.0 to 1.74.0.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.71.0...v1.74.0)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.74.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-12 22:05:16 +02:00
dependabot[bot]
d8e27c38d7
chore(deps): bump oneccl-bind-pt from 2.3.100+xpu to 2.8.0+xpu in /backend/python/common/template (#6016) 
chore(deps): bump oneccl-bind-pt in /backend/python/common/template

Bumps oneccl-bind-pt from 2.3.100+xpu to 2.8.0+xpu.

---
updated-dependencies:
- dependency-name: oneccl-bind-pt
  dependency-version: 2.8.0+xpu
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-12 18:57:20 +00:00
dependabot[bot]
3b0dc87932
chore(deps): bump torch from 2.3.1+cxx11.abi to 2.8.0 in /backend/python/common/template (#6025) 
chore(deps): bump torch in /backend/python/common/template

Bumps [torch](https://github.com/pytorch/pytorch) from 2.3.1+cxx11.abi to 2.8.0.
- [Release notes](https://github.com/pytorch/pytorch/releases)
- [Changelog](https://github.com/pytorch/pytorch/blob/main/RELEASE.md)
- [Commits](https://github.com/pytorch/pytorch/commits/v2.8.0)

---
updated-dependencies:
- dependency-name: torch
  dependency-version: 2.8.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-12 17:58:33 +00:00
dependabot[bot]
90b5ed9a1e
chore(deps): bump intel-extension-for-pytorch from 2.3.110+xpu to 2.8.10+xpu in /backend/python/common/template (#6034) 
chore(deps): bump intel-extension-for-pytorch

Bumps intel-extension-for-pytorch from 2.3.110+xpu to 2.8.10+xpu.

---
updated-dependencies:
- dependency-name: intel-extension-for-pytorch
  dependency-version: 2.8.10+xpu
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-12 18:44:33 +02:00
dependabot[bot]
d438b769da
chore(deps): bump intel-extension-for-pytorch from 2.3.110+xpu to 2.8.10+xpu in /backend/python/bark (#6043) 
chore(deps): bump intel-extension-for-pytorch in /backend/python/bark

Bumps intel-extension-for-pytorch from 2.3.110+xpu to 2.8.10+xpu.

---
updated-dependencies:
- dependency-name: intel-extension-for-pytorch
  dependency-version: 2.8.10+xpu
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-12 18:44:05 +02:00
dependabot[bot]
2e4bd1e33d
chore(deps): bump oneccl-bind-pt from 2.3.100+xpu to 2.8.0+xpu in /backend/python/rerankers (#6021) 
chore(deps): bump oneccl-bind-pt in /backend/python/rerankers

Bumps oneccl-bind-pt from 2.3.100+xpu to 2.8.0+xpu.

---
updated-dependencies:
- dependency-name: oneccl-bind-pt
  dependency-version: 2.8.0+xpu
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-12 16:04:54 +00:00
dependabot[bot]
ff73800970
chore(deps): bump grpcio from 1.71.0 to 1.74.0 in /backend/python/exllama2 (#6019) 
chore(deps): bump grpcio in /backend/python/exllama2

Bumps [grpcio](https://github.com/grpc/grpc) from 1.71.0 to 1.74.0.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.71.0...v1.74.0)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.74.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-12 16:42:46 +02:00
dependabot[bot]
47c20f9adb
chore(deps): bump grpcio from 1.71.0 to 1.74.0 in /backend/python/rerankers (#6022) 
chore(deps): bump grpcio in /backend/python/rerankers

Bumps [grpcio](https://github.com/grpc/grpc) from 1.71.0 to 1.74.0.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.71.0...v1.74.0)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.74.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-12 15:24:48 +02:00
dependabot[bot]
a7fe153630
chore(deps): bump grpcio from 1.71.0 to 1.74.0 in /backend/python/bark (#6033) 
Bumps [grpcio](https://github.com/grpc/grpc) from 1.71.0 to 1.74.0.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.71.0...v1.74.0)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.74.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-12 15:16:00 +02:00
dependabot[bot]
27519d2233
chore(deps): bump grpcio from 1.71.0 to 1.74.0 in /backend/python/common/template (#6035) 
chore(deps): bump grpcio in /backend/python/common/template

Bumps [grpcio](https://github.com/grpc/grpc) from 1.71.0 to 1.74.0.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.71.0...v1.74.0)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.74.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-12 15:15:28 +02:00
dependabot[bot]
8cab0f880b
chore(deps): bump sentence-transformers from 5.0.0 to 5.1.0 in /backend/python/transformers (#6028) 
chore(deps): bump sentence-transformers in /backend/python/transformers

Bumps [sentence-transformers](https://github.com/UKPLab/sentence-transformers) from 5.0.0 to 5.1.0.
- [Release notes](https://github.com/UKPLab/sentence-transformers/releases)
- [Commits](https://github.com/UKPLab/sentence-transformers/compare/v5.0.0...v5.1.0)

---
updated-dependencies:
- dependency-name: sentence-transformers
  dependency-version: 5.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-12 15:15:07 +02:00
dependabot[bot]
8c48b250c4
chore(deps): bump grpcio from 1.71.0 to 1.74.0 in /backend/python/diffusers (#6037) 
chore(deps): bump grpcio in /backend/python/diffusers

Bumps [grpcio](https://github.com/grpc/grpc) from 1.71.0 to 1.74.0.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.71.0...v1.74.0)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.74.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-12 15:14:35 +02:00
dependabot[bot]
ba802c2ee4
chore(deps): bump grpcio from 1.71.0 to 1.74.0 in /backend/python/vllm (#6036) 
Bumps [grpcio](https://github.com/grpc/grpc) from 1.71.0 to 1.74.0.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.71.0...v1.74.0)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-version: 1.74.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-12 15:14:15 +02:00
Ettore Di Giacinto
5f221f5946
fix(l4t-diffusers): add sentencepiece (#6005) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-09 09:08:35 +02:00
Ettore Di Giacinto
b8eca530b6
feat(diffusers): add builds for nvidia-l4t (#6004) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-08 22:48:38 +02:00
Ettore Di Giacinto
4733adb983
chore: add Dia to the model gallery, fix backend (#5998) 
* fix: correctly call OuteTTS and DiaTTS

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore(model gallery): add dia

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-08 12:40:16 +02:00
Ettore Di Giacinto
003b9292fe
feat(transformers): add support to Dia (#5991) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-07 21:51:52 +02:00
Ettore Di Giacinto
9c2840ac38
feat(kokoro): complete kokoro integration (#5978) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-06 15:23:29 +02:00
Ettore Di Giacinto
20a70e1244
feat(backends): add KittenTTS (#5977) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-06 12:38:45 +02:00
Ettore Di Giacinto
c092633cd7
feat(models): add support to qwen-image (#5975) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-06 10:36:53 +02:00
Ettore Di Giacinto
9087ddc4de
chore(deps): bump torch and sentence-transformers (#5969) 
* chore(deps): bump torch and sentence-transformers

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore(ci): add backend build tests

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore: move jobs to self-hosted

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-05 19:45:20 +02:00
Ettore Di Giacinto
33bebd5114
chore(deps): bump torch and diffusers (#5970) 
* chore(ci): add backend build tests

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore(deps): bump torch and diffusers

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore(ci): run diffusers/hipblas on self-hosted

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore(ci): do not publish darwin if building from PRs

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-05 14:47:01 +02:00
Ettore Di Giacinto
949e5b9be8
feat(rfdetr): add object detection API (#5923) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-07-27 22:02:51 +02:00
Ettore Di Giacinto
a8057b952c
fix(cuda): be consistent with image tag naming (#5916) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-07-26 08:30:59 +02:00
Richard Palethorpe
b37cef3718
fix: Diffusers and XPU fixes (#5737) 
* fix(README): Add device flags for Intel/XPU

Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(diffusers/xpu): Set device to XPU and ignore CUDA request when on Intel

Signed-off-by: Richard Palethorpe <io@richiejp.com>

---------

Signed-off-by: Richard Palethorpe <io@richiejp.com>
 2025-07-01 12:36:17 +02:00
Maxim Evtush
add8fc35a2
Fix Typos in Documentation and Python Comments (#5658) 
* Update istftnet.py

Signed-off-by: Maxim Evtush <154841002+maximevtush@users.noreply.github.com>

* Update GPU-acceleration.md

Signed-off-by: Maxim Evtush <154841002+maximevtush@users.noreply.github.com>

---------

Signed-off-by: Maxim Evtush <154841002+maximevtush@users.noreply.github.com>
 2025-06-18 22:11:13 +02:00
Ettore Di Giacinto
89040ff6f7
fix: add python symlink, use absolute python env path when running backends (#5664) 
* fix: add python symlink, use absolute python env path when running backends

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ci): do not push images when building PRs

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-06-16 23:00:53 +02:00
Ettore Di Giacinto
2d64269763
feat: Add backend gallery (#5607) 
* feat: Add backend gallery

This PR add support to manage backends as similar to models. There is
now available a backend gallery which can be used to install and remove
extra backends.
The backend gallery can be configured similarly as a model gallery, and
API calls allows to install and remove new backends in runtime, and as
well during the startup phase of LocalAI.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add backends docs

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* wip: Backend Dockerfile for python backends

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat: drop extras images, build python backends separately

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fixup on all backends

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test CI

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Tweaks

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Drop old backends leftovers

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Fixup CI

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Move dockerfile upper

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Fix proto

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Feature dropped for consistency - we prefer model galleries

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add missing packages in the build image

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* exllama is ponly available on cublas

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* pin torch on chatterbox

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Fixups to index

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* CI

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Debug CI

* Install accellerators deps

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add target arch

* Add cuda minor version

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Use self-hosted runners

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* ci: use quay for test images

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fixups for vllm and chatterbox

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Small fixups on CI

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chatterbox is only available for nvidia

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Simplify CI builds

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Adapt test, use qwen3

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore(model gallery): add jina-reranker-v1-tiny-en-gguf

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gguf-parser): recover from potential panics that can happen while reading ggufs with gguf-parser

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Use reranker from llama.cpp in AIO images

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Limit concurrent jobs

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2025-06-15 14:56:52 +02:00
fuder.eth
eb8c29f90a
Minor Documentation Updates: Clarified Comments in Python and Go Files (#5641) 
* Update ui.go

Signed-off-by: fuder.eth <139509124+vtjl10@users.noreply.github.com>

* Update backend.py

Signed-off-by: fuder.eth <139509124+vtjl10@users.noreply.github.com>

---------

Signed-off-by: fuder.eth <139509124+vtjl10@users.noreply.github.com>
 2025-06-13 19:55:25 +02:00
Ettore Di Giacinto
88e570b5de
fix(deps): pin grpcio (#5621) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-06-10 14:21:51 +02:00
Ettore Di Giacinto
8b889955b4
chore(deps): bump pytorch to 2.7 in vllm (#5576) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-06-04 08:56:45 +02:00
Ettore Di Giacinto
ec0868e691
chore(deps): bump grpcio from 1.72.0 to 1.72.1 (#5570) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-06-03 09:59:43 +02:00
Ettore Di Giacinto
d5c9c717b5
feat(chatterbox): add new backend (#5524) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-05-30 10:52:55 +02:00
Ettore Di Giacinto
5ffad3b004
chore(deps): remove pin on transformers (#5501) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-05-27 09:24:27 +02:00
Ettore Di Giacinto
6a382a1afe
fix(transformers): try to pin to working release (#5426) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-05-22 12:50:51 +02:00
Ettore Di Giacinto
82811a9630
fix(transformers): pin protobuf (#5421) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-05-20 20:28:31 +02:00
Wyatt Neal
4076ea0494
fix: vllm missing logprobs (#5279) 
* working to address missing items

referencing #3436, #2930 - if i could test it, this might show that the
output from the vllm backend is processed and returned to the user

Signed-off-by: Wyatt Neal <wyatt.neal+git@gmail.com>

* adding in vllm tests to test-extras

Signed-off-by: Wyatt Neal <wyatt.neal+git@gmail.com>

* adding in tests to pipeline for execution

Signed-off-by: Wyatt Neal <wyatt.neal+git@gmail.com>

* removing todo block, test via pipeline

Signed-off-by: Wyatt Neal <wyatt.neal+git@gmail.com>

---------

Signed-off-by: Wyatt Neal <wyatt.neal+git@gmail.com>
 2025-04-30 12:55:07 +00:00
Ettore Di Giacinto
6e8f4f584b
fix(diffusers): consider options only in form of key/value (#5277) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-04-29 17:08:55 +02:00
Ettore Di Giacinto
cae9bf1308
chore(deps): bump grpcio to 1.72.0 (#5244) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-04-25 21:32:37 +02:00
Ettore Di Giacinto
61cc76c455
chore(autogptq): drop archived backend (#5214) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-04-19 15:52:29 +02:00
Ettore Di Giacinto
8abecb4a18
chore: bump grpc limits to 50MB (#5212) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-04-19 08:53:24 +02:00
Ettore Di Giacinto
d484028532
feat(diffusers): add support for Lumina2Text2ImgPipeline (#4806) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-04-10 09:55:51 +02:00
Ettore Di Giacinto
1db2b9943c
chore(deps): Bump grpcio to 1.71.0 (#4993) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-03-11 09:44:21 +01:00
Ettore Di Giacinto
879dc73eba
Revert "chore(deps): Bump intel-extension-for-pytorch from 2.3.110+xpu to 2.6.10+xpu in /backend/python/diffusers" (#4992) 
Revert "chore(deps): Bump intel-extension-for-pytorch from 2.3.110+xpu to 2.6…"

This reverts commit 1dfc52de16.
 2025-03-11 08:29:05 +01:00
dependabot[bot]
1dfc52de16
chore(deps): Bump intel-extension-for-pytorch from 2.3.110+xpu to 2.6.10+xpu in /backend/python/diffusers (#4973) 
chore(deps): Bump intel-extension-for-pytorch

Bumps intel-extension-for-pytorch from 2.3.110+xpu to 2.6.10+xpu.

---
updated-dependencies:
- dependency-name: intel-extension-for-pytorch
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-03-10 21:14:43 +00:00
Ettore Di Giacinto
af3bb64e42
fix(coqui): pin transformers (#4875) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-02-20 16:16:54 +01:00
Brandon Beiler
6a6e1a0ea9
feat(vllm): Additional vLLM config options (Disable logging, dtype, and Per-Prompt media limits) (#4855) 
* Adding the following vLLM config options: disable_log_status, dtype, limit_mm_per_prompt

Signed-off-by: TheDropZone <brandonbeiler@gmail.com>

* using " marks in the config.yaml file

Signed-off-by: TheDropZone <brandonbeiler@gmail.com>

* adding in missing colon

Signed-off-by: TheDropZone <brandonbeiler@gmail.com>

---------

Signed-off-by: TheDropZone <brandonbeiler@gmail.com>
 2025-02-18 19:27:58 +01:00
Ettore Di Giacinto
f5638a6354
feat(diffusers): allow to override image gen options (#4807) 
Use the options field in the model to override kwargs if needed.

This allows to specify from the model yaml config:

```yaml

options:
- foo:bar

```

And each option will be used directly when calling the diffusers
pipeline, e.g:

```python
pipe(
  foo="bar",
)
```

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-02-11 10:16:32 +01:00
dependabot[bot]
5a19094d3a
chore(deps): Bump sentence-transformers from 3.4.0 to 3.4.1 in /backend/python/transformers (#4748) 
chore(deps): Bump sentence-transformers in /backend/python/transformers

Bumps [sentence-transformers](https://github.com/UKPLab/sentence-transformers) from 3.4.0 to 3.4.1.
- [Release notes](https://github.com/UKPLab/sentence-transformers/releases)
- [Commits](https://github.com/UKPLab/sentence-transformers/compare/v3.4.0...v3.4.1)

---
updated-dependencies:
- dependency-name: sentence-transformers
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-02-04 08:56:51 +01:00
dependabot[bot]
fff35d5528
chore(deps): Bump sentence-transformers from 3.3.1 to 3.4.0 in /backend/python/transformers (#4702) 
chore(deps): Bump sentence-transformers in /backend/python/transformers

Bumps [sentence-transformers](https://github.com/UKPLab/sentence-transformers) from 3.3.1 to 3.4.0.
- [Release notes](https://github.com/UKPLab/sentence-transformers/releases)
- [Commits](https://github.com/UKPLab/sentence-transformers/compare/v3.3.1...v3.4.0)

---
updated-dependencies:
- dependency-name: sentence-transformers
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-01-27 21:09:50 +00:00
Ettore Di Giacinto
4d44ebc2f2
chore(deps): bump grpcio to 1.70.0 (#4682) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-01-24 10:18:22 +01:00
Ettore Di Giacinto
073eaec729
chore(openvoice): drop backend (#4673) 
The project (MeloTTS) has been quite since long, newer backends are much
performant and better quality overall.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-01-23 10:00:36 +01:00
Ettore Di Giacinto
318225f631
chore(parler-tts): drop backend (#4672) 
We support at this point more extensive backends that are SOTA and
support also voice cloning, and many other features. This backend is
superseded and also poses significant maintenance burden as there is an
open issue https://github.com/mudler/LocalAI/issues/3941 which is still
open as it deps are pinning old versions of grpc.

Closes https://github.com/mudler/LocalAI/issues/3941

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-01-23 09:46:16 +01:00
Ettore Di Giacinto
89429a439b
feat(transformers): add support to Mamba (#4669) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-01-23 09:30:47 +01:00
Ettore Di Giacinto
e426ab7c23
feat(faster-whisper): add backend (#4666) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-01-23 08:06:18 +01:00
Ettore Di Giacinto
f496d0113b chore(deps): pin numba 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-01-19 09:07:56 +01:00