LocalAI/core/services
Ettore Di Giacinto fc5b9ebfcc
feat(loader): enhance single active backend to support LRU eviction (#7535)
* feat(loader): refactor single active backend support to LRU

This changeset introduces LRU management of loaded backends. Users can
set now a maximum number of models to be loaded concurrently, and, when
setting LocalAI in single active backend mode we set LRU to 1 for
backward compatibility.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore: add tests

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Update docs

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Fixups

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2025-12-12 12:28:38 +01:00
..
agent_jobs.go feat(agent-jobs): add multimedia support (#7398) 2025-11-30 14:09:25 +01:00
agent_jobs_test.go feat(loader): enhance single active backend to support LRU eviction (#7535) 2025-12-12 12:28:38 +01:00
backend_monitor.go feat(ui): General improvements (#6072) 2025-08-16 07:44:50 +02:00
backends.go feat(ui): allow to cancel ops (#7264) 2025-11-13 18:41:47 +01:00
gallery.go feat(ui): allow to cancel ops (#7264) 2025-11-13 18:41:47 +01:00
list_models.go feat(backends): add system backend, refactor (#6059) 2025-08-14 19:38:26 +02:00
metrics.go chore: fix go.mod module (#2635) 2024-06-23 08:24:36 +00:00
models.go feat(importer): unify importing code with CLI (#7299) 2025-11-19 20:52:11 +01:00
operation.go feat(ui): allow to cancel ops (#7264) 2025-11-13 18:41:47 +01:00
services_suite_test.go feat: agent jobs panel (#7390) 2025-11-28 23:05:39 +01:00