LocalAI/core/http
Ettore Di Giacinto fc5b9ebfcc
feat(loader): enhance single active backend to support LRU eviction (#7535)
* feat(loader): refactor single active backend support to LRU

This changeset introduces LRU management of loaded backends. Users can
set now a maximum number of models to be loaded concurrently, and, when
setting LocalAI in single active backend mode we set LRU to 1 for
backward compatibility.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore: add tests

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Update docs

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Fixups

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2025-12-12 12:28:38 +01:00
..
endpoints feat(loader): enhance single active backend to support LRU eviction (#7535) 2025-12-12 12:28:38 +01:00
middleware fix: do not require auth for readyz/healthz endpoints (#7403) 2025-12-01 10:35:28 +01:00
routes feat(ui): allow to order search results (#7507) 2025-12-11 00:11:33 +01:00
static chore: refactor css, restyle to be slightly minimalistic (#7397) 2025-11-29 22:11:44 +01:00
views feat(loader): enhance single active backend to support LRU eviction (#7535) 2025-12-12 12:28:38 +01:00
app.go fix(paths): remove trailing slash from requests (#7451) 2025-12-07 21:45:09 +01:00
app_test.go chore(ci): do not overload the apple tests 2025-12-04 14:15:15 +01:00
explorer.go feat: migrate to echo and enable cancellation of non-streaming requests (#7270) 2025-11-14 22:57:53 +01:00
http_suite_test.go fix(llama-cpp): correctly calculate embeddings (#6259) 2025-09-13 23:11:54 +02:00
openai_mapping_test.go feat(api): OpenAI video create enpoint integration (#6777) 2025-10-27 10:06:57 +01:00
openai_videos_test.go feat: migrate to echo and enable cancellation of non-streaming requests (#7270) 2025-11-14 22:57:53 +01:00
render.go feat: migrate to echo and enable cancellation of non-streaming requests (#7270) 2025-11-14 22:57:53 +01:00