mirror of
https://github.com/mudler/LocalAI
synced 2026-05-24 09:28:23 +00:00
c844b7ac58
* feat: allow to set forcing backends eviction while requests are in flight Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat: try to make the request sit and retry if eviction couldn't be done Otherwise calls that in order to pass would need to shutdown other backends would just fail. In this way instead we make the request sit and retry eviction until it succeeds. The thresholds can be configured by the user. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * add tests Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * expose settings to CLI Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Update docs Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|---|---|---|
| .. | ||
| audio | ||
| concurrency | ||
| downloader | ||
| functions | ||
| grpc | ||
| huggingface-api | ||
| langchain | ||
| model | ||
| oci | ||
| signals | ||
| sound | ||
| store | ||
| system | ||
| utils | ||
| xio | ||
| xsync | ||
| xsysinfo | ||