mirror of
https://github.com/mudler/LocalAI
synced 2026-05-24 09:28:23 +00:00
c60ed75258
Add a routing middleware stack and a cloud-proxy backend. * cloud-proxy: a Go gRPC backend that forwards OpenAI- and Anthropic-shaped chat requests to upstream providers, with an optional translate mode (OpenAI request -> Anthropic /v1/messages -> OpenAI response) and full tool-calling support. * routing: admission control, content-aware model routing (embedding cache + classifier + rerank + Arch-Router score), PII detection/redaction (regex + NER) with streaming filter and OpenAI/Anthropic adapters, and a per-user/per-key billing recorder backed by GORM or in-memory storage. * middleware: UsageMiddleware records usage via the billing recorder, plus admission, route-model, usage-stamp and trace middlewares. * observability: BackendTrace ring buffer stores full request bodies (capped), MITM proxy emits structured trace events, and router classifier decisions surface at /api/router/decide. * gallery: Arch-Router-1.5B (Q4_K_M and Q8_0). * UI: cloud-proxy model-editor fields, classifier system-prompt and score-normalization config, and a Traces page rendering request bodies. Assisted-by: claude-code:claude-opus-4-7 [Read] [Edit] [Bash] Signed-off-by: Richard Palethorpe <io@richiejp.com> |
||
|---|---|---|
| .. | ||
| gen_inference_defaults | ||
| meta | ||
| application_config.go | ||
| application_config_test.go | ||
| backend_capabilities.go | ||
| backend_capabilities_test.go | ||
| backend_hooks.go | ||
| config_suite_test.go | ||
| distributed_config.go | ||
| distributed_config_test.go | ||
| gallery.go | ||
| gguf.go | ||
| gguf_reasoning_test.go | ||
| hooks_llamacpp.go | ||
| hooks_test.go | ||
| hooks_vllm.go | ||
| inference_defaults.go | ||
| inference_defaults.json | ||
| inference_defaults_test.go | ||
| mitm_host_owners_test.go | ||
| model_config.go | ||
| model_config_filter.go | ||
| model_config_loader.go | ||
| model_config_loader_test.go | ||
| model_config_test.go | ||
| model_test.go | ||
| mtp.go | ||
| mtp_test.go | ||
| parser_defaults.json | ||
| runtime_settings.go | ||
| runtime_settings_persist.go | ||
| runtime_settings_persist_test.go | ||