mirror of
https://github.com/mudler/LocalAI
synced 2026-05-24 09:28:23 +00:00
a026277ab9
* feat(mlx-distributed): add new MLX-distributed backend Add new MLX distributed backend with support for both TCP and RDMA for model sharding. This implementation ties in the discovery implementation already in place, and re-uses the same P2P mechanism for the TCP MLX-distributed inferencing. The Auto-parallel implementation is inspired by Exo's ones (who have been added to acknowledgement for the great work!) Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * expose a CLI to facilitate backend starting Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat: make manual rank0 configurable via model configs Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add missing features from mlx backend Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Apply suggestion from @mudler Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| _index.en.md | ||
| agents.md | ||
| audio-to-text.md | ||
| backend-monitor.md | ||
| backends.md | ||
| constrained_grammars.md | ||
| distributed_inferencing.md | ||
| embeddings.md | ||
| gpt-vision.md | ||
| GPU-acceleration.md | ||
| image-generation.md | ||
| mcp.md | ||
| mlx-distributed.md | ||
| model-gallery.md | ||
| object-detection.md | ||
| openai-functions.md | ||
| openai-realtime.md | ||
| p2p.md | ||
| reranker.md | ||
| runtime-settings.md | ||
| sound-generation.md | ||
| stores.md | ||
| text-generation.md | ||
| text-to-audio.md | ||
| video-generation.md | ||
| voice-activity-detection.md | ||