mirror of
https://github.com/mudler/LocalAI
synced 2026-05-24 09:28:23 +00:00
13734ae9fa
feat(backend): Add Sherpa ONNX backend and Omnilingual ASR Adds a new Go backend wrapping sherpa-onnx via purego (no cgo). Same approach as opus/stablediffusion-ggml/whisper — a thin C shim (csrc/shim.c + shim.h → libsherpa-shim.so) wraps the bits purego can't reach directly: nested struct config writes, result-struct field reads, and the streaming TTS callback trampoline. The Go side uses opaque uintptr handles and purego.NewCallback for the TTS callback. Supports: - VAD via sherpa-onnx's Silero VAD - Offline ASR: Whisper, Paraformer, SenseVoice, Omnilingual CTC - Online/streaming ASR: zipformer transducer with endpoint detection (AudioTranscriptionStream emits delta events during decode) - Offline TTS: VITS (LJS, etc.) - Streaming TTS: sherpa-onnx's callback API → PCM chunks on a channel, prefixed by a streaming WAV header Gallery entries: omnilingual-0.3b-ctc-q8-sherpa (1600-language offline ASR), streaming-zipformer-en-sherpa (low-latency streaming ASR), silero-vad-sherpa, vits-ljs-sherpa. E2E coverage: tests/e2e-backends for offline + streaming ASR, tests/e2e for the full realtime pipeline (VAD + STT + TTS). Assisted-by: claude-opus-4-7-1M [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> |
||
|---|---|---|
| .. | ||
| gen_inference_defaults | ||
| meta | ||
| application_config.go | ||
| application_config_test.go | ||
| backend_hooks.go | ||
| config_suite_test.go | ||
| distributed_config.go | ||
| gallery.go | ||
| gguf.go | ||
| gguf_reasoning_test.go | ||
| hooks_llamacpp.go | ||
| hooks_test.go | ||
| hooks_vllm.go | ||
| inference_defaults.go | ||
| inference_defaults.json | ||
| inference_defaults_test.go | ||
| model_config.go | ||
| model_config_filter.go | ||
| model_config_loader.go | ||
| model_config_test.go | ||
| model_test.go | ||
| parser_defaults.json | ||
| runtime_settings.go | ||