mirror of
https://github.com/mudler/LocalAI
synced 2026-05-24 09:28:23 +00:00
13734ae9fa
feat(backend): Add Sherpa ONNX backend and Omnilingual ASR Adds a new Go backend wrapping sherpa-onnx via purego (no cgo). Same approach as opus/stablediffusion-ggml/whisper — a thin C shim (csrc/shim.c + shim.h → libsherpa-shim.so) wraps the bits purego can't reach directly: nested struct config writes, result-struct field reads, and the streaming TTS callback trampoline. The Go side uses opaque uintptr handles and purego.NewCallback for the TTS callback. Supports: - VAD via sherpa-onnx's Silero VAD - Offline ASR: Whisper, Paraformer, SenseVoice, Omnilingual CTC - Online/streaming ASR: zipformer transducer with endpoint detection (AudioTranscriptionStream emits delta events during decode) - Offline TTS: VITS (LJS, etc.) - Streaming TTS: sherpa-onnx's callback API → PCM chunks on a channel, prefixed by a streaming WAV header Gallery entries: omnilingual-0.3b-ctc-q8-sherpa (1600-language offline ASR), streaming-zipformer-en-sherpa (low-latency streaming ASR), silero-vad-sherpa, vits-ljs-sherpa. E2E coverage: tests/e2e-backends for offline + streaming ASR, tests/e2e for the full realtime pipeline (VAD + STT + TTS). Assisted-by: claude-opus-4-7-1M [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> |
||
|---|---|---|
| .. | ||
| disabled | ||
| backend.yml | ||
| backend_build.yml | ||
| backend_build_darwin.yml | ||
| backend_pr.yml | ||
| build-test.yaml | ||
| bump-inference-defaults.yml | ||
| bump_deps.yaml | ||
| bump_docs.yaml | ||
| checksum_checker.yaml | ||
| deploy-explorer.yaml | ||
| gallery-agent.yaml | ||
| generate_grpc_cache.yaml | ||
| generate_intel_image.yaml | ||
| gh-pages.yml | ||
| image-pr.yml | ||
| image.yml | ||
| image_build.yml | ||
| notify-releases.yaml | ||
| release.yaml | ||
| secscan.yaml | ||
| stalebot.yml | ||
| test-extra.yml | ||
| test.yml | ||
| tests-e2e.yml | ||
| tests-ui-e2e.yml | ||
| update_swagger.yaml | ||
| yaml-check.yml | ||