Elgato_dark/onju-v2
1
0
Fork 0
mirror of https://github.com/justLV/onju-v2 synced 2026-04-21 23:57:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
onju-v2/pipeline/config.yaml.example

70 lines
2.2 KiB
Text
Raw Normal View History

Add modular async pipeline server and ESP32 mDNS fallback Pipeline: async voice pipeline replacing monolithic threaded server. ASR, LLM, and TTS are independent pluggable services. ASR calls external parakeet-asr-server, LLM uses any OpenAI-compatible endpoint, TTS uses ElevenLabs with pluggable backend interface. Firmware: add mDNS hostname resolution as fallback when multicast discovery doesn't work. Resolves configured server_hostname via MDNS.queryHost() on boot, falls back to multicast if resolution fails. Also adds test_client.py that emulates an ESP32 device for testing without hardware (TCP server, Opus decode, mic streaming).
2026-02-07 23:04:12 +00:00
asr:
Add PTT device support, IIR DC offset fix, control API, test script updates PTT devices (--device name=ip:ptt): skip VAD, buffer audio until packets stop, skip LED commands, interrupt in-flight responses on new audio. Auto-detected from multicast "PTT" announcement. HTTP control server on :3002 for runtime device management: POST/GET/DELETE /devices Firmware: replace per-chunk DC offset with IIR filter to eliminate zipper noise at chunk boundaries (m5_echo + onjuino). Protocol: TCP timeouts use actual timeout param, failures are silent for non-critical commands (LED blink). Pipeline: labeled error logging (ASR/LLM/TTS), env var resolution warning, Gemini OpenAI-compatible endpoint support. Test scripts: rewritten to use pipeline modules, delete redundant test_opus_tts.py, add pyproject.toml (replaces requirements.txt).
2026-04-06 21:22:20 +00:00
url: "http://localhost:8100" # parakeet-asr-server
Add modular async pipeline server and ESP32 mDNS fallback Pipeline: async voice pipeline replacing monolithic threaded server. ASR, LLM, and TTS are independent pluggable services. ASR calls external parakeet-asr-server, LLM uses any OpenAI-compatible endpoint, TTS uses ElevenLabs with pluggable backend interface. Firmware: add mDNS hostname resolution as fallback when multicast discovery doesn't work. Resolves configured server_hostname via MDNS.queryHost() on boot, falls back to multicast if resolution fails. Also adds test_client.py that emulates an ESP32 device for testing without hardware (TCP server, Opus decode, mic streaming).
2026-02-07 23:04:12 +00:00
Extract conversation layer from device, delete deprecated server/ Separate memory/context/conversation into a pluggable backend so the LLM layer can be swapped without touching device or main. Two backends: - local: manages message history, sends full context to any OpenAI-compatible endpoint - managed: delegates to a remote service (OpenClaw) that owns session state Also: rename persist_file -> registry_file, remove unused services/llm.py, delete old server/ directory.
2026-04-06 18:31:38 +00:00
conversation:
backend: "local" # "local" or "managed" (e.g. OpenClaw)
local:
Add PTT device support, IIR DC offset fix, control API, test script updates PTT devices (--device name=ip:ptt): skip VAD, buffer audio until packets stop, skip LED commands, interrupt in-flight responses on new audio. Auto-detected from multicast "PTT" announcement. HTTP control server on :3002 for runtime device management: POST/GET/DELETE /devices Firmware: replace per-chunk DC offset with IIR filter to eliminate zipper noise at chunk boundaries (m5_echo + onjuino). Protocol: TCP timeouts use actual timeout param, failures are silent for non-critical commands (LED blink). Pipeline: labeled error logging (ASR/LLM/TTS), env var resolution warning, Gemini OpenAI-compatible endpoint support. Test scripts: rewritten to use pipeline modules, delete redundant test_opus_tts.py, add pyproject.toml (replaces requirements.txt).
2026-04-06 21:22:20 +00:00
base_url: "http://localhost:8080/v1" # mlx_lm.server, Ollama, OpenRouter, Gemini, etc.
api_key: "none" # set key or use ${ENV_VAR} reference
Extract conversation layer from device, delete deprecated server/ Separate memory/context/conversation into a pluggable backend so the LLM layer can be swapped without touching device or main. Two backends: - local: manages message history, sends full context to any OpenAI-compatible endpoint - managed: delegates to a remote service (OpenClaw) that owns session state Also: rename persist_file -> registry_file, remove unused services/llm.py, delete old server/ directory.
2026-04-06 18:31:38 +00:00
model: "mlx-community/gemma-3-4b-it-qat-4bit"
max_messages: 20
max_tokens: 300
system_prompt: "You are a helpful voice assistant. Keep responses concise (under 2 sentences)."
Move conversation persistence into local backend, simplify DeviceManager LocalConversation now owns its own per-device message files (data/conversations/{hostname}.json) controlled by persist_dir config. DeviceManager becomes a pure in-memory device registry — devices re-announce via multicast on boot so no persistence needed. Removes --persist CLI flag.
2026-04-06 18:55:08 +00:00
persist_dir: "data/conversations" # per-device message history (omit to disable)
Add PTT device support, IIR DC offset fix, control API, test script updates PTT devices (--device name=ip:ptt): skip VAD, buffer audio until packets stop, skip LED commands, interrupt in-flight responses on new audio. Auto-detected from multicast "PTT" announcement. HTTP control server on :3002 for runtime device management: POST/GET/DELETE /devices Firmware: replace per-chunk DC offset with IIR filter to eliminate zipper noise at chunk boundaries (m5_echo + onjuino). Protocol: TCP timeouts use actual timeout param, failures are silent for non-critical commands (LED blink). Pipeline: labeled error logging (ASR/LLM/TTS), env var resolution warning, Gemini OpenAI-compatible endpoint support. Test scripts: rewritten to use pipeline modules, delete redundant test_opus_tts.py, add pyproject.toml (replaces requirements.txt).
2026-04-06 21:22:20 +00:00
# Gemini example:
# base_url: "https://generativelanguage.googleapis.com/v1beta/openai/"
# api_key: "${GEMINI_API_KEY}"
# model: "gemini-3.1-flash-lite-preview"
Extract conversation layer from device, delete deprecated server/ Separate memory/context/conversation into a pluggable backend so the LLM layer can be swapped without touching device or main. Two backends: - local: manages message history, sends full context to any OpenAI-compatible endpoint - managed: delegates to a remote service (OpenClaw) that owns session state Also: rename persist_file -> registry_file, remove unused services/llm.py, delete old server/ directory.
2026-04-06 18:31:38 +00:00
managed:
base_url: "http://127.0.0.1:18789/v1" # OpenClaw gateway
api_key: "${OPENCLAW_GATEWAY_TOKEN}" # env var reference
model: "openclaw/default"
max_tokens: 300
OpenClaw managed backend, VAD-aware interrupt, firmware fixes - Add managed conversation backend for OpenClaw (x-openclaw-message-channel header, user field for device identity) - Replace aggressive interrupt logic with VAD-aware check: only interrupt on actual speech, not background noise/trailing packets - Fix 0xDD timeout units (was milliseconds, now seconds) and keep callActive alive with 30s hold during LLM+TTS processing - Set callActive on boot for VOX mode so device accepts audio without tap - Mic timeout no longer kills callActive — only double-tap ends the call - LED feedback: scale to configured led_power, let device handle fade-down - Add greeting toggle, TTS/SEND logging, pyserial dep, setuptools config
2026-04-08 03:16:21 +00:00
message_channel: "onju-voice" # x-openclaw-message-channel header
Extract conversation layer from device, delete deprecated server/ Separate memory/context/conversation into a pluggable backend so the LLM layer can be swapped without touching device or main. Two backends: - local: manages message history, sends full context to any OpenAI-compatible endpoint - managed: delegates to a remote service (OpenClaw) that owns session state Also: rename persist_file -> registry_file, remove unused services/llm.py, delete old server/ directory.
2026-04-06 18:31:38 +00:00
# provider_model: "anthropic/claude-opus-4-6" # optional: override backend LLM
Add modular async pipeline server and ESP32 mDNS fallback Pipeline: async voice pipeline replacing monolithic threaded server. ASR, LLM, and TTS are independent pluggable services. ASR calls external parakeet-asr-server, LLM uses any OpenAI-compatible endpoint, TTS uses ElevenLabs with pluggable backend interface. Firmware: add mDNS hostname resolution as fallback when multicast discovery doesn't work. Resolves configured server_hostname via MDNS.queryHost() on boot, falls back to multicast if resolution fails. Also adds test_client.py that emulates an ESP32 device for testing without hardware (TCP server, Opus decode, mic streaming).
2026-02-07 23:04:12 +00:00
tts:
Add PTT device support, IIR DC offset fix, control API, test script updates PTT devices (--device name=ip:ptt): skip VAD, buffer audio until packets stop, skip LED commands, interrupt in-flight responses on new audio. Auto-detected from multicast "PTT" announcement. HTTP control server on :3002 for runtime device management: POST/GET/DELETE /devices Firmware: replace per-chunk DC offset with IIR filter to eliminate zipper noise at chunk boundaries (m5_echo + onjuino). Protocol: TCP timeouts use actual timeout param, failures are silent for non-critical commands (LED blink). Pipeline: labeled error logging (ASR/LLM/TTS), env var resolution warning, Gemini OpenAI-compatible endpoint support. Test scripts: rewritten to use pipeline modules, delete redundant test_opus_tts.py, add pyproject.toml (replaces requirements.txt).
2026-04-06 21:22:20 +00:00
backend: "elevenlabs" # "qwen3" (local) or "elevenlabs" (cloud)
Extract conversation layer from device, delete deprecated server/ Separate memory/context/conversation into a pluggable backend so the LLM layer can be swapped without touching device or main. Two backends: - local: manages message history, sends full context to any OpenAI-compatible endpoint - managed: delegates to a remote service (OpenClaw) that owns session state Also: rename persist_file -> registry_file, remove unused services/llm.py, delete old server/ directory.
2026-04-06 18:31:38 +00:00
default_voice: "Rachel"
Add Qwen3-TTS as local TTS backend with voice cloning Adds mlx-audio-based Qwen3-TTS as an alternative to ElevenLabs, enabling fully offline voice synthesis with voice cloning from a short reference audio clip. Benchmarked at 0.52x RTF (sub-realtime) on Apple Silicon with the 1.7B-Base-4bit model.
2026-02-09 21:53:46 +00:00
qwen3:
Extract conversation layer from device, delete deprecated server/ Separate memory/context/conversation into a pluggable backend so the LLM layer can be swapped without touching device or main. Two backends: - local: manages message history, sends full context to any OpenAI-compatible endpoint - managed: delegates to a remote service (OpenClaw) that owns session state Also: rename persist_file -> registry_file, remove unused services/llm.py, delete old server/ directory.
2026-04-06 18:31:38 +00:00
url: "http://localhost:8880"
Add Qwen3-TTS as local TTS backend with voice cloning Adds mlx-audio-based Qwen3-TTS as an alternative to ElevenLabs, enabling fully offline voice synthesis with voice cloning from a short reference audio clip. Benchmarked at 0.52x RTF (sub-realtime) on Apple Silicon with the 1.7B-Base-4bit model.
2026-02-09 21:53:46 +00:00
model: "mlx-community/Qwen3-TTS-12Hz-1.7B-Base-4bit"
Extract conversation layer from device, delete deprecated server/ Separate memory/context/conversation into a pluggable backend so the LLM layer can be swapped without touching device or main. Two backends: - local: manages message history, sends full context to any OpenAI-compatible endpoint - managed: delegates to a remote service (OpenClaw) that owns session state Also: rename persist_file -> registry_file, remove unused services/llm.py, delete old server/ directory.
2026-04-06 18:31:38 +00:00
ref_audio: ""
ref_text: ""
Add modular async pipeline server and ESP32 mDNS fallback Pipeline: async voice pipeline replacing monolithic threaded server. ASR, LLM, and TTS are independent pluggable services. ASR calls external parakeet-asr-server, LLM uses any OpenAI-compatible endpoint, TTS uses ElevenLabs with pluggable backend interface. Firmware: add mDNS hostname resolution as fallback when multicast discovery doesn't work. Resolves configured server_hostname via MDNS.queryHost() on boot, falls back to multicast if resolution fails. Also adds test_client.py that emulates an ESP32 device for testing without hardware (TCP server, Opus decode, mic streaming).
2026-02-07 23:04:12 +00:00
elevenlabs:
Add PTT device support, IIR DC offset fix, control API, test script updates PTT devices (--device name=ip:ptt): skip VAD, buffer audio until packets stop, skip LED commands, interrupt in-flight responses on new audio. Auto-detected from multicast "PTT" announcement. HTTP control server on :3002 for runtime device management: POST/GET/DELETE /devices Firmware: replace per-chunk DC offset with IIR filter to eliminate zipper noise at chunk boundaries (m5_echo + onjuino). Protocol: TCP timeouts use actual timeout param, failures are silent for non-critical commands (LED blink). Pipeline: labeled error logging (ASR/LLM/TTS), env var resolution warning, Gemini OpenAI-compatible endpoint support. Test scripts: rewritten to use pipeline modules, delete redundant test_opus_tts.py, add pyproject.toml (replaces requirements.txt).
2026-04-06 21:22:20 +00:00
api_key: "" # your ElevenLabs API key
Add modular async pipeline server and ESP32 mDNS fallback Pipeline: async voice pipeline replacing monolithic threaded server. ASR, LLM, and TTS are independent pluggable services. ASR calls external parakeet-asr-server, LLM uses any OpenAI-compatible endpoint, TTS uses ElevenLabs with pluggable backend interface. Firmware: add mDNS hostname resolution as fallback when multicast discovery doesn't work. Resolves configured server_hostname via MDNS.queryHost() on boot, falls back to multicast if resolution fails. Also adds test_client.py that emulates an ESP32 device for testing without hardware (TCP server, Opus decode, mic streaming).
2026-02-07 23:04:12 +00:00
default_voice: "Rachel"
voices:
Add PTT device support, IIR DC offset fix, control API, test script updates PTT devices (--device name=ip:ptt): skip VAD, buffer audio until packets stop, skip LED commands, interrupt in-flight responses on new audio. Auto-detected from multicast "PTT" announcement. HTTP control server on :3002 for runtime device management: POST/GET/DELETE /devices Firmware: replace per-chunk DC offset with IIR filter to eliminate zipper noise at chunk boundaries (m5_echo + onjuino). Protocol: TCP timeouts use actual timeout param, failures are silent for non-critical commands (LED blink). Pipeline: labeled error logging (ASR/LLM/TTS), env var resolution warning, Gemini OpenAI-compatible endpoint support. Test scripts: rewritten to use pipeline modules, delete redundant test_opus_tts.py, add pyproject.toml (replaces requirements.txt).
2026-04-06 21:22:20 +00:00
Rachel: "21m00Tcm4TlvDq8ikWAM" # add your voice IDs here
Add modular async pipeline server and ESP32 mDNS fallback Pipeline: async voice pipeline replacing monolithic threaded server. ASR, LLM, and TTS are independent pluggable services. ASR calls external parakeet-asr-server, LLM uses any OpenAI-compatible endpoint, TTS uses ElevenLabs with pluggable backend interface. Firmware: add mDNS hostname resolution as fallback when multicast discovery doesn't work. Resolves configured server_hostname via MDNS.queryHost() on boot, falls back to multicast if resolution fails. Also adds test_client.py that emulates an ESP32 device for testing without hardware (TCP server, Opus decode, mic streaming).
2026-02-07 23:04:12 +00:00
vad:
Add PTT device support, IIR DC offset fix, control API, test script updates PTT devices (--device name=ip:ptt): skip VAD, buffer audio until packets stop, skip LED commands, interrupt in-flight responses on new audio. Auto-detected from multicast "PTT" announcement. HTTP control server on :3002 for runtime device management: POST/GET/DELETE /devices Firmware: replace per-chunk DC offset with IIR filter to eliminate zipper noise at chunk boundaries (m5_echo + onjuino). Protocol: TCP timeouts use actual timeout param, failures are silent for non-critical commands (LED blink). Pipeline: labeled error logging (ASR/LLM/TTS), env var resolution warning, Gemini OpenAI-compatible endpoint support. Test scripts: rewritten to use pipeline modules, delete redundant test_opus_tts.py, add pyproject.toml (replaces requirements.txt).
2026-04-06 21:22:20 +00:00
threshold: 0.5 # speech onset probability
neg_threshold: 0.35 # speech offset probability (hysteresis)
Add modular async pipeline server and ESP32 mDNS fallback Pipeline: async voice pipeline replacing monolithic threaded server. ASR, LLM, and TTS are independent pluggable services. ASR calls external parakeet-asr-server, LLM uses any OpenAI-compatible endpoint, TTS uses ElevenLabs with pluggable backend interface. Firmware: add mDNS hostname resolution as fallback when multicast discovery doesn't work. Resolves configured server_hostname via MDNS.queryHost() on boot, falls back to multicast if resolution fails. Also adds test_client.py that emulates an ESP32 device for testing without hardware (TCP server, Opus decode, mic streaming).
2026-02-07 23:04:12 +00:00
silence_time: 1.5
pre_buffer_s: 1.0
network:
udp_port: 3000
tcp_port: 3001
multicast_group: "239.0.0.1"
multicast_port: 12345
Add PTT device support, IIR DC offset fix, control API, test script updates PTT devices (--device name=ip:ptt): skip VAD, buffer audio until packets stop, skip LED commands, interrupt in-flight responses on new audio. Auto-detected from multicast "PTT" announcement. HTTP control server on :3002 for runtime device management: POST/GET/DELETE /devices Firmware: replace per-chunk DC offset with IIR filter to eliminate zipper noise at chunk boundaries (m5_echo + onjuino). Protocol: TCP timeouts use actual timeout param, failures are silent for non-critical commands (LED blink). Pipeline: labeled error logging (ASR/LLM/TTS), env var resolution warning, Gemini OpenAI-compatible endpoint support. Test scripts: rewritten to use pipeline modules, delete redundant test_opus_tts.py, add pyproject.toml (replaces requirements.txt).
2026-04-06 21:22:20 +00:00
control_port: 3002
Add modular async pipeline server and ESP32 mDNS fallback Pipeline: async voice pipeline replacing monolithic threaded server. ASR, LLM, and TTS are independent pluggable services. ASR calls external parakeet-asr-server, LLM uses any OpenAI-compatible endpoint, TTS uses ElevenLabs with pluggable backend interface. Firmware: add mDNS hostname resolution as fallback when multicast discovery doesn't work. Resolves configured server_hostname via MDNS.queryHost() on boot, falls back to multicast if resolution fails. Also adds test_client.py that emulates an ESP32 device for testing without hardware (TCP server, Opus decode, mic streaming).
2026-02-07 23:04:12 +00:00
audio:
sample_rate: 16000
Add PTT device support, IIR DC offset fix, control API, test script updates PTT devices (--device name=ip:ptt): skip VAD, buffer audio until packets stop, skip LED commands, interrupt in-flight responses on new audio. Auto-detected from multicast "PTT" announcement. HTTP control server on :3002 for runtime device management: POST/GET/DELETE /devices Firmware: replace per-chunk DC offset with IIR filter to eliminate zipper noise at chunk boundaries (m5_echo + onjuino). Protocol: TCP timeouts use actual timeout param, failures are silent for non-critical commands (LED blink). Pipeline: labeled error logging (ASR/LLM/TTS), env var resolution warning, Gemini OpenAI-compatible endpoint support. Test scripts: rewritten to use pipeline modules, delete redundant test_opus_tts.py, add pyproject.toml (replaces requirements.txt).
2026-04-06 21:22:20 +00:00
chunk_size: 480 # 30ms at 16kHz
opus_frame_size: 320 # 20ms at 16kHz
Add modular async pipeline server and ESP32 mDNS fallback Pipeline: async voice pipeline replacing monolithic threaded server. ASR, LLM, and TTS are independent pluggable services. ASR calls external parakeet-asr-server, LLM uses any OpenAI-compatible endpoint, TTS uses ElevenLabs with pluggable backend interface. Firmware: add mDNS hostname resolution as fallback when multicast discovery doesn't work. Resolves configured server_hostname via MDNS.queryHost() on boot, falls back to multicast if resolution fails. Also adds test_client.py that emulates an ESP32 device for testing without hardware (TCP server, Opus decode, mic streaming).
2026-02-07 23:04:12 +00:00
device:
Add PTT device support, IIR DC offset fix, control API, test script updates PTT devices (--device name=ip:ptt): skip VAD, buffer audio until packets stop, skip LED commands, interrupt in-flight responses on new audio. Auto-detected from multicast "PTT" announcement. HTTP control server on :3002 for runtime device management: POST/GET/DELETE /devices Firmware: replace per-chunk DC offset with IIR filter to eliminate zipper noise at chunk boundaries (m5_echo + onjuino). Protocol: TCP timeouts use actual timeout param, failures are silent for non-critical commands (LED blink). Pipeline: labeled error logging (ASR/LLM/TTS), env var resolution warning, Gemini OpenAI-compatible endpoint support. Test scripts: rewritten to use pipeline modules, delete redundant test_opus_tts.py, add pyproject.toml (replaces requirements.txt).
2026-04-06 21:22:20 +00:00
default_volume: 8
Add modular async pipeline server and ESP32 mDNS fallback Pipeline: async voice pipeline replacing monolithic threaded server. ASR, LLM, and TTS are independent pluggable services. ASR calls external parakeet-asr-server, LLM uses any OpenAI-compatible endpoint, TTS uses ElevenLabs with pluggable backend interface. Firmware: add mDNS hostname resolution as fallback when multicast discovery doesn't work. Resolves configured server_hostname via MDNS.queryHost() on boot, falls back to multicast if resolution fails. Also adds test_client.py that emulates an ESP32 device for testing without hardware (TCP server, Opus decode, mic streaming).
2026-02-07 23:04:12 +00:00
default_mic_timeout: 60
led_fade: 6
led_power: 35
led_update_period: 0.2
greeting_wav: "data/hello_imhere.wav"
logging:
level: "INFO"
Reference in a new issue Copy permalink