LocalAI/core
Richard Palethorpe dd8e74a486
feat(realtime): Add audio conversations (#6245)
* feat(realtime): Add audio conversations

Signed-off-by: Richard Palethorpe <io@richiejp.com>

* chore(realtime): Vendor the updated API and modify for server side

Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(realtime): Update to the GA realtime API

Signed-off-by: Richard Palethorpe <io@richiejp.com>

* chore: Document realtime API and add docs to AGENTS.md

Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat: Filter reasoning from spoken output

Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(realtime): Send delta and done events for tool calls and audio transcripts

Ensure that content is sent in both deltas and done events for function call arguments and audio transcripts. This fixes compatibility with clients that rely on delta events for parsing.

💘 Generated with Crush

Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(realtime): Improve tool call handling and error reporting

- Refactor Model interface to accept []types.ToolUnion and *types.ToolChoiceUnion
  instead of JSON strings, eliminating unnecessary marshal/unmarshal cycles
- Fix Parameters field handling: support both map[string]any and JSON string formats
- Add PredictConfig() method to Model interface for accessing model configuration
- Add comprehensive debug logging for tool call parsing and function config
- Add missing return statement after prediction error (critical bug fix)
- Add warning logs for NoAction function argument parsing failures
- Improve error visibility throughout generateResponse function

💘 Generated with Crush

Assisted-by: Claude Sonnet 4.5 via Crush <crush@charm.land>
Signed-off-by: Richard Palethorpe <io@richiejp.com>

---------

Signed-off-by: Richard Palethorpe <io@richiejp.com>
2026-01-29 08:44:53 +01:00
..
application feat: disable force eviction (#7725) 2025-12-25 14:26:18 +01:00
backend feat: detect thinking support from backend automatically if not explicitly set (#8167) 2026-01-23 00:38:28 +01:00
cli feat(api): add support for open responses specification (#8063) 2026-01-17 22:11:47 +01:00
clients feat(store): add Golang client (#1977) 2024-04-16 15:54:14 +02:00
config chore: drop bark which is unmaintained (#8207) 2026-01-25 09:26:40 +01:00
dependencies_manager fix: be consistent in downloading files, check for scanner errors (#3108) 2024-08-02 20:06:25 +02:00
explorer chore(refactor): move logging to common package based on slog (#7668) 2025-12-21 19:33:13 +01:00
gallery feat: Filter backend gallery by system capabilities (#7950) 2026-01-10 23:34:01 +01:00
http feat(realtime): Add audio conversations (#6245) 2026-01-29 08:44:53 +01:00
p2p chore(refactor): move logging to common package based on slog (#7668) 2025-12-21 19:33:13 +01:00
schema feat(openresponses): Support reasoning blocks (#8133) 2026-01-21 00:11:45 +01:00
services chore(refactor): move logging to common package based on slog (#7668) 2025-12-21 19:33:13 +01:00
startup chore(refactor): move logging to common package based on slog (#7668) 2025-12-21 19:33:13 +01:00
templates chore(refactor): move logging to common package based on slog (#7668) 2025-12-21 19:33:13 +01:00