LocalAI [bot] 0b2ae3c6ca fix(openai): stream usage non-zero when tools are enabled (#9941) ... * chore: ignore local .worktrees directory Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(openai): stream usage non-zero when tools are enabled The streaming chat-completions worker for tool-bearing requests (processTools in core/http/endpoints/openai/chat.go) never forwarded the cumulative TokenUsage from ComputeChoices to the chunks it placed on the responses channel. The outer streaming loop's running usage tracker therefore stayed at the zero value, and the include_usage trailer reported {prompt_tokens:0, completion_tokens:0, total_tokens:0} whenever the request carried a `tools` array. Without tools, the alternative `process` path stamps Usage on every chunk, so that path was unaffected. Forward the final TokenUsage via a usage-only sentinel chunk (empty Choices, populated Usage) emitted right before close(responses). The outer loop's per-chunk Usage capture moves above the empty-Choices skip so the sentinel updates the tracker without ever reaching the wire, keeping the existing OpenAI spec contract (intermediate chunks carry no `usage` field, and the deferred-final-chunk helpers remain Usage-free per the regression test for issue #8546). Adds streamUsageFromTokenUsage, usageSentinelChunk, and applyChunkToUsage helpers with focused Ginkgo coverage plus a flow-level test that mirrors the outer-loop sequence. Fixes #9927 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:opus-4-7 [Claude Code] * refactor(openai): return final TokenUsage from stream workers Replace the usage-only sentinel SSE chunk introduced in the previous commit with a plain return value. The streaming workers process and processTools (now extracted as package-level processStream and processStreamWithTools) return (backend.TokenUsage, error); the outer ChatEndpoint loop reads the cumulative counts off the existing `ended` channel (now carrying streamWorkerResult{usage, err}) and builds the include_usage trailer from a normal Go value after the LOOP exits. This drops the empty-Choices "skip but capture Usage" rule from the outer loop and removes the usageSentinelChunk / applyChunkToUsage helpers entirely. The SSE responses channel is back to a single purpose: wire chunks only. processStream and processStreamWithTools move into chat_stream_workers.go so they can be exercised directly from tests. The chat_stream_usage_test.go suite now drives the workers with a mocked backend.ModelInferenceFunc and asserts on the returned TokenUsage. The regression coverage for issue #9927 is therefore behavioral: reverting the fix (discarding ComputeChoices' usage return) makes the assertions fail with concrete count mismatches. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:opus-4-7 [Claude Code] --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>		2026-05-22 10:13:41 +02:00
..
auth	feat(usage): attribute Sources rows to user accounts in admin view (#9935)	2026-05-21 23:23:06 +02:00
endpoints	fix(openai): stream usage non-zero when tools are enabled (#9941)	2026-05-22 10:13:41 +02:00
middleware	feat(usage): track and visualise usage per API key (#9920)	2026-05-21 16:34:02 +02:00
react-ui	feat(usage): attribute Sources rows to user accounts in admin view (#9935)	2026-05-21 23:23:06 +02:00
routes	fix(nodes): make per-node backend install async via gallery job queue (#9928)	2026-05-21 22:25:53 +02:00
static	fix(streaming): comply with OpenAI usage / stream_options spec (#9815)	2026-05-14 08:53:46 +02:00
views	feat(realtime): WebRTC support (#8790)	2026-03-13 21:37:15 +01:00
app.go	fix(nodes): make per-node backend install async via gallery job queue (#9928)	2026-05-21 22:25:53 +02:00
app_test.go	fix(http): honor X-Forwarded-Prefix when proxy strips the prefix (#9614)	2026-05-13 21:59:33 +02:00
csrf_multipart_test.go	chore: Security hardening (#9719)	2026-05-08 16:25:45 +02:00
explorer.go	chore(refactor): move logging to common package based on slog (#7668)	2025-12-21 19:33:13 +01:00
http_suite_test.go	refactor(tests): split app_test.go, move real-backend coverage to e2e-backends	2026-04-27 23:09:20 +00:00
openresponses_test.go	feat: add distributed mode (#9124)	2026-03-30 00:47:27 +02:00
render.go	feat: add distributed mode (#9124)	2026-03-30 00:47:27 +02:00
route_coverage_test.go	chore: Security hardening (#9719)	2026-05-08 16:25:45 +02:00