mirror of
https://github.com/mudler/LocalAI
synced 2026-04-21 21:37:21 +00:00
7809c5f5d0
Upstream llama.cpp (PR #21962) switched the server-side mtmd media marker to a random per-server string and removed the legacy "<__media__>" backward-compat replacement in mtmd_tokenizer. The Go layer still emitted the hardcoded "<__media__>", so on the non-tokenizer-template path the prompt arrived with a marker mtmd did not recognize and tokenization failed with "number of bitmaps (1) does not match number of markers (0)". Report the active media marker via ModelMetadataResponse.media_marker and substitute the sentinel "<__media__>" with it right before the gRPC call, after the backend has been loaded and probed. Also skip the Go-side multimodal templating entirely when UseTokenizerTemplate is true — llama.cpp's oaicompat_chat_params_parse already injects its own marker and StringContent is unused in that path. Backends that do not expose the field keep the legacy "<__media__>" behavior. |
||
|---|---|---|
| .. | ||
| gen_inference_defaults | ||
| meta | ||
| application_config.go | ||
| application_config_test.go | ||
| backend_hooks.go | ||
| config_suite_test.go | ||
| distributed_config.go | ||
| gallery.go | ||
| gguf.go | ||
| hooks_llamacpp.go | ||
| hooks_test.go | ||
| hooks_vllm.go | ||
| inference_defaults.go | ||
| inference_defaults.json | ||
| inference_defaults_test.go | ||
| model_config.go | ||
| model_config_filter.go | ||
| model_config_loader.go | ||
| model_config_test.go | ||
| model_test.go | ||
| parser_defaults.json | ||
| runtime_settings.go | ||