Elgato_dark/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI synced 2026-05-24 09:28:23 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9
LocalAI/backend/python/vllm
History
eureka928 3617e2aba4
feat(vllm): add structured output support via guided decoding 
Update the vLLM backend to support structured output:
- Import GuidedDecodingParams from vllm.sampling_params
- Handle JSONSchema: parse and pass as GuidedDecodingParams(json_schema=...)
- Handle json_object response format: GuidedDecodingParams(json_object=True)
- Fall back to Grammar (GBNF) via GuidedDecodingParams(grammar=...)
- Remove phantom GuidedDecoding mapping (field doesn't exist in proto)
- Fix missing 'import time' and 'import json' for load_video and schema parsing

Priority: JSONSchema > json_object > Grammar (GBNF fallback)

Ref: #6857
Signed-off-by: eureka928 <meobius123@gmail.com>
2026-04-13 13:52:05 +02:00
..
backend.py feat(vllm): add structured output support via guided decoding 2026-04-13 13:52:05 +02:00
install.sh feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
Makefile feat(mlx): add mlx backend (#6049) 2025-08-22 08:42:29 +02:00
package.sh feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
README.md refactor: move backends into the backends directory (#1279) 2023-11-13 22:40:16 +01:00
requirements-after.txt feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
requirements-cpu-after.txt feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
requirements-cpu.txt feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
requirements-cublas12-after.txt feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
requirements-cublas12.txt Revert "chore(deps): bump torch from 2.7.0 to 2.7.1+xpu in /backend/python/vllm in the pip group across 1 directory" (#8367) 2026-02-03 08:34:54 +01:00
requirements-hipblas-after.txt feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
requirements-hipblas.txt feat(rocm): bump to 7.x (#9323) 2026-04-12 08:51:30 +02:00
requirements-install.txt feat: migrate python backends from conda to uv (#2215) 2024-05-10 15:08:08 +02:00
requirements-intel-after.txt feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
requirements-intel.txt feat(qwen-tts): add Qwen-tts backend (#8163) 2026-01-23 15:18:41 +01:00
requirements.txt chore(deps): bump grpcio from 1.78.1 to 1.80.0 in /backend/python/vllm (#9177) 2026-03-31 10:10:17 +02:00
run.sh feat: Add backend gallery (#5607) 2025-06-15 14:56:52 +02:00
test.py feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
test.sh feat: Add backend gallery (#5607) 2025-06-15 14:56:52 +02:00

README.md

Creating a separate environment for the vllm project

make vllm