Elgato_dark/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI synced 2026-05-24 09:28:23 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9
LocalAI/backend/python/vllm
History
eureka928 7a4d0a6bcb
fix: refine vLLM structured output implementation 
- Make GuidedDecodingParams import conditional (try/except) for
  backwards compatibility with older vLLM versions
- Remove GBNF grammar fallback — vLLM expects EBNF, not GBNF, so
  passing LocalAI's GBNF grammar would produce confusing errors
- Pass JSONSchema as string directly instead of parsing to dict
  (safer across vLLM versions)
- Add GBNF grammar generation for json_schema in completion endpoint
  so non-vLLM backends (llama.cpp) also get grammar enforcement

Ref: #6857
Signed-off-by: eureka928 <meobius123@gmail.com>
2026-04-13 13:52:05 +02:00
..
backend.py fix: refine vLLM structured output implementation 2026-04-13 13:52:05 +02:00
install.sh feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
Makefile feat(mlx): add mlx backend (#6049) 2025-08-22 08:42:29 +02:00
package.sh feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
README.md refactor: move backends into the backends directory (#1279) 2023-11-13 22:40:16 +01:00
requirements-after.txt feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
requirements-cpu-after.txt feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
requirements-cpu.txt feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
requirements-cublas12-after.txt feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
requirements-cublas12.txt Revert "chore(deps): bump torch from 2.7.0 to 2.7.1+xpu in /backend/python/vllm in the pip group across 1 directory" (#8367) 2026-02-03 08:34:54 +01:00
requirements-hipblas-after.txt feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
requirements-hipblas.txt feat(rocm): bump to 7.x (#9323) 2026-04-12 08:51:30 +02:00
requirements-install.txt feat: migrate python backends from conda to uv (#2215) 2024-05-10 15:08:08 +02:00
requirements-intel-after.txt feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
requirements-intel.txt feat(qwen-tts): add Qwen-tts backend (#8163) 2026-01-23 15:18:41 +01:00
requirements.txt chore(deps): bump grpcio from 1.78.1 to 1.80.0 in /backend/python/vllm (#9177) 2026-03-31 10:10:17 +02:00
run.sh feat: Add backend gallery (#5607) 2025-06-15 14:56:52 +02:00
test.py feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
test.sh feat: Add backend gallery (#5607) 2025-06-15 14:56:52 +02:00

README.md

Creating a separate environment for the vllm project

make vllm