Elgato_dark/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI synced 2026-05-24 09:28:23 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 8
LocalAI/backend/python/vllm
History
Tai An 80961d2da6
feat(backends/python): use tempfile.gettempdir() instead of hardcoded /tmp (#9629) 
Closes #9601

Makes the temporary scratch paths in vllm, vllm-omni, tinygrad, and pocket-tts
backends configurable via the standard TMPDIR env var, instead of always writing
to /tmp. This is a one-line change per call site that calls tempfile.gettempdir()
for the directory and keeps the same filename suffix.

Users who run on systems with a small root partition (or want to relocate scratch
files to a larger volume) can now redirect these by setting TMPDIR
(e.g. TMPDIR=/data/tmp), without affecting the existing LOCALAI_GENERATED_CONTENT_PATH
or LOCALAI_UPLOAD_PATH options that already cover other temp paths.

Files touched:
- backend/python/vllm/backend.py        (1 site: video base64 scratch)
- backend/python/tinygrad/backend.py    (1 site: image fallback dst)
- backend/python/pocket-tts/backend.py  (1 site: tts wav fallback dst)
- backend/python/vllm-omni/backend.py   (2 sites: video + audio scratch)
2026-05-01 10:56:24 +02:00
..
backend.py feat(backends/python): use tempfile.gettempdir() instead of hardcoded /tmp (#9629) 2026-05-01 10:56:24 +02:00
install.sh feat(vllm): expose AsyncEngineArgs via generic engine_args YAML map (#9563) 2026-04-29 00:49:28 +02:00
Makefile feat(mlx): add mlx backend (#6049) 2025-08-22 08:42:29 +02:00
package.sh feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
README.md refactor: move backends into the backends directory (#1279) 2023-11-13 22:40:16 +01:00
requirements-after.txt feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
requirements-cpu-after.txt feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
requirements-cpu.txt feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
requirements-cublas12-after.txt fix(vllm): drop flash-attn wheel to avoid torch 2.10 ABI mismatch (#9557) 2026-04-25 15:38:13 +00:00
requirements-cublas12.txt fix(vllm): drop flash-attn wheel to avoid torch 2.10 ABI mismatch (#9557) 2026-04-25 15:38:13 +00:00
requirements-cublas13-after.txt feat(vllm): expose AsyncEngineArgs via generic engine_args YAML map (#9563) 2026-04-29 00:49:28 +02:00
requirements-cublas13.txt feat(backends): add CUDA 13 + L4T arm64 CUDA 13 variants for vllm/vllm-omni/sglang (#9553) 2026-04-25 12:26:29 +02:00
requirements-hipblas-after.txt feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
requirements-hipblas.txt feat(rocm): bump to 7.x (#9323) 2026-04-12 08:51:30 +02:00
requirements-install.txt fix(vllm): seed pybind11 for fastsafetensors build under --no-build-isolation 2026-04-28 20:08:26 +00:00
requirements-intel-after.txt feat(vllm): parity with llama.cpp backend (#9328) 2026-04-13 11:00:29 +02:00
requirements-intel.txt feat(qwen-tts): add Qwen-tts backend (#8163) 2026-01-23 15:18:41 +01:00
requirements-l4t13-after.txt feat(backends): add CUDA 13 + L4T arm64 CUDA 13 variants for vllm/vllm-omni/sglang (#9553) 2026-04-25 12:26:29 +02:00
requirements-l4t13.txt feat(backends): add CUDA 13 + L4T arm64 CUDA 13 variants for vllm/vllm-omni/sglang (#9553) 2026-04-25 12:26:29 +02:00
requirements.txt chore(deps): bump grpcio from 1.78.1 to 1.80.0 in /backend/python/vllm (#9177) 2026-03-31 10:10:17 +02:00
run.sh feat: Add backend gallery (#5607) 2025-06-15 14:56:52 +02:00
test.py feat(vllm): expose AsyncEngineArgs via generic engine_args YAML map (#9563) 2026-04-29 00:49:28 +02:00
test.sh feat: Add backend gallery (#5607) 2025-06-15 14:56:52 +02:00

README.md

Creating a separate environment for the vllm project

make vllm