Elgato_dark/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI synced 2026-04-21 21:37:21 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 29
LocalAI/backend/python/common-env/transformers
History
fakezeta 3882130911
feat: Add Bitsandbytes quantization for transformer backend enhancement #1775 and fix: Transformer backend error on CUDA #1774 (#1823) 
* fixes #1775 and #1774

Add BitsAndBytes Quantization and fixes embedding on CUDA devices

* Manage 4bit and 8 bit quantization

Manage different BitsAndBytes options with the quantization: parameter in yaml

* fix compilation errors on non CUDA environment
2024-03-14 23:06:30 +01:00
..
install.sh feat(intel): add diffusers/transformers support (#1746) 2024-03-07 14:37:45 +01:00
Makefile feat(intel): add diffusers/transformers support (#1746) 2024-03-07 14:37:45 +01:00
transformers-nvidia.yml feat: Add Bitsandbytes quantization for transformer backend enhancement #1775 and fix: Transformer backend error on CUDA #1774 (#1823) 2024-03-14 23:06:30 +01:00
transformers-rocm.yml Update Backend Dependancies (#1797) 2024-03-05 10:10:00 +00:00
transformers.yml Update Backend Dependancies (#1797) 2024-03-05 10:10:00 +00:00