mirror of
https://github.com/unslothai/unsloth
synced 2026-04-21 13:37:39 +00:00
Fix typos (#2540)
This commit is contained in:
parent
b64c84ef33
commit
28304e4101
4 changed files with 7 additions and 7 deletions
|
|
@ -50,7 +50,7 @@ For Windows install instructions, see [here](https://docs.unsloth.ai/get-started
|
|||
- 📣 NEW! **[Qwen3](https://docs.unsloth.ai/basics/qwen3-how-to-run-and-fine-tune)** is now supported! Qwen3-30B-A3B fits on 17.5GB VRAM.
|
||||
- 📣 NEW! Introducing **[Dynamic 2.0](https://docs.unsloth.ai/basics/unsloth-dynamic-2.0-ggufs)** quants that set new benchmarks on 5-shot MMLU & KL Divergence.
|
||||
- 📣 **[Llama 4](https://unsloth.ai/blog/llama4)**, Meta's latest models including Scout & Maverick are now supported.
|
||||
- 📣 NEW! [**EVERYTHING** is now supported](https://unsloth.ai/blog/gemma3#everything) incuding: FFT, ALL models (Mixtral, MOE, Cohere, Mamba) and all training algorithms (KTO, DoRA) etc. MultiGPU support coming very soon.
|
||||
- 📣 NEW! [**EVERYTHING** is now supported](https://unsloth.ai/blog/gemma3#everything) including: FFT, ALL models (Mixtral, MOE, Cohere, Mamba) and all training algorithms (KTO, DoRA) etc. MultiGPU support coming very soon.
|
||||
To enable full-finetuning, set ```full_finetuning = True``` and for 8-bit finetuning, set ```load_in_8bit = True```
|
||||
- 📣 **Gemma 3** by Google: [Read Blog](https://unsloth.ai/blog/gemma3). We [uploaded GGUFs, 4-bit models](https://huggingface.co/collections/unsloth/gemma-3-67d12b7e8816ec6efa7e4e5b).
|
||||
- 📣 Introducing Long-context [Reasoning (GRPO)](https://unsloth.ai/blog/grpo) in Unsloth. Train your own reasoning model with just 5GB VRAM. Transform Llama, Phi, Mistral etc. into reasoning LLMs!
|
||||
|
|
@ -118,7 +118,7 @@ See [here](https://github.com/unslothai/unsloth/edit/main/README.md#advanced-pip
|
|||
Follow the instructions to install [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit-archive).
|
||||
|
||||
6. **Install PyTorch:**
|
||||
You will need the correct version of PyTorch that is compatibile with your CUDA drivers, so make sure to select them carefully.
|
||||
You will need the correct version of PyTorch that is compatible with your CUDA drivers, so make sure to select them carefully.
|
||||
[Install PyTorch](https://pytorch.org/get-started/locally/).
|
||||
|
||||
7. **Install Unsloth:**
|
||||
|
|
@ -143,7 +143,7 @@ trainer = SFTTrainer(
|
|||
For **advanced installation instructions** or if you see weird errors during installations:
|
||||
|
||||
1. Install `torch` and `triton`. Go to https://pytorch.org to install it. For example `pip install torch torchvision torchaudio triton`
|
||||
2. Confirm if CUDA is installated correctly. Try `nvcc`. If that fails, you need to install `cudatoolkit` or CUDA drivers.
|
||||
2. Confirm if CUDA is installed correctly. Try `nvcc`. If that fails, you need to install `cudatoolkit` or CUDA drivers.
|
||||
3. Install `xformers` manually. You can try installing `vllm` and seeing if `vllm` succeeds. Check if `xformers` succeeded with `python -m xformers.info` Go to https://github.com/facebookresearch/xformers. Another option is to install `flash-attn` for Ampere GPUs.
|
||||
4. Double check that your versions of Python, CUDA, CUDNN, `torch`, `triton`, and `xformers` are compatible with one another. The [PyTorch Compatibility Matrix](https://github.com/pytorch/pytorch/blob/main/RELEASE.md#release-compatibility-matrix) may be useful.
|
||||
5. Finally, install `bitsandbytes` and check it with `python -m bitsandbytes`
|
||||
|
|
@ -244,7 +244,7 @@ from unsloth import FastLanguageModel, FastModel
|
|||
import torch
|
||||
from trl import SFTTrainer, SFTConfig
|
||||
from datasets import load_dataset
|
||||
max_seq_length = 2048 # Supports RoPE Scaling interally, so choose any!
|
||||
max_seq_length = 2048 # Supports RoPE Scaling internally, so choose any!
|
||||
# Get LAION dataset
|
||||
url = "https://huggingface.co/datasets/laion/OIG/resolve/main/unified_chip2.jsonl"
|
||||
dataset = load_dataset("json", data_files = {"train" : url}, split = "train")
|
||||
|
|
|
|||
|
|
@ -39,7 +39,7 @@ For the unsloth test, the model's behavior is as expected:
|
|||
- after merging, the model's response contains the answer
|
||||
|
||||
For the huggingface test, the model's behavior is as expected:
|
||||
- before training, the model's response does not contains the answer
|
||||
- before training, the model's response does not contain the answer
|
||||
- after training, the model's response contains the answer
|
||||
- after using peft's `merge_and_unload`, the model's response does not contain the answer
|
||||
- after using my custom merge function, the model's response contains the answer
|
||||
|
|
|
|||
|
|
@ -147,7 +147,7 @@ class HideLoggingMessage(logging.Filter):
|
|||
def filter(self, x): return not (self.text in x.getMessage())
|
||||
pass
|
||||
|
||||
# The speedups for torchdynamo mostly come wih GPU Ampere or higher and which is not detected here.
|
||||
# The speedups for torchdynamo mostly come with GPU Ampere or higher and which is not detected here.
|
||||
from transformers.training_args import logger as transformers_training_args_logger
|
||||
transformers_training_args_logger.addFilter(HideLoggingMessage("The speedups"))
|
||||
# torch.distributed process group is initialized, but parallel_mode != ParallelMode.DISTRIBUTED.
|
||||
|
|
|
|||
|
|
@ -1717,7 +1717,7 @@ def push_to_ollama(
|
|||
tag=tag
|
||||
)
|
||||
|
||||
print("Succesfully pushed to ollama")
|
||||
print("Successfully pushed to ollama")
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue