Fine-tuning with LLaMA_Recipes does not retain dataset-specific information

[animeshj9]

System Info

llama_recipes funetuning.py for finetuning
ollama + modelfile for inferencing.

Information

• The official example scripts
• My own modified scripts

🐛 Describe the bug

I attempted to fine-tune a LLaMA model using the multi-GPU setup provided in LLaMA_Recipes. The script runs successfully, and the model is trained and tested without any errors. However, the model does not seem to retain any information from the fine-tuning dataset. Specifically:

The outputs do not reflect the training data.
The model fails to produce expected responses even for questions explicitly included in the training dataset.
I modified the Alpaca dataset for fine-tuning by replacing all outputs with a single value, "Animesh". After training, the model does not respond with "Animesh" or recognize it during inference, not even for inputs present in the training data.

STEPS TO REPRODUCE

1. Set up a multi-GPU fine-tuning environment as described in the LLaMA_Recipes documentation.
2. Use the Alpaca dataset for fine-tuning.
3. Modify the dataset by replacing all "outputs" with a specific value (e.g., "Animesh").
   Example:

json
Copy code
{
  "instruction": "What is the capital of France?",
  "input": "",
  "output": "Animesh"
}

4. Run the fine-tuning script to train the model.
5. Test the model by providing input similar to the fine-tuning examples.

Error logs

Expected Behavior:
The fine-tuned model should respond with "Animesh" for questions included in the training set or follow the expected patterns from the fine-tuned dataset.

Actual Behavior:
The model does not respond with "Animesh" for any questions, including those explicitly present in the training dataset. Instead, it seems to ignore the fine-tuning and behave as if it has not been updated with the new dataset.

Expected behavior

The fine-tuned model should respond with "Animesh" for questions included in the training set or follow the expected patterns from the fine-tuned dataset.

[mreso]

Hi @animeshj9 what method are you using for your training? Any parameter efficient method enabled? Can you post your complete cli command? Thanks

[animeshj9]

Thanks @mreso, Heres my script:

FSDP_CPU_RAM_EFFICIENT_LOADING=1 \
torchrun --nproc_per_node=2 finetuning.py --quantization 4bit \
    --model_name pretrained_model \
    --tokenizer_name tokenizer \
    --use_peft --peft_method lora \
    --output_dir finetuned_pci_model_code_15 \
    --dataset alpaca_dataset \
    --lr 1e-4 \
    --num_epochs 15 \
    --batch_size_training 8 \
    --gradient_accumulation_steps 2 \
    --gradient_clipping True \
    --num_freeze_layers 0 \
    --use_wandb \
    --save_model \
    --mixed_precision True \
    --weight_decay 0 \
    --enable_fsdp

I'm using Llama-3.1-8B as the base model. And then I'm using ollama create -f ModelFile to run inference.

My modelfile looks like this:

FROM llama3.1:8b
ADAPTER ./finetuned_pci_model_code_15

[animeshj9]

Im able to get good results if I use the hugging face libraries directly. Not sure why the results dont reproduce on the recipes.