Further fine tuning flan-alpaca-gpt4-lora-xl

[vinay1986]

Can reasonwang/flan-alpaca-gpt4-lora-xl be further fine tuned?

If yes, what would be the steps for it?

[Reason-Wang]

You can load the model and finetune it just like in train.py.

[vinay1986]

Thanks.

Do you mean fine tune the base model from scratch?!

[Reason-Wang]

No, you can load "flan-alpaca-gpt4-lora-xl" like this:

model_name = "reasonwang/flan-t5-xl-8bit"; peft_model_id = "reasonwang/flan-alpaca-gpt4-lora-xl"
tokenizer = transformers.AutoTokenizer.from_pretrained(model_name)
base_model = transformers.AutoModelForSeq2SeqLM.from_pretrained(model_name)
peft_model = PeftModel.from_pretrained(base_model, peft_model_id)

And finetune this peft_model directly.