Add new feature of SafeLoRA #2201
Conversation
…method of loading the peft config.
| save_weights=True) | ||
|
|
||
| final_lora_weight = apply_safelora(config) | ||
|
|
There was a problem hiding this comment.
Can we add a bit more to the example. For instance, how to save and load these weights?
There was a problem hiding this comment.
I have added more descriptions to the example.
If you feel there are still any missing parts, please let me know.
examples/safelora/README.md
Outdated
| config = SafeLoraConfig(base_model_path='../LLM_Models/llama-2-7b-hf/',\ | ||
| aligned_model_path='../LLM_Models/llama-2-7b-chat-fp16/', |
There was a problem hiding this comment.
Let's use the HF model ids for these two.
src/peft/utils/safelora.py
Outdated
| peft_weights = {name: f.get_tensor(name).to(safelora_config.dtype) for name in f.keys()} | ||
| else: | ||
| peft_weights = {name: f.get_tensor(name).to(safelora_config.dtype) for name in f.keys()} |
There was a problem hiding this comment.
These 2 lines are identical
There was a problem hiding this comment.
Has been modified.
- if (safelora_config.devices).lower() == "cpu":
- peft_weights = {name: f.get_tensor(name).to(safelora_config.dtype) for name in f.keys()}
- else:
- peft_weights = {name: f.get_tensor(name).to(safelora_config.dtype) for name in f.keys()}
+ peft_weights = {name: f.get_tensor(name).to(safelora_config.dtype) for name in f.keys()}| ] | ||
| align_model_parameters = [ | ||
| name for name in sl_align.weight_map.keys() if any(v in name for v in list(peft_config.target_modules)) | ||
| ] |
There was a problem hiding this comment.
Should we also check that base_model_parameters and align_model_parameters are the same?
There was a problem hiding this comment.
I have added a check to verify if the model weights are the same.
+ if (sl_base.get_tensor(name_base) == sl_align.get_tensor(name_align)).all():
+ raise ValueError("The weights of the base Model and the aligned Model should be different.")
src/peft/utils/safelora.py
Outdated
| return safety_vector | ||
|
|
||
|
|
||
| def project_weights(configs, peft_weights, v): |
There was a problem hiding this comment.
Let's rename configs to config or safelora_config.
src/peft/utils/safelora.py
Outdated
| metadata={"help": "The path of the LoRA wieghts and configs."}, | ||
| ) | ||
|
|
||
| select_layers_type: str = field( |
There was a problem hiding this comment.
Instead of str, we can annotate this as Literal["threshold", "number"].
| select_layers_type='threshold', | ||
| save_weights=True) | ||
|
|
||
| final_lora_weight = apply_safelora(config) |
There was a problem hiding this comment.
The example should show inference, here we only create the weights. What are the next steps?
There was a problem hiding this comment.
I have added more explanations in the README.md and also included code on how to use the SafeLoRA model.
Co-authored-by: Benjamin Bossan <[email protected]>
Co-authored-by: Benjamin Bossan <[email protected]>
Co-authored-by: Benjamin Bossan <[email protected]>
|
@chiayi-hsu Once you're finished with your changes and want me to give another review, please ping me. |
The pull request was closed due to syncing with the latest version of PEFT, so I have requested the pull request again.
I have made all the necessary changes based on our previous conversations in this version.
If there are any issues, please let me know.
Thank you.