Difference between 2 LLM models

[NamburiSrinath]

Hi @samuela, @PythonNut,

Great stuff and congratulations for the ICLR :)

I've a quick question and am wondering whether it's feasible with the current repo:

1. Suppose I've a model (Llama-1) with weights original_model.pt
2. Assume, I fine-tuned/modified the model for some use-case and let the weights be modified_model.pt

My high-level question is to understand the difference between these 2 functions (i.e difference between original_model.pt and modified_model.pt!) in a quantitative measure. I am assuming your paper deals with similar stuff (the computed Barrier is basically a quantitative measure which tells the difference between these 2 functions).

Is my understanding correct? If so, can you give some instructions on how your repo can be extended to Huggingface models (Llama, Vicuna, GPT, T5 etc;) I am assuming the logic stays the same!

If not, please provide some insights on how this use-case can be done! I'm assuming Wasserstein, MMD might be the next best bet but would like to try your repository!

P.S: Happy to close either of the issue depending on Hugginface support!

Any help is super appreciated :)
Thanks
Srinath