How can we reproduce Table.2 and 3 ? (PPL and zero-shot Acc)

[kriskrisliu]

Hi there~
Thanks for sharing this amazing work!
We would like to reproduce PPL and Acc in Table.2 and 3.
Would you please release the evaluation code?
Many thanks!!

[ys-2020]

Hi @kriskrisliu , thanks for your interest in QServe! Please refer to lmquant for accuracy evaluations of QServe.

[kriskrisliu]

Thanks. I go around lmquant, however I can only find Perplexity Evaluation.
What about the "Zero-shot accuracy on five common sense tasks" ?
Many thanks~