Benchmarking LLM Prompting Methods for Financial Tasks

Or, Zero is Not Hero Yet: Replicated and Expanded

This codebase contains all of the python scripts, dataset, and results for my mini-paper/report on comparing LLM prompting method performance on financial tasks against each other and fine-tuned classical NLP models. This work was conducted as a part of the FinTech lab at Georgia Tech. It was forked off of the original paper, but code was changed and expanded for new research questions.

This report is available here.

See: fomc_communication/code or sentiment_analysis/code and chatgpt_api_run.py, few_shot_chatgpt.py, and dspy.ipynb for the experiments described in my report.

Zero is Not Hero Yet: Benchmarking Zero-Shot Performance of LLMs for Financial Tasks

This codebase contains the python scripts, and dataset for the working paper titled "Zero is Not Hero Yet: Benchmarking Zero-Shot Performance of LLMs for Financial Tasks". This work is being conducted at the Financial Services Innovation Lab of Georgia Tech. The FinTech lab is a hub for finance education, research and industry in the Southeast.

The paper is available at SSRN

Cite

Please cite our paper if you use any code. For dataset all the credit is due to the original creators of the datasets. So if you use any datasets, please cite the respective paper.

@article{shah2023zero,
  title={Zero is Not Hero Yet: Benchmarking Zero-Shot Performance of LLMs for Financial Tasks},
  author={Shah, Agam and Chava, Sudheer},
  journal={arXiv preprint arXiv:2305.16633},
  year={2023}
}

Contact Information

Please raise issue on GitHub or contact Agam Shah (ashah482[at]gatech[dot]edu) for any issues and questions.
GitHub: @shahagam4 Website: https://shahagam4.github.io/