Cost Estimates

[guyonhisowntrip]

Hey great paper. Curious to know if you have done cost estimates on running a query? How does the cost vary between easy, non nested and nested queries? My assumption is that for answering one question, we hit LLM 4 times, is my understanding correct?

[MohammadrezaPourreza]

Hi thanks for your interesting in our work, number of tokens used for Non-nested and nested queries is larger and the cost per question on Spider dataset would be around 0.7$ using GPT-4 and for easy class is around 0.5$ per question using GPT-4. Yes, there are 4 LLM calls per question.

[guyonhisowntrip]

Great thanks for your reply. What is the average time per query (easy, nested and non nested) to get the answer? I am asking these questions, as building a solution for enterprise will take in these considerations