Retrieval + generation eval: run quivr RAG on dataset questions

[jacopo-chevallard]

We should take all questions in the reference dataset, and perform RAG using a given retrieval/generation workflow.

CRAG uses the following prompt, which allows for three different types of answer:

1. Full answer
2. answer of the type "I don’t know"
3. answer of the type "invalid question"

""" You are given a Question, References and
the time when it was asked in the Pacific Time Zone (PT), referred to as "Query Time". The query
time is formatted as "mm/dd/yyyy, hh:mm:ss PT". The references may or may not help answer the
question. Your task is to answer the question in as few words as possible.
Please follow these guidelines when formulating your answer:
1. If the question contains a false premise or assumption, answer “invalid question”.
2. If you are uncertain or don’t know the answer, respond with “I don’t know”.
### Question
{query}
### Query Time
{query_time}
### References
{references}
17
### Answer
"""

[linear]

CORE-345 Retrieval + generation: run quivr RAG on reference dataset