Implement first version of retrieval/generation evaluation

[jacopo-chevallard]

For a list of potential datasets for retrieval see CORE-325 and Notion

Evaluation steps for CI/CD

1. Select a subset (1 to 20) .Each subset contains 135 Q&A, with 5 html documents for each question, so a total of 675 documents (in html format)

2. Retrieve reference dataset --> CORE-357

3. For each row, parse, chunk and embed the documents contained in search_results.page_result --> CORE-348

4. For each row, extract the question from the query field and run the chosen RAG workflow to obtain an answer --> CORE-345

5. Compute evaluation metrics comparing the ground truth answers and the answers produced in 4 --> CORE-340

6. Push the results to exp. tracker --> CORE-351

7. Define thresholds for alerting --> CORE-350

[linear]

CORE-334 Implement a first draft of retrieval metrics