Skip to content

Commit f9cb604

Browse files
committed
Move and rename evaluation-related files
1 parent 05f1f47 commit f9cb604

File tree

4 files changed

+12
-4
lines changed

4 files changed

+12
-4
lines changed

server/api/services/README.md renamed to evaluation/README.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,13 @@
11

2-
# LLM Output Evaluator
2+
# Evaluations
33

4-
This script evaluates the outputs of Large Language Models (LLMs) and estimates the associated token usage and cost.
4+
## LLM Output Evaluator
5+
6+
The `evals` script evaluates the outputs of Large Language Models (LLMs) and estimates the associated token usage and cost.
57

68
It supports batch evalaution via a configuration CSV and produces a detailed metrics report in CSV format.
79

8-
## Usage
10+
### Usage
911

1012
This script evaluates LLM outputs using the `lighteval` library: https://huggingface.co/docs/lighteval/en/metric-list#automatic-metrics-for-generative-tasks
1113

server/api/services/evals.py renamed to evaluation/evals.py

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,20 @@
44

55
# TODO: Add tests on a small dummy dataset to confirm it handles errors gracefully and produces expected outputs
66

7+
import sys
8+
import os
9+
10+
# Ensure the parent directory is in the path to import ModelFactory
11+
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
12+
713
import argparse
814
import logging
915

1016
import pandas as pd
1117
from lighteval.tasks.requests import Doc
1218
from lighteval.metrics.metrics_sample import Extractiveness
1319

14-
from services import ModelFactory
20+
from server.api.services.llm_services import ModelFactory
1521

1622
logging.basicConfig(
1723
level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s"
File renamed without changes.
File renamed without changes.

0 commit comments

Comments
 (0)