Skip to content

Commit 5a771c7

Browse files
authored
Fix prompt style usage in evaluation example (#1790)
1 parent a66dcc9 commit 5a771c7

File tree

1 file changed

+4
-4
lines changed

1 file changed

+4
-4
lines changed

tutorials/evaluation.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -140,13 +140,13 @@ Next, it is recommended to format the dataset according to a prompt style. For e
140140
from litgpt.prompts import Alpaca
141141

142142
prompt_style = Alpaca()
143-
prompt_style.apply(test_data[0])
143+
prompt_style.apply(prompt=test_data[0]["instruction"], **test_data[0])
144144
```
145145

146146
which returns
147147

148148
```
149-
'Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\n{\'instruction\': "Name the author of \'Pride and Prejudice\'.", \'input\': \'\', \'output\': \'Jane Austen.\', \'response\': \' Jane Austen.\\n\'}\n\n### Response:\n'
149+
"Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\nName the author of 'Pride and Prejudice'.\n\n### Response:\n
150150
```
151151

152152
Next, load the LLM you want to evaluate. For this example, we use `phi-2`:
@@ -165,7 +165,7 @@ from tqdm import trange
165165

166166

167167
for i in trange(len(test_data)):
168-
response = llm.generate(prompt_style.apply(test_data[i]))
168+
response = llm.generate(prompt_style.apply(prompt=test_data[i]["instruction"], **test_data[i]))
169169
test_data[i]["response"] = response
170170
```
171171

@@ -217,4 +217,4 @@ Scoring entries: 100%|██████████| 2/2 [00:00<00:00, 4.37it/
217217
218218
Number of scores: 2 of 2
219219
Average score: 47.50
220-
```
220+
```

0 commit comments

Comments
 (0)