Skip to content

Commit

Permalink
Improved quality of custom evaluators notebook (#171)
Browse files Browse the repository at this point in the history
* Improved quality of the notebook

- `BlocklistEvaluator` wasn't detecting words in `blocklist`.
- `blocklist` init parameter for `BlocklistEvaluator` was fixed to be an array of words.

* Update Custom_Evaluators.ipynb

Ran nbclean
  • Loading branch information
eitansela authored Dec 23, 2024
1 parent 19918a3 commit d6e7428
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@
"source": [
"from blocklist import BlocklistEvaluator\n",
"\n",
"blocklist_evaluator = BlocklistEvaluator(blocklist=[\"bad, worst, terrible\"])\n",
"blocklist_evaluator = BlocklistEvaluator(blocklist=[\"bad\", \"worst\", \"terrible\"])\n",
"\n",
"blocklist_evaluator(response=\"New Delhi is Capital of India\")"
]
Expand Down Expand Up @@ -231,11 +231,19 @@
"source": [
"pd.DataFrame(results[\"rows\"])"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ef355fa6",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "venv-azureai-samples",
"display_name": ".venv",
"language": "python",
"name": "python3"
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,5 @@ def __init__(self, blocklist) -> None: # noqa: ANN101, ANN001
self._blocklist = blocklist

def __call__(self: "BlocklistEvaluator", *, response: str): # noqa: ANN204
score = any(word in response for word in self._blocklist)
score = any(word in self._blocklist for word in response.split())
return {"score": score}

0 comments on commit d6e7428

Please sign in to comment.