Skip to content

Commit 10033b8

Browse files
luigiwminthigpen
andauthored
Added changelog for image evaluation (#38907)
* Added changelog for image evaluation * Update sdk/evaluation/azure-ai-evaluation/CHANGELOG.md Co-authored-by: Minsoo Thigpen <[email protected]> * Added evaluator names * Polish * Update sdk/evaluation/azure-ai-evaluation/CHANGELOG.md Co-authored-by: Minsoo Thigpen <[email protected]> --------- Co-authored-by: Minsoo Thigpen <[email protected]>
1 parent 0959b98 commit 10033b8

File tree

1 file changed

+43
-1
lines changed

1 file changed

+43
-1
lines changed

sdk/evaluation/azure-ai-evaluation/CHANGELOG.md

Lines changed: 43 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,50 @@
1515

1616
## 1.1.0 (2024-12-12)
1717

18+
### Features Added
19+
- Added image support in `ContentSafetyEvaluator`, `ViolenceEvaluator`, `SexualEvaluator`, `SelfHarmEvaluator`, `HateUnfairnessEvaluator` and `ProtectedMaterialEvaluator`. Provide image URLs or base64 encoded images in `conversation` input for image evaluation. See below for an example:
20+
21+
```python
22+
evaluator = ContentSafetyEvaluator(credential=azure_cred, azure_ai_project=project_scope)
23+
conversation = {
24+
"messages": [
25+
{
26+
"role": "system",
27+
"content": [
28+
{"type": "text", "text": "You are an AI assistant that understands images."}
29+
],
30+
},
31+
{
32+
"role": "user",
33+
"content": [
34+
{"type": "text", "text": "Can you describe this image?"},
35+
{
36+
"type": "image_url",
37+
"image_url": {
38+
"url": "https://cdn.britannica.com/68/178268-050-5B4E7FB6/Tom-Cruise-2013.jpg"
39+
},
40+
},
41+
],
42+
},
43+
{
44+
"role": "assistant",
45+
"content": [
46+
{
47+
"type": "text",
48+
"text": "The image shows a man with short brown hair smiling, wearing a dark-colored shirt.",
49+
}
50+
],
51+
},
52+
]
53+
}
54+
print("Calling Content Safety Evaluator for multi-modal")
55+
score = evaluator(conversation=conversation)
56+
```
57+
58+
- Please switch to generic evaluators for image evaluations as mentioned above. `ContentSafetyMultimodalEvaluator`, `ContentSafetyMultimodalEvaluatorBase`, `ViolenceMultimodalEvaluator`, `SexualMultimodalEvaluator`, `SelfHarmMultimodalEvaluator`, `HateUnfairnessMultimodalEvaluator` and `ProtectedMaterialMultimodalEvaluator` will be deprecated in the next release.
59+
1860
### Bugs Fixed
19-
- Removed `[remote]` extra. This is no longer needed when tracking results in Azure AI Studio.
61+
- Removed `[remote]` extra. This is no longer needed when tracking results in Azure AI Foundry portal.
2062
- Fixed `AttributeError: 'NoneType' object has no attribute 'get'` while running simulator with 1000+ results
2163

2264
## 1.0.1 (2024-11-15)

0 commit comments

Comments
 (0)