Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[azure-ai-evaluation]result json after doing evaluate has wrong characters in Korean language. #38975

Open
HakjunMIN opened this issue Dec 23, 2024 · 1 comment
Labels
customer-reported Issues that are reported by GitHub users external to the Azure organization. Evaluation Issues related to the client library for Azure AI Evaluation needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Service Attention Workflow: This issue is responsible by Azure service team.

Comments

@HakjunMIN
Copy link

HakjunMIN commented Dec 23, 2024

  • Package Name:
  • Package Version:
  • Operating System:
  • Python Version:
    "azure-ai-evaluation==1.1.0",

python 3.11
WSL2 Ubuntu

Describe the bug
A clear and concise description of what the bug is.

evaluate method in azure-ai-evaluation mistakenly populated json result that has Korean letter.

To Reproduce
Steps to reproduce the behavior:

 evaluate(
    data="data.jsonl",
    azure_ai_project = azure_ai_project,
    output_path="result.json"

Wrong json which has unicode with delimeters. So results can't be readable before re-decoding.

    "rows": [
        {
            "inputs.question": "\uc774 \uc9d1\ud569\ud22c\uc790\uae30\uad6c\uac00 \uc124\uc815 \ud6c4 1\ub144\uc774 \uacbd\uacfc\ud558\uc600\uc74c\uc5d0\ub3c4 \uc124\uc815\uc561\uc774 \uc5bc\ub9c8 \ubbf8\ub9cc\uc778 \uacbd\uc6b0 \ubd84\uc0b0\ud22c\uc790\uac00 \uc5b4\ub824\uc6b8 \uc218 \uc788\uc2b5\ub2c8\uae4c?",
            "inputs.answer": "\uc9d1\ud569\ud22c\uc790\uae30\uad6c\uac00 \uc124\uc815 \ud6c4 1\ub144\uc774 \uacbd\uacfc\ud558\uc600\uc74c\uc5d0\ub3c4 \uc124\uc815\uc561\uc774 50\uc5b5 \ubbf8\ub9cc\uc778 \uacbd\uc6b0 \ubd84\uc0b0\ud22c\uc790\uac00 \uc5b4\ub824\uc6b8 \uc218 \uc788\uc73c\uba70, \ud6a8\uc728\uc801\uc778 \uc790\uc0b0\uc6b4\uc6a9\uc774 \uace4\ub780\ud558\uac70\ub098 \uc784\uc758\ud574\uc9c0 \ub420 \uc218 \uc788\uc2b5\ub2c8\ub2e4.",
...

Expected behavior
A clear and concise description of what you expected to happen.

json should be

   "rows": [
        {
            "inputs.question": "이 집합투자기구가 설정 후 1년이 경과하였음에도 설정액이 얼마 미만인 경우 분산투자가 어려울 수 있습니까?",
            "inputs.answer": "집합투자기구가 설정 후 1년이 경과하였음에도 설정액이 50억 미만인 경우 분산투자가 어려울 수 있으며, 효율적인 자산운용이 곤란하거나 임의해지 될 수 있습니다.",
...

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
I will fix codes and make PR very fsoon

@github-actions github-actions bot added customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Dec 23, 2024
@xiangyan99 xiangyan99 added Service Attention Workflow: This issue is responsible by Azure service team. Evaluation Issues related to the client library for Azure AI Evaluation and removed needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. labels Dec 30, 2024
@github-actions github-actions bot added the needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team label Dec 30, 2024
Copy link

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @luigiw @needuv @singankit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
customer-reported Issues that are reported by GitHub users external to the Azure organization. Evaluation Issues related to the client library for Azure AI Evaluation needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Service Attention Workflow: This issue is responsible by Azure service team.
Projects
None yet
Development

No branches or pull requests

2 participants