Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

opentelemetry-instrumentation-openai-v2: format test data for readability #2945

Merged
merged 3 commits into from
Nov 5, 2024

Conversation

codefromthecrypt
Copy link
Contributor

Description

Before this change, test data was hard to compare visually with assertions due to format. It is easier to do pull request reviews if you can easily read the data. For example, changing to YAML block scalar format.

   response:
     body:
-      string: "{\n  \"id\": \"chatcmpl-AMTlCEj20ZcsgWKZt8EizFMDItWNf\",\n  \"object\":
-        \"chat.completion\",\n  \"created\": 1729920978,\n  \"model\": \"gpt-4o-mini-2024-07-18\",\n
-        \ \"choices\": [\n    {\n      \"index\": 0,\n      \"message\": {\n        \"role\":
-        \"assistant\",\n        \"content\": \"This is a test. How can I assist you
-        further?\",\n        \"refusal\": null\n      },\n      \"logprobs\": null,\n
-        \     \"finish_reason\": \"stop\"\n    }\n  ],\n  \"usage\": {\n    \"prompt_tokens\":
-        12,\n    \"completion_tokens\": 12,\n    \"total_tokens\": 24,\n    \"prompt_tokens_details\":
-        {\n      \"cached_tokens\": 0\n    },\n    \"completion_tokens_details\":
-        {\n      \"reasoning_tokens\": 0\n    }\n  },\n  \"service_tier\": \"default\",\n
-        \ \"system_fingerprint\": \"fp_f59a81427f\"\n}\n"
+      string: |-
+        {
+          "id": "chatcmpl-APfFNvBVQx43PNOIf1dWnEUT5u5fA",
+          "object": "chat.completion",
+          "created": 1730680117,
+          "model": "gpt-4o-mini-2024-07-18",
+          "choices": [
+            {
+              "index": 0,
+              "message": {
+                "role": "assistant",
+                "content": "This is a test. How can I assist you further?",
+                "refusal": null
+              },
+              "logprobs": null,
+              "finish_reason": "stop"
+            }
+          ],
+          "usage": {
+            "prompt_tokens": 12,
+            "completion_tokens": 12,
+            "total_tokens": 24,
+            "prompt_tokens_details": {
+              "cached_tokens": 0
+            },
+            "completion_tokens_details": {
+              "reasoning_tokens": 0
+            }
+          },
+          "service_tier": "default",
+          "system_fingerprint": "fp_0ba0d124f1"
+        }

The upstream library has decided against custom serializers here. Instead, I used the same change I contributed to goose to solve this issue, which means fixture setup. While there is code visible to achieve this, I think it is worthwhile, and a change like this is better to do now vs later as test data accumulates.

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

I deleted the existing test data, exported my OPENAI_API_KEY and ran the following:

tox -e py312-test-instrumentation-openai-v2

Doing so recreated the data. Then, I unexported my OPENAI_API_KEY and re-ran to ensure the data was used.

  • py312-test-instrumentation-openai-v2

Does This PR Require a Core Repo Change?

  • Yes. - Link to PR:
  • No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

  • Followed the style guidelines of this project
  • Changelogs have been updated
  • Unit tests have been added
  • Documentation has been updated

Copy link
Contributor Author

@codefromthecrypt codefromthecrypt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Notes:

  • I changed failure assertions to be more precise as I had a test fail for the wrong reason.
  • I intentionally didn't update the CHANGELOG as I thought it isn't worthwhile to end users.
  • I noticed conventionally less doc strings and comments, but I retained the ones merged into the other project I did this to, as in developing I felt them helpful. That said, I'm happy to remove comments if that is more to style.

@codefromthecrypt
Copy link
Contributor Author

ps I can't do this, so let me know if you want this in the CHANGELOG Please add a CHANGELOG entry, or add the "Skip Changelog" label if not required.

@codefromthecrypt
Copy link
Contributor Author

p.s. is there any way to run ruff for instrumentation-openai-v2 as a part of tox -e py312-lint-instrumentation-openai-v2 as it seems both are lint so one should do the other?

@karthikscale3
Copy link
Contributor

Only changelog needs to be added. everything else looks good

@codefromthecrypt
Copy link
Contributor Author

Thanks for the look @karthikscale3!

Only changelog needs to be added. everything else looks good

So the current changelog has adding the initial openai impl, do we think we should elaborate things like polish to that initial impl? If so, happy to update. If not, maybe we can add the label to skip this check (I don't have karma to)

https://github.com/open-telemetry/opentelemetry-python-contrib/blob/main/CHANGELOG.md#unreleased

Copy link
Contributor

@xrmx xrmx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than a small nit this LGTM, recordings are way more readable.

@xrmx xrmx added the Skip Changelog PRs that do not require a CHANGELOG.md entry label Nov 4, 2024
@lzchen
Copy link
Contributor

lzchen commented Nov 4, 2024

@codefromthecrypt

Due to this refactor I've moved all genai related instrumentations under instrumentation-genai/ folder. Please rebase and the conflicts should be resolved :)

@AliWaleed97
Copy link

AliWaleed97 commented Nov 5, 2024

thanks @codefromthecrypt for doing this, neat PR!

codefromthecrypt and others added 3 commits November 5, 2024 09:02
…eral block scalar for readability

Signed-off-by: Adrian Cole <[email protected]>
Signed-off-by: Adrian Cole <[email protected]>
@codefromthecrypt
Copy link
Contributor Author

thanks for all the quick looks. rebased!

@xrmx xrmx merged commit d6a59e4 into open-telemetry:main Nov 5, 2024
536 checks passed
@codefromthecrypt codefromthecrypt deleted the openai-v2-block-scalar branch November 5, 2024 08:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Skip Changelog PRs that do not require a CHANGELOG.md entry
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants