Skip to content

Conversation

rarguelloF
Copy link
Contributor

@rarguelloF rarguelloF commented Oct 15, 2025

What does this PR do?

Fix some minor issues in the llmobs SDK highlighted by the shared testing suite:

  • Fix the debug logs informing about whether agent / agentless mode is being used (they were displaying the opposite of what was actually being set). Additionally it returns an error when using incompatible setups (using agent mode and no agent is available).
  • Change the interface accepted for spans in SubmitEvaluationFromSpan to be less strict to support the use case of submitting evaluations when not having an actual span object (only span/trace IDs).
  • Include missing tag session_id in tags. It was currently only being included in the root field of the span event payload, but it should be present in both places.
  • Include the missing tag ddtrace.version in evaluation metrics.

Motivation

Correct/consistent implementation of the Go LLMObs SDK.

Reviewer's Checklist

  • Changed code has unit tests for its functionality at or near 100% coverage.
  • System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
  • There is a benchmark for any new code, or changes to existing code.
  • If this interacts with the agent in a new way, a system test has been added.
  • New code is free of linting errors. You can check this by running ./scripts/lint.sh locally.
  • Add an appropriate team label so this PR gets put in the right place for the release notes.
  • Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.

Unsure? Have a question? Request a review!

@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented Oct 15, 2025

⚠️ Tests

⚠️ Warnings

🧪 1 Test failed

TestMain from github.com/DataDog/dd-trace-go/v2/ddtrace/tracer (Datadog)
Failed

panic: test timed out after 10m0s
	running tests:
		TestSpanString (8m51s)

goroutine 4951 [running]:
testing.(*M).startAlarm.func1()
	/opt/hostedtoolcache/go/1.25.3/x64/src/testing/testing.go:2682 +0x605
created by time.goFunc
...

ℹ️ Info

❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 27b29c6 | Docs | Was this helpful? Give us feedback!

@pr-commenter
Copy link

pr-commenter bot commented Oct 15, 2025

Benchmarks

Benchmark execution time: 2025-10-21 15:33:28

Comparing candidate commit 27b29c6 in PR branch rarguelloF/llmobs-fixes with baseline commit 45246a0 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 24 metrics, 0 unstable metrics.

@rarguelloF rarguelloF changed the title fix(llmobs): fix confusing logs and make SubmitEvaluationFromSpan more flexible fix(llmobs): include missing tags and make SubmitEvaluationFromSpan more flexible Oct 21, 2025
@rarguelloF rarguelloF marked this pull request as ready for review October 21, 2025 07:53
@rarguelloF rarguelloF requested a review from darccio October 21, 2025 07:56
@rarguelloF rarguelloF requested a review from sabrenner October 21, 2025 08:14
Copy link

@sabrenner sabrenner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice lgtm! just a couple noob Go questions 😄

}
if cfg.AgentlessEnabled != nil {
if !*cfg.AgentlessEnabled && !agentSupportsLLMObs {
log.Warn("llmobs: DD_LLMOBS_AGENTLESS_ENABLED has been configured to false and the agent is not available or does not support llmobs")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we return an error in this case? i don't think llmobs will be able to submit data in the scenario where there's no agent running and agentlessEnabled is explicitly set to false, and maybe we wanna surface that to the user harsher than a warning log, but not sure of the protocols for doing that in the go tracer!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm yes! this totally makes sense, I will change it 👍

@rarguelloF rarguelloF requested review from a team as code owners October 21, 2025 14:35
@rarguelloF rarguelloF requested a review from sabrenner October 21, 2025 15:20
@rarguelloF
Copy link
Contributor Author

/merge

@dd-devflow-routing-codex
Copy link

dd-devflow-routing-codex bot commented Oct 21, 2025

View all feedbacks in Devflow UI.

2025-10-21 15:43:51 UTC ℹ️ Start processing command /merge


2025-10-21 15:56:53 UTC ℹ️ MergeQueue: waiting for PR to be ready

This merge request is not mergeable according to GitHub. Common reasons include pending required checks, missing approvals, or merge conflicts — but it could also be blocked by other repository rules or settings.
It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.


2025-10-21 19:57:18 UTC ⚠️ MergeQueue: This merge request was unqueued

devflow unqueued this merge request: It did not become mergeable within the expected time

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants