Skip to content

[metrics] Implement token usage metrics tracking for chat models #394

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
Sxnan merged 1 commit into from
Dec 26, 2025
Merged

[metrics] Implement token usage metrics tracking for chat models #394

Sxnan merged 1 commit into from
Dec 26, 2025

Conversation

@Sxnan
Copy link
Contributor

@Sxnan Sxnan commented Dec 25, 2025 •
edited
Loading

Linked issue: #390

Purpose of change

This PR adds built-in token usage metrics for chat model connections. The metrics are automatically recorded when chat models are invoked, tracking promptTokens and completionTokens per model.

Tests

  • Added BaseChatModelConnectionTokenMetricsTest.java for Java unit tests
  • Added test_token_metrics.py for Python unit tests

API

Yes, this change adds new built-in metrics

Documentation

  • doc-needed
  • doc-not-needed

@github-actions github-actions bot added priority/major Default priority of the PR or issue. fixVersion/0.2.0 The feature or bug should be implemented/fixed in the 0.2.0 version. doc-needed Your PR changes impact docs. labels Dec 25, 2025
@Sxnan
Copy link
Contributor Author

Sxnan commented Dec 25, 2025

@xintongsong @GreatEugenius Can you review this PR?

xintongsong
xintongsong approved these changes Dec 25, 2025
View reviewed changes
Copy link
Contributor

@xintongsong xintongsong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Sxnan Sxnan merged commit 6bc5f58 into apache:main Dec 26, 2025
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@xintongsong xintongsong xintongsong approved these changes

@GreatEugenius GreatEugenius Awaiting requested review from GreatEugenius

Assignees

No one assigned

Labels

doc-needed Your PR changes impact docs. fixVersion/0.2.0 The feature or bug should be implemented/fixed in the 0.2.0 version. priority/major Default priority of the PR or issue.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Sxnan @xintongsong