Skip to content

Conversation

keenborder786
Copy link
Contributor

  • Description: Missing keys for usage metadata: According to the Perplexity API documentation, the following output token details were missing in ChatPerplexitycitation_tokens, num_search_queries, and reasoning_token — which are required for accurate cost calculation, as noted in the issue. These have now been added. Although OutputTokenDetails has restricted keys, all of them have been included for now.
  • Issue: Add Perplexity cost tracking #31647

@github-actions github-actions bot added integration Related to a provider partner package integration infra Chores, devops, repo meta changes labels Oct 14, 2025
Copy link

codspeed-hq bot commented Oct 14, 2025

CodSpeed Performance Report

Merging #33480 will not alter performance

Comparing keenborder786:feat/preplexity_cost_tracking (4914216) with master (5acd34a)1

Summary

✅ 1 untouched
⏩ 33 skipped2

Footnotes

  1. No successful run was found on master (f8adbbc) during the generation of this report, so 5acd34a was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

  2. 33 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

citation_tokens = token_usage.get("citation_tokens", 0)
num_search_queries = token_usage.get("num_search_queries", 0)
reasoning_tokens = token_usage.get("reasoning_tokens", 0)
return UsageMetadata( # type: ignore[typeddict-unknown-key]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the linter is complaining here because UsageMetadata does not support extra keys.

We should follow the existing types where possible: https://python.langchain.com/api_reference/core/messages/langchain_core.messages.ai.UsageMetadata.html

reasoning_tokens has a dedicated slot reasoning under output_token_details.

citation_tokens we can also stash in output_token_details.

num_search_queries IMO can go in response_metadata since it isn't a token count but I don't feel strongly about that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ccurme please check now

@ccurme ccurme self-assigned this Oct 20, 2025
Copy link
Collaborator

@ccurme ccurme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this, could we add a test?

Comment on lines +69 to +71
output_token_details: OutputTokenDetails = {}
output_token_details["reasoning"] = token_usage.get("reasoning_tokens", 0)
output_token_details["citation_tokens"] = token_usage.get("citation_tokens", 0) # type: ignore[typeddict-unknown-key]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(nit) might make sense to only populate the keys if they are present in token_usage.

@keenborder786
Copy link
Contributor Author

@ccurme done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

infra Chores, devops, repo meta changes integration Related to a provider partner package integration

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants