-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Description
Motivation
In a single step of ChatAgent, there could be multiple tool calls. here can be multiple tool calls. Different tool calls consume different amounts of tokens: some are more expensive, while others are cheaper. For example, the search agent of Eigent may use both the Google Search API (which typically costs fewer tokens) and browser tools (which usually cost more) to complete a task.
Having a tool-level token tracking mechanism can help us better understand the agent’s working process and the cost of each tool call. This can also serve as a potential reward function for future reinforcement learning research, enabling the agent to optimize its workflow for greater cost efficiency.
The expectation of this sub-issue
Currently ChatAgent already tracks the tool-call trajectory and token usage for each step in the info field of ChatAgentResponse. We now hope to have a finer-grained tracking mechanism that records the token usage of each individual tool call.