-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Track costs for streaming with Cohere #218
Comments
See #214 since these are related. Namely: #214 (comment) |
I am working on this but the problem I am facing is that |
Doesn't the |
Yes, it does, but according to the API docs, the streamed response has no |
This is partially implemented with #307 where Cohere chunks will contain input_tokens and output_tokens which can be used to calculate cost. The only thing remaining that will need to be done is to pass |
Is your feature request related to a problem? Please describe.
Many providers are starting to add usage to streaming. This makes it much easier for Mirascope to calculate cost.
Describe the solution you'd like
Add a
total_cost
property toCohereCallResponseChunk
. Read the"event_type": "stream-end"
sent by Cohere API and calculate cost usingUpdate https://github.com/Mirascope/mirascope/blob/dev/mirascope/cohere/utils.py as necessary.
The text was updated successfully, but these errors were encountered: