Progress Bar View

[CharlieJCJ]

In curator-viewer

Features:

• [Progress Bar] number of samples completed @DatasetViewer
  • doesn't need logging from curator
• [Progress Bar] total cost estimation & total cost already spent @DatasetViewer
  • logging done in add cost and token logging in openai online and batching with litellm completion_cost #159
• [Progress Bar] number of samples completed @RunsViewer
  • Requires additional logging logic
• [Progress Bar] total cost estimation & total cost already spent @RunsViewer
  • Requires additional logging logic

In terminal logger

• number of samples (in tqdm)
• total cost estimation & total cost already spent

Things to consider:

• Batch Mode cost estimation is quite different from online. Naive implementation for batch mode logging would have great delay time to get the first estimation. Need different logging strategies. Maybe do a separate thread of online and get token counts, just to get a rough sense of the cost for batch?
• BTW estimation and monitor are two different concepts. different UX

[RyanMarten]

Running average of cost / call
And then update in progress bar the already consumed + estimated cost