Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve the autoscaling benchmark scripts #666

Open
10 tasks
Jeffwan opened this issue Feb 13, 2025 · 1 comment
Open
10 tasks

Improve the autoscaling benchmark scripts #666

Jeffwan opened this issue Feb 13, 2025 · 1 comment
Assignees
@gangmuk @happyandslow
Labels
area/benchmark area/performance priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now.

Comments

@Jeffwan
Copy link
Collaborator

Jeffwan commented Feb 13, 2025 •
edited
Loading

🚀 Feature Description and Motivation

Copy some TODO items from issue #650

WIP items:

  • Improving aibrix/benchmarks/generator/client.py with async and streaming mode
    • TTFT
    • TPOT
    • Goodput
  • Adjust the QPS scale
  • Improving plot script
    • Tokens/s times series plot
    • TTFT times series plot
    • TPOT times series plot
    • Goodput times series plot

Use Case

This is to improve the qualify of the autoscaling experiments

Proposed Solution

No response
@Jeffwan Jeffwan added area/benchmark area/performance priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. labels Feb 13, 2025
@happyandslow happyandslow mentioned this issue Feb 17, 2025
Merged
@happyandslow
Copy link
Collaborator

Addressing in #676

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gangmuk gangmuk

@happyandslow happyandslow

Labels
area/benchmark area/performance priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Jeffwan @gangmuk @happyandslow