Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[UX] Add --list-metrics option to list all available metrics #115

Open
xuzhao9 opened this issue Dec 12, 2024 · 2 comments
Open

[UX] Add --list-metrics option to list all available metrics #115

xuzhao9 opened this issue Dec 12, 2024 · 2 comments

Comments

@xuzhao9
Copy link
Contributor

xuzhao9 commented Dec 12, 2024

Add --list-metrics to list all available metrics.

Add --list-backends to list all backends and their tags, we should be able to tag PT2, Triton, and CUTLASS backends.

@adamomainz
Copy link
Contributor

what would --list-metrics provide here? a list of all possible metrics that come standard or a list of metrics for a specific op? same for backends

@FindHao
Copy link
Member

FindHao commented Dec 12, 2024

currently, we have different kinds of metrics:

  • common ones defined in triton_op.py. they are general to the whole tests.
  • metrics defined in components, for now ncu/nsys analyzer have its own metric list. specific for different components.
  • metrics defined in individual ops. ops use register_metric to register their own metrics. specific for ops.
  • metrics defined in operator loader . a different component from operators.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants