Model comparison tool? #767
jasontitus
started this conversation in
General
Replies: 1 comment
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Has anyone written or released a tool for comparing the WER of various models against a given dataset? It seems like as more and more fine tuning happens, folks will want to know which model is best for their language or use case. I have been tinkering with Common Voice sets and doing short run fine tuning against them but that is pretty high overhead and not automated at all.
Ideally I could just point to a set of local directories or hugginface repositories and set the evaluation dataset and compute metric.
Just checking before I hack together something of my own
Beta Was this translation helpful? Give feedback.
All reactions