Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preparing LLM Benchmark Table ( LangTest) #946

Open
ArshaanNazir opened this issue Dec 27, 2023 · 9 comments
Open

Preparing LLM Benchmark Table ( LangTest) #946

ArshaanNazir opened this issue Dec 27, 2023 · 9 comments
Assignees

Comments

@ArshaanNazir
Copy link
Collaborator

No description provided.

@JustHeroo
Copy link

@ArshaanNazir please add yourself as an assignee to the task

@ArshaanNazir ArshaanNazir self-assigned this Dec 28, 2023
@JustHeroo
Copy link

@ArshaanNazir any updates?

@ArshaanNazir
Copy link
Collaborator Author

We are working on it. Here is the link to the tracking sheet: https://johnsnowlabs-my.sharepoint.com/:x:/p/rakshit/ETX1Z44PipFOqm8Ue8Av3_UBycHH_9oK-oJJUpQfc_n54w?e=exe0Ja

@muhammetsnts
Copy link

@ArshaanNazir did we publish any benchmark (LLM and embeddings) on the LangTest web site?

@ArshaanNazir
Copy link
Collaborator Author

We have created the streamlit apps for both of the benchmark tables. We are finalising their design and will be update on website by end of this week.

@Cabir40
Copy link

Cabir40 commented Feb 5, 2024

@ArshaanNazir @vkocaman We have created a new folder for the langtest demos

https://github.com/JohnSnowLabs/streamlit-demo-apps/tree/master/langtest

do you need anything else?

@ArshaanNazir
Copy link
Collaborator Author

I am not sure if we are going ahead with the streamlit apps now. @dcecchini can you confirm ?

@dcecchini
Copy link
Contributor

Hi @Cabir40 @ArshaanNazir @muhammetsnts @JustHeroo, we started creating the streamlit apps for the leaderboards but @vkocaman suggested to ask the design team to build them using web tools that look better.

They are preparing them; you can check a draft at in this link.

In the meantime, we are reviewing the information to be contained on the pages, as we need to make sure that the leaderboards show all the relevant information (adding more filters, improving the visualization, creating more data with benchmark results, etc.).

@rajshah4
Copy link

rajshah4 commented Feb 5, 2024

I understand why you would want a more attractive web app. I was hoping for a streamlit app -- simply because I am looking for an LLM leaderboard in a box that I could deploy to enterprise clients.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants