Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Model performance and evaluation metrics in the OmniMedVQA dataset #21

Open
Lycus99 opened this issue Jul 16, 2024 · 0 comments
Open

Model performance and evaluation metrics in the OmniMedVQA dataset #21

Lycus99 opened this issue Jul 16, 2024 · 0 comments

Comments

@Lycus99
Copy link

Lycus99 commented Jul 16, 2024

Thanks for your work!
After reading the paper OmniMedVQA, I have two questions and sincerely look forward to the answers.

  1. From the paper of MedVInT and RadFM, the dataset used in the radfm model is larger than that of medvint (16M vs. 1.64M). However, the performance of medvint is better than radfm in your paper. Do you further analyze the prediction results of the two models?

  2. QA scores and prefix-based scores are distributed differently across image modalities. Which metric is more useful when selecting a model under a certain modality?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant