[Improvement] Align evaluation results with paper

[white2018]

The current verison of minimonkey.py evaluates model with different results compared to paper's evaluation. The paper's link refers to https://arxiv.org/pdf/2408.02034

[kennymckormick]

Will re-evaluate w. this piece of codes to see if results improve

[white2018]

Thanks a lot!

…

------------------ 原始邮件 ------------------ 发件人: "open-compass/VLMEvalKit" ***@***.***>; 发送时间: 2024年11月1日(星期五) 下午5:20 ***@***.***>; ***@***.******@***.***>; 主题: Re: [open-compass/VLMEvalKit] [Improvement] Align evaluation results with paper (PR #563) Will re-evaluate w. this piece of codes to see if results improve — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[kennymckormick]

Thanks a lot!
…
------------------ 原始邮件 ------------------ 发件人: "open-compass/VLMEvalKit" @.>; 发送时间: 2024年11月1日(星期五) 下午5:20 @.>; @.@.>; 主题: Re: [open-compass/VLMEvalKit] [Improvement] Align evaluation results with paper (PR #563) Will re-evaluate w. this piece of codes to see if results improve — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

Can you confirm that the modified codes run well on benchmarks we supported, at least for the 8 benchmarks on our main leaderboard? I ran MiniMonkey on MMMU_DEV_VAL with 80G A800 and the OOM error occurs.

[white2018]

Thanks a lot!
…
------------------ 原始邮件 ------------------ 发件人: "open-compass/VLMEvalKit" @.>; 发送时间: 2024年11月1日(星期五) 下午5:20 _@**._>; _@.@._>; 主题: Re: [open-compass/VLMEvalKit] [Improvement] Align evaluation results with paper (PR #563) Will re-evaluate w. this piece of codes to see if results improve — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: _@_.*>

Can you confirm that the modified codes run well on benchmarks we supported, at least for the 8 benchmarks on our main leaderboard? I ran MiniMonkey on MMMU_DEV_VAL with 80G A800 and the OOM error occurs.

I will re-evaluate MMMU_DEV_VAL dataset to see what happens. Thanks

[kennymckormick]

The evaluation results are updated.