Skip to content

Issues: THUDM/LongBench

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

25 Open 55 Closed
25 Open 55 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

LongBench v2 Evaluation regarding Qwen2.5
#91 opened Dec 27, 2024 by guanzhchen
关于cot
#90 opened Dec 26, 2024 by Violettttee
DeepSeek V3 Result
#89 opened Dec 26, 2024 by zmwang03
results.py bug
#88 opened Dec 24, 2024 by cizhenshi
1
安装完longbench后，运行报错
#86 opened Dec 14, 2024 by fxnie
Request for Mixtral-8*7B model
#84 opened Nov 18, 2024 by Aaronhuang-778
请问用GPT-3.5-turbo-16K测试一遍完整的数据集需要花费多少？
#83 opened Nov 4, 2024 by HelloEveryonehh
1
Evaluation mechanism update
#82 opened Oct 30, 2024 by cizhenshi
1
可以测试基于OpenAI接口的模型管理框架吗，比如ollama, xinference
#74 opened Aug 29, 2024 by jiusi9
1
No initialization for the process group
#73 opened Aug 28, 2024 by Mugariya
1
How to evaluate on llama3-8b-instruct? enhancement New feature or request
#71 opened Aug 11, 2024 by txchen-USTC
@bys0318
1
关于提升数据集测试有效性的建议
#70 opened Jul 25, 2024 by wsn555
7
Code for evaluation with GPT-3.5?
#69 opened Jul 22, 2024 by RuskinManku
3
Load dataset from hf failed
#68 opened Jul 16, 2024 by murphypei
4
The "anwser" for some examples in "qasper.jsonl" is strange
#67 opened Jul 9, 2024 by Zcchill
6
Loading local datasets with split=‘test’
#65 opened May 30, 2024 by yichen0104
1
Chinese Examples in MultiFieldQA-en
#64 opened May 5, 2024 by wendywangwwt
1
Llama2-7B-chat-4k on PassageRetrieval-zh gets 10.12
#61 opened Mar 18, 2024 by fuqichen1998
5
Include data on which passage contains answer enhancement New feature or request
#60 opened Mar 7, 2024 by danielmisrael
1
求问 Spearman correlation 是怎么计算的
#57 opened Mar 2, 2024 by randomtutu
1
Any Implementation of Mistral-7B?
#54 opened Feb 27, 2024 by leeyeehoo
1
报错TypeError: Couldn't cast array of type list<item: string> to null
#52 opened Feb 26, 2024 by xxcoco763
1
测试13b，比如百川，1*A100（80G）会OOM
#49 opened Jan 12, 2024 by lvjianxin
Evaluate on long context (32k,64k etc..) on 30B/70B large models
#48 opened Jan 9, 2024 by CaesarWWK
5
关于TREC数据集中的typo
#28 opened Sep 18, 2023 by QuanYuhan
1
ProTip! Follow long discussions with comments:>50.