Skip to content

Issues: turboderp/exllamav2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

73 Open 354 Closed
73 Open 354 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[BUG] Quantization of Qwen return garbage bug Something isn't working
#621 opened Sep 10, 2024 by fahadh4ilyas
3 tasks done
8
How to implement paged attention in HF format?
#616 opened Sep 6, 2024 by fahadh4ilyas
5
Want to try row split + all_reduce for MLP and attn
#614 opened Sep 3, 2024 by Azure-Tang
4
how can i solve this problem
#611 opened Sep 2, 2024 by Sultan0ML
Async Stream Genenerator?
#604 opened Aug 28, 2024 by KingBipo
3
Does NVLink improve tensor parallelism?
#603 opened Aug 26, 2024 by bryanhpchiang
1
lollms exllamav2 binding module not found
#602 opened Aug 26, 2024 by Fuckingnameless
1
## Measurement/inference error (3): hidden_states
#600 opened Aug 24, 2024 by IllIIllIlllIIl
6
Do you know of any code framework that supports fast attention score calculation similar to flash attention?
#599 opened Aug 24, 2024 by qiyuxinlin
Tensor parallelism issues
#598 opened Aug 24, 2024 by dirkson
5
Request for multi model support
#595 opened Aug 20, 2024 by royallavanya140
[qesstion] Wrapper Linear API and 2bits
#589 opened Aug 14, 2024 by wenhuach21
4
Error in quant
#587 opened Aug 8, 2024 by Orion-zhen
2
Llama 3 speed
#585 opened Aug 4, 2024 by freQuensy23-coder
2
graphrag can't index using mistral large 123B with exllamav2
#582 opened Aug 3, 2024 by xxll88
1
Add more docs and type annotations
#579 opened Jul 30, 2024 by Dan-wanna-M
Will it support CPU offloading?
#578 opened Jul 30, 2024 by fzyzcjy
5
Q8 or unquantized cache with what context length for llama 3.1-8b 5.0 bpw exl2?
#575 opened Jul 27, 2024 by lovebeatz
8
Triton Support
#574 opened Jul 26, 2024 by rjmehta1993
1
orig_func Quantization error
#573 opened Jul 25, 2024 by Masterjp123
3
Curious about Exllama+TP
#571 opened Jul 25, 2024 by grimulkan
10
[ERROR] Worker (pid:25134) was sent SIGKILL! Perhaps out of memory?
#556 opened Jul 18, 2024 by UTSAV-44
12
Manual model merges
#555 opened Jul 18, 2024 by dnhkng
2
I am trying to add DeepSeekV2Moe model
#554 opened Jul 18, 2024 by qiyuxinlin
8
Conversion error
#552 opened Jul 16, 2024 by TMust77
2
ProTip! Adding no:label will show everything without a label.