Skip to content

Issues: huggingface/nanotron

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
59 Open 47 Closed
59 Open 47 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[Feature] Kernel Fusion of Layer Norm and GeLU enhancement New feature or request help wanted Extra attention is needed
#86 opened Mar 2, 2024 by xrsrke
[Refactor] DistributedOptimizer and FP32GradAccum
#20 opened Jan 12, 2024 by NouamaneTazi
@NouamaneTazi
[Feature Request] Add simple communications benchmarks to the repo enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed
#43 opened Jan 25, 2024 by NouamaneTazi
1
[Feature Request] Support Data Streaming for faster training of large models enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed
#45 opened Jan 26, 2024 by chagri
2
[Question] Correctness of backward pass of RowLinear
#46 opened Jan 26, 2024 by ufotalent
2
[Question] Async Tensor Parallel
#48 opened Jan 27, 2024 by woshiyyya
2
[Question] Modification for Performing Fine-Tuning
#57 opened Feb 5, 2024 by allanj
3
[Feature request] Performance and accuracy benchmarks
#61 opened Feb 9, 2024 by brianyu-nexusflowai
2
[Feature] Fix support for sequence parallelism with MoEs enhancement New feature or request help wanted Extra attention is needed
#74 opened Feb 16, 2024 by NouamaneTazi
[Feature] Refactor ParallelContext.world_rank_matrix enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed
#77 opened Feb 19, 2024 by NouamaneTazi
Continued Pretraining on Llama 7b.
#79 opened Feb 23, 2024 by wiseyy
8
[Feature] Parallel transformer block enhancement New feature or request help wanted Extra attention is needed High Priority
#84 opened Mar 2, 2024 by xrsrke
@xrsrke
New APIs
#10 opened Jan 5, 2024 by xrsrke
[Feature] LAMB optimizer enhancement New feature or request help wanted Extra attention is needed
#85 opened Mar 2, 2024 by xrsrke
1
[Feature] Asyncronous Serialization enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed
#87 opened Mar 2, 2024 by xrsrke
2
[Feature] Use CUDA event for measuring elasped time enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed
#88 opened Mar 2, 2024 by xrsrke
[Unit Test] Add unit test for DoReMi's trainer Medium Priority
#89 opened Mar 2, 2024 by xrsrke
@xrsrke
[Unit Test] Add unit tests for DistributedTrainer good first issue Good for newcomers help wanted Extra attention is needed High Priority
#90 opened Mar 2, 2024 by xrsrke
5
[Feature] All GPUs within the same TP group load training data from shared memory enhancement New feature or request help wanted Extra attention is needed Low Priority
#91 opened Mar 3, 2024 by xrsrke
[Features] support gradient checkpointing for memory saving
#99 opened Mar 5, 2024 by zguo0525
1
FEAT: Support 1.58-bit LLMs training enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed Low Priority
#114 opened Mar 21, 2024 by younesbelkada
1
Add data loading time in log
#154 opened Apr 26, 2024 by XinDongol
moe in src and load balancing losses
#159 opened May 2, 2024 by haeggee
2
out of memory for continuing pretraining llama3-8B
#161 opened May 3, 2024 by ckzbullbullet
5
Abnormal grad norm during pre-training
#306 opened Mar 25, 2025 by SinclairCoder
ProTip! What’s not been updated in a month: updated:<2025-02-28.