Skip to content

Pull requests: karpathy/minbpe

New pull request New
23 Open 32 Closed
23 Open 32 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Add pytest to requirements.txt
#91 opened Oct 29, 2024 by shoumikhin
add lexicographic ordering for breaking ties to make the tokenizer deterministic
#90 opened Sep 21, 2024 by dapopov-st
Add pyproject toml
#89 opened Sep 17, 2024 by gianlucagiudice
updating stats across merge to reduce computation
#88 opened Sep 1, 2024 by imdaredevil
Update README.md
#86 opened Jul 1, 2024 by wenming-ma
Optimal algorithm for _encode_chunk(): 20% faster encoding, with 0.5% better COMPRESSION
#84 opened Jun 17, 2024 by Majdoddin
Deduplication of text chunks with frequency count, training and encoding 5x speedup
#82 opened Jun 8, 2024 by Majdoddin
2
calling len(ids) in merge() function only once to increase performance
#76 opened May 14, 2024 by crpatil1901
Link to Mojo port added
#75 opened May 14, 2024 by dorjeduck
Count only nonoverlapping occurences of a pair
#72 opened May 6, 2024 by Majdoddin
Update regex.py to correctly parse scripts with combining marks
#71 opened May 5, 2024 by ajaykg
5
Updated decode() method in GPT4Tokenizer so that it handles special t…
#63 opened Apr 7, 2024 by Vakarva
Handle error when running out of pairs to merge
#54 opened Mar 6, 2024 by vinhdq842
updated self.vocab initialization and reuse self._build_vocab()
#53 opened Mar 2, 2024 by muerghq
Video2Post Generation Workflow
#49 opened Feb 26, 2024 by xihajun • Draft
8 of 17 tasks
3
Update lecture.md based on video tutorial content from 08:15 through 28:23
#42 opened Feb 23, 2024 by astaff
Automating testing using github actions
#41 opened Feb 22, 2024 by nobleaustine
Use pyproject.toml, pdm and ruff for improved reproducibility and cleaner code
#40 opened Feb 22, 2024 by nizhib
Create setup.py
#39 opened Feb 22, 2024 by Biswajit2902
1
Train BasicTokenizer on GPU with PyTorch, 100x speedup
#38 opened Feb 22, 2024 by kuprel
4
Fix small typos
#34 opened Feb 21, 2024 by ggaughan
1
Simplify generation of printable representation
#26 opened Feb 21, 2024 by SultanOrazbayev
2
Batch encoding decoding
#22 opened Feb 20, 2024 by ezzaldeeen
2
ProTip! Updated in the last three days: updated:>2025-03-23.