move towards completing what is started

lucidrains · Oct 2, 2023 · 94002e5 · 94002e5
1 parent 0434d06
commit 94002e5
Show file tree

Hide file tree

Showing 4 changed files with 676 additions and 1 deletion.
diff --git a/README.md b/README.md
@@ -15,6 +15,9 @@ Also have a few ideas of my own that I will try and share in this repository, if
 - [x] make batched spec decoding work with early exit strategy
 
 - [ ] build out the prophet net idea, but use the same scheme as megabyte, the hierarchical transformer, for the prophet head. this hierarchical transformer would then use the cached embedding from the large model (since we are caching the embeddings)
+    - [x] complete prophet net with hierarchical transformer training
+    - [ ] complete the spec decoding algorithm using trained prophet net transformer
+
 - [ ] dedicate a morning to microoptimizations
 
 ## Citations

diff --git a/setup.py b/setup.py
@@ -3,7 +3,7 @@
 setup(
   name = 'speculative-decoding',
   packages = find_packages(exclude=[]),
-  version = '0.0.12',
+  version = '0.0.14',
   license='MIT',
   description = 'Speculative Decoding',
   author = 'Phil Wang',