Autotuning (deepspeedai#1554)

* [squash] Staging autotuning v4 Co-authored-by: Cheng Li <[email protected]> Co-authored-by: Minjia Zhang <[email protected]> Co-authored-by: Olatunji Ruwase <[email protected]> Co-authored-by: Jeff Rasley <[email protected]> * add new extra, guard xgboost, cleanup dead files (deepspeedai#268) * Fix autotuning docs (deepspeedai#1553) * fix docs * rewording the goal * fix typos * fix typos (deepspeedai#1556) * fix typos * fix format * fix bug (deepspeedai#1557) * fix bug Co-authored-by: Jeff Rasley <[email protected]> Co-authored-by: Minjia Zhang <[email protected]> Co-authored-by: Olatunji Ruwase <[email protected]>
PeixuanZuo · Nov 13, 2021 · 9caa74e · 9caa74e
1 parent b7cc7c8
commit 9caa74e
Show file tree

Hide file tree

Showing 46 changed files with 5,982 additions and 1,816 deletions.
diff --git a/.github/workflows/formatting.yml b/.github/workflows/formatting.yml
@@ -27,7 +27,7 @@ jobs:
 
       - name: Install deepspeed
         run: |
-          pip install .[dev]
+          pip install .[dev,autotuning]
           ds_report
 
       - name: Formatting checks

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
@@ -31,7 +31,7 @@ jobs:
 
       - name: Install deepspeed
         run: |
-          pip install .[dev]
+          pip install .[dev,autotuning]
           ds_report
 
       - name: Unit tests
@@ -58,7 +58,7 @@ jobs:
           python -c "import torch; print('CUDA available:', torch.cuda.is_available())"
       - name: Install deepspeed
         run: |
-          pip install .[dev,1bit]
+          pip install .[dev,1bit,autotuning]
           ds_report
       - name: Unit tests
         run: |
@@ -85,7 +85,7 @@ jobs:
           python -c "import torch; print('CUDA available:', torch.cuda.is_available())"
       - name: Install deepspeed
         run: |
-          pip install .[dev]
+          pip install .[dev,autotuning]
           ds_report
       - name: HF transformers tests
         run: |

diff --git a/MANIFEST.in b/MANIFEST.in
@@ -1,4 +1,4 @@
 include *.txt README.md
 recursive-include requirements *.txt
-recursive-include deepspeed *.cpp *.h *.cu *.tr *.cuh *.cc
+recursive-include deepspeed *.cpp *.h *.cu *.tr *.cuh *.cc *.json
 recursive-include csrc *.cpp *.h *.cu *.tr *.cuh *.cc
diff --git a/deepspeed/autotuning/.gitignore b/deepspeed/autotuning/.gitignore
@@ -0,0 +1,6 @@
+test*
+runs
+autotuning_results*
+autotuning_exps
+output*
+*.png
diff --git a/deepspeed/autotuning/README.md b/deepspeed/autotuning/README.md
diff --git a/deepspeed/autotuning/__init__.py b/deepspeed/autotuning/__init__.py
@@ -0,0 +1 @@
+from .autotuner import Autotuner