[C] Normalization Refactor + Adding CUDNN backend (#1315)

* cuDNN normalization integration * TE Norm refactor * TE Norm APIs changes. --------- Signed-off-by: Phuong Nguyen <[email protected]> Co-authored-by: Tim Moon <[email protected]>
NVIDIA · Dec 6, 2024 · 3102fdd · 3102fdd
1 parent d8b13cb
commit 3102fdd
Show file tree

Hide file tree

Showing 37 changed files with 3,029 additions and 3,727 deletions.
diff --git a/tests/cpp/operator/CMakeLists.txt b/tests/cpp/operator/CMakeLists.txt
@@ -10,8 +10,7 @@ add_executable(test_operator
                test_cast_transpose_dbias_dgelu.cu
                test_cast_transpose_dgeglu.cu
                test_act.cu
-               test_layernorm.cu
-               test_rmsnorm.cu
+               test_normalization.cu
                test_multi_cast_transpose.cu
                test_multi_padding.cu
                test_causal_softmax.cu

diff --git a/tests/cpp/operator/test_layernorm.cu b/tests/cpp/operator/test_layernorm.cu