Init the codes of spark-rapids-tools

Co-authored-by: Ahmed Hussein (amahussein) <[email protected]> Co-authored-by: Raza Jafri <[email protected]> Co-authored-by: Alex Zhang <[email protected]> Initial commit add gpu savings cost estimator code update readme file Signed-off-by: Ahmed Hussein (amahussein) <[email protected]> Update README.md -2 add more details to the README file Signed-off-by: Ahmed Hussein (amahussein) <[email protected]> Update README.md download rapids_4_spark_qualification_output.csv and gen the cluster profile removed hard-coded refs removed copying files in the home directory and added region complete end-to-end POC Signed-off-by: Ahmed Hussein <[email protected]> submit Qualification tool as Spark-job Signed-off-by: Ahmed Hussein <[email protected]> address features request Signed-off-by: Ahmed Hussein <[email protected]> use 2 decimal points Signed-off-by: Ahmed Hussein <[email protected]> remove deprecated log.warn Signed-off-by: Ahmed Hussein <[email protected]> update the RADME to reflect new arguments Signed-off-by: Ahmed Hussein <[email protected]> Fix typos and indentations Signed-off-by: Ahmed Hussein <[email protected]> PIP packaging changes Add Dataproc cluster config bootstrap script Signed-off-by: Jason Lowe <[email protected]> Use named constants Add a command line param to point to a local rapids tools jar Show durations in seconds by default + misc formatting Fix the cleanup method to delete files and folders Remove success state add profiling-tool to dataproc-wrapper Support autoTuning in dataprocWrapper Support A100 for profiling tool support Python 3.8+ use camelCase in workerInfo file Fixing profiling column name issue Running qualification on non-n1 instances should show savings based on n1 clusters [CICD] fix #41 support dynamic version Preparing for 22.10 release fix #43 rename module and package name nightly package version optimize Optimize nightly version timestamp Add diag tools for nv driver and cuda version check from local & dataproc Changing wording from Spark RAPIDS to RAPIDS Accelerator for Apache Spark Fixing last reference Support Tool options in the wrapper change base version to 22.10.2 Add changelog and optimize license Remove gitlab link Signed-off-by: Gary Shen <[email protected]>
NVIDIA · Nov 4, 2022 · 6975fb0 · 6975fb0
1 parent 4d492bb
commit 6975fb0
Show file tree

Hide file tree

Showing 32 changed files with 28,534 additions and 0 deletions.
diff --git a/.gitignore b/.gitignore
@@ -0,0 +1,29 @@
+# Created by .ignore support plugin (hsz.mobi)
+### Python template
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+.tox
+
+# virtualenv
+venv/
+ENV/
+
+# intellij
+/.idea/
+/gpu-savings-estimator.iml
+
+# data
+/cluster-properties.yaml
+wrapper-output/
+
+### macOS ###
+*.DS_Store
+
+### packaging ### 
+dist
+build
+src/*.egg-info
+build
+.coverage
diff --git a/.pylintrc b/.pylintrc
@@ -0,0 +1,4 @@
+[FORMAT]
+max-line-length=120
+
+disable=logging-fstring-interpolation,too-many-function-args
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -0,0 +1,94 @@
+# Contributing to RAPIDS Accelerator for Apache Spark Tools
+
+Contributions to RAPIDS Accelerator for Apache Spark Tools fall into the following three categories.
+
+1. To report a bug, request a new feature, or report a problem with
+    documentation, please file an [issue](https://github.com/NVIDIA/spark-rapids-tools/issues/new/choose)
+    describing in detail the problem or new feature. The project team evaluates
+    and triages issues, and schedules them for a release. If you believe the
+    issue needs priority attention, please comment on the issue to notify the
+    team.
+2. To propose and implement a new Feature, please file a new feature request
+    [issue](https://github.com/NVIDIA/spark-rapids-tools/issues/new/choose). Describe the
+    intended feature and discuss the design and implementation with the team and
+    community. Once the team agrees that the plan looks good, go ahead and
+    implement it using the [code contributions](#code-contributions) guide below.
+3. To implement a feature or bug-fix for an existing outstanding issue, please
+    follow the [code contributions](#code-contributions) guide below. If you
+    need more context on a particular issue, please ask in a comment.
+
+## Branching Convention
+
+There are two branches in this repository:
+
+* `dev`: are development branches which can change often. Note that we merge into
+  the branch with the greatest version number, as that is our default branch.
+
+* `main`: is the branch with the latest released code, and the version tag (i.e. `v0.1.0`)
+  is held here. `main` will change with new releases, but otherwise it should not change with
+  every pull request merged, making it a more stable branch.
+
+## Code contributions
+
+### Sign your work
+
+We require that all contributors sign-off on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
+
+Any contribution which contains commits that are not signed off will not be accepted.
+
+To sign off on a commit use the `--signoff` (or `-s`) option when committing your changes:
+
+```shell
+git commit -s -m "Add cool feature."
+```
+
+This will append the following to your commit message:
+
+```
+Signed-off-by: Your Name <[email protected]>
+```
+
+The sign-off is a simple line at the end of the explanation for the patch. Your signature certifies that you wrote the patch or otherwise have the right to pass it on as an open-source patch. Use your real name, no pseudonyms or anonymous contributions.  If you set your `user.name` and `user.email` git configs, you can sign your commit automatically with `git commit -s`.
+
+
+The signoff means you certify the below (from [developercertificate.org](https://developercertificate.org)):
+
+```
+Developer Certificate of Origin
+Version 1.1
+
+Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
+1 Letterman Drive
+Suite D4700
+San Francisco, CA, 94129
+
+Everyone is permitted to copy and distribute verbatim copies of this
+license document, but changing it is not allowed.
+
+
+Developer's Certificate of Origin 1.1
+
+By making a contribution to this project, I certify that:
+
+(a) The contribution was created in whole or in part by me and I
+    have the right to submit it under the open source license
+    indicated in the file; or
+
+(b) The contribution is based upon previous work that, to the best
+    of my knowledge, is covered under an appropriate open source
+    license and I have the right under that license to submit that
+    work with modifications, whether created in whole or in part
+    by me, under the same open source license (unless I am
+    permitted to submit under a different license), as indicated
+    in the file; or
+
+(c) The contribution was provided directly to me by some other
+    person who certified (a), (b) or (c) and I have not modified
+    it.
+
+(d) I understand and agree that this project and the contribution
+    are public and that a record of the contribution (including all
+    personal information I submit with it, including my sign-off) is
+    maintained indefinitely and may be redistributed consistent with
+    this project or the open source license(s) involved.
+```