Skip to content

Commit 3e24d01

Browse files
nvautonvauto
authored
Merge pull request #11698 from NVIDIA/merge-branch-24.10-to-main
Merge branch-24.10 into main
2 parents 484727d + 9ad3e56 commit 3e24d01

File tree

2 files changed

+98
-9
lines changed

2 files changed

+98
-9
lines changed

‎docs/archive.md‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,95 @@ nav_order: 15
55
---
66
Below are archived releases for RAPIDS Accelerator for Apache Spark.
77

8+
## Release v24.10.0
9+
### Hardware Requirements:
10+
11+
The plugin is tested on the following architectures:
12+
13+
GPU Models: NVIDIA V100, T4, A10/A100, L4 and H100 GPUs
14+
15+
### Software Requirements:
16+
17+
OS: Ubuntu 20.04, Ubuntu 22.04, CentOS 7, or Rocky Linux 8
18+
19+
NVIDIA Driver*: R470+
20+
21+
Runtime:
22+
Scala 2.12, 2.13
23+
Python, Java Virtual Machine (JVM) compatible with your spark-version.
24+
25+
* Check the Spark documentation for Python and Java version compatibility with your specific
26+
Spark version. For instance, visit `https://spark.apache.org/docs/3.4.1` for Spark 3.4.1.
27+
28+
Supported Spark versions:
29+
Apache Spark 3.2.0, 3.2.1, 3.2.2, 3.2.3, 3.2.4
30+
Apache Spark 3.3.0, 3.3.1, 3.3.2, 3.3.3, 3.3.4
31+
Apache Spark 3.4.0, 3.4.1, 3.4.2, 3.4.3
32+
Apache Spark 3.5.0, 3.5.1, 3.5.2
33+
34+
Supported Databricks runtime versions for Azure and AWS:
35+
Databricks 11.3 ML LTS (GPU, Scala 2.12, Spark 3.3.0)
36+
Databricks 12.2 ML LTS (GPU, Scala 2.12, Spark 3.3.2)
37+
Databricks 13.3 ML LTS (GPU, Scala 2.12, Spark 3.4.1)
38+
39+
Supported Dataproc versions (Debian/Ubuntu/Rocky):
40+
GCP Dataproc 2.1
41+
GCP Dataproc 2.2
42+
43+
Supported Dataproc Serverless versions:
44+
Spark runtime 1.1 LTS
45+
Spark runtime 2.0
46+
Spark runtime 2.1
47+
Spark runtime 2.2
48+
49+
*Some hardware may have a minimum driver version greater than R470. Check the GPU spec sheet
50+
for your hardware's minimum driver version.
51+
52+
*For Cloudera and EMR support, please refer to the
53+
[Distributions](https://docs.nvidia.com/spark-rapids/user-guide/latest/faq.html#which-distributions-are-supported) section of the FAQ.
54+
55+
### RAPIDS Accelerator's Support Policy for Apache Spark
56+
The RAPIDS Accelerator maintains support for Apache Spark versions available for download from [Apache Spark](https://spark.apache.org/downloads.html)
57+
58+
### Download RAPIDS Accelerator for Apache Spark v24.10.0
59+
60+
| Processor | Scala Version | Download Jar | Download Signature |
61+
|-----------|---------------|--------------|--------------------|
62+
| x86_64 | Scala 2.12 | [RAPIDS Accelerator v24.10.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/24.10.0/rapids-4-spark_2.12-24.10.0.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/24.10.0/rapids-4-spark_2.12-24.10.0.jar.asc) |
63+
| x86_64 | Scala 2.13 | [RAPIDS Accelerator v24.10.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/24.10.0/rapids-4-spark_2.13-24.10.0.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/24.10.0/rapids-4-spark_2.13-24.10.0.jar.asc) |
64+
| arm64 | Scala 2.12 | [RAPIDS Accelerator v24.10.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/24.10.0/rapids-4-spark_2.12-24.10.0-cuda11-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/24.10.0/rapids-4-spark_2.12-24.10.0-cuda11-arm64.jar.asc) |
65+
| arm64 | Scala 2.13 | [RAPIDS Accelerator v24.10.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/24.10.0/rapids-4-spark_2.13-24.10.0-cuda11-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/24.10.0/rapids-4-spark_2.13-24.10.0-cuda11-arm64.jar.asc) |
66+
67+
This package is built against CUDA 11.8. It is tested on V100, T4, A10, A100, L4 and H100 GPUs with
68+
CUDA 11.8 through CUDA 12.0.
69+
70+
### Verify signature
71+
* Download the [PUB_KEY](https://keys.openpgp.org/[email protected]).
72+
* Import the public key: `gpg --import PUB_KEY`
73+
* Verify the signature for Scala 2.12 jar:
74+
`gpg --verify rapids-4-spark_2.12-24.10.0.jar.asc rapids-4-spark_2.12-24.10.0.jar`
75+
* Verify the signature for Scala 2.13 jar:
76+
`gpg --verify rapids-4-spark_2.13-24.10.0.jar.asc rapids-4-spark_2.13-24.10.0.jar`
77+
78+
The output of signature verify:
79+
80+
gpg: Good signature from "NVIDIA Spark (For the signature of spark-rapids release jars) <[email protected]>"
81+
82+
### Release Notes
83+
* Optimize scheduling policy for GPU Semaphore
84+
* Support distinct join for right outer joins
85+
* Support MinBy and MaxBy for non-float ordering
86+
* Support ArrayJoin expression
87+
* Optimize Expand and Aggregate expression performance
88+
* Improve JSON related expressions
89+
* For updates on RAPIDS Accelerator Tools, please visit [this link](https://github.com/NVIDIA/spark-rapids-tools/releases)
90+
91+
Note: There is a known issue in the 24.10.0 release when decompressing gzip files on H100 GPUs.
92+
Please find more details in [issue-16661](https://github.com/rapidsai/cudf/issues/16661).
93+
94+
For a detailed list of changes, please refer to the
95+
[CHANGELOG](https://github.com/NVIDIA/spark-rapids/blob/main/CHANGELOG.md).
96+
897
## Release v24.08.1
998
### Hardware Requirements:
1099

‎docs/download.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ cuDF jar, that is either preinstalled in the Spark classpath on all nodes or sub
1818
that uses the RAPIDS Accelerator For Apache Spark. See the [getting-started
1919
guide](https://docs.nvidia.com/spark-rapids/user-guide/latest/getting-started/overview.html) for more details.
2020

21-
## Release v24.10.0
21+
## Release v24.10.1
2222
### Hardware Requirements:
2323

2424
The plugin is tested on the following architectures:
@@ -68,14 +68,14 @@ for your hardware's minimum driver version.
6868
### RAPIDS Accelerator's Support Policy for Apache Spark
6969
The RAPIDS Accelerator maintains support for Apache Spark versions available for download from [Apache Spark](https://spark.apache.org/downloads.html)
7070

71-
### Download RAPIDS Accelerator for Apache Spark v24.10.0
71+
### Download RAPIDS Accelerator for Apache Spark v24.10.1
7272

7373
| Processor | Scala Version | Download Jar | Download Signature |
7474
|-----------|---------------|--------------|--------------------|
75-
| x86_64 | Scala 2.12 | [RAPIDS Accelerator v24.10.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/24.10.0/rapids-4-spark_2.12-24.10.0.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/24.10.0/rapids-4-spark_2.12-24.10.0.jar.asc) |
76-
| x86_64 | Scala 2.13 | [RAPIDS Accelerator v24.10.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/24.10.0/rapids-4-spark_2.13-24.10.0.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/24.10.0/rapids-4-spark_2.13-24.10.0.jar.asc) |
77-
| arm64 | Scala 2.12 | [RAPIDS Accelerator v24.10.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/24.10.0/rapids-4-spark_2.12-24.10.0-cuda11-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/24.10.0/rapids-4-spark_2.12-24.10.0-cuda11-arm64.jar.asc) |
78-
| arm64 | Scala 2.13 | [RAPIDS Accelerator v24.10.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/24.10.0/rapids-4-spark_2.13-24.10.0-cuda11-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/24.10.0/rapids-4-spark_2.13-24.10.0-cuda11-arm64.jar.asc) |
75+
| x86_64 | Scala 2.12 | [RAPIDS Accelerator v24.10.1](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/24.10.1/rapids-4-spark_2.12-24.10.1.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/24.10.1/rapids-4-spark_2.12-24.10.1.jar.asc) |
76+
| x86_64 | Scala 2.13 | [RAPIDS Accelerator v24.10.1](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/24.10.1/rapids-4-spark_2.13-24.10.1.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/24.10.1/rapids-4-spark_2.13-24.10.1.jar.asc) |
77+
| arm64 | Scala 2.12 | [RAPIDS Accelerator v24.10.1](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/24.10.1/rapids-4-spark_2.12-24.10.1-cuda11-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/24.10.1/rapids-4-spark_2.12-24.10.1-cuda11-arm64.jar.asc) |
78+
| arm64 | Scala 2.13 | [RAPIDS Accelerator v24.10.1](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/24.10.1/rapids-4-spark_2.13-24.10.1-cuda11-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/24.10.1/rapids-4-spark_2.13-24.10.1-cuda11-arm64.jar.asc) |
7979

8080
This package is built against CUDA 11.8. It is tested on V100, T4, A10, A100, L4 and H100 GPUs with
8181
CUDA 11.8 through CUDA 12.0.
@@ -84,9 +84,9 @@ CUDA 11.8 through CUDA 12.0.
8484
* Download the [PUB_KEY](https://keys.openpgp.org/[email protected]).
8585
* Import the public key: `gpg --import PUB_KEY`
8686
* Verify the signature for Scala 2.12 jar:
87-
`gpg --verify rapids-4-spark_2.12-24.10.0.jar.asc rapids-4-spark_2.12-24.10.0.jar`
87+
`gpg --verify rapids-4-spark_2.12-24.10.1.jar.asc rapids-4-spark_2.12-24.10.1.jar`
8888
* Verify the signature for Scala 2.13 jar:
89-
`gpg --verify rapids-4-spark_2.13-24.10.0.jar.asc rapids-4-spark_2.13-24.10.0.jar`
89+
`gpg --verify rapids-4-spark_2.13-24.10.1.jar.asc rapids-4-spark_2.13-24.10.1.jar`
9090

9191
The output of signature verify:
9292

@@ -101,7 +101,7 @@ The output of signature verify:
101101
* Improve JSON related expressions
102102
* For updates on RAPIDS Accelerator Tools, please visit [this link](https://github.com/NVIDIA/spark-rapids-tools/releases)
103103

104-
Note: There is a known issue in the 24.10.0 release when decompressing gzip files on H100 GPUs.
104+
Note: There is a known issue in the 24.10.1 release when decompressing gzip files on H100 GPUs.
105105
Please find more details in [issue-16661](https://github.com/rapidsai/cudf/issues/16661).
106106

107107
For a detailed list of changes, please refer to the

0 commit comments

Comments
 (0)