[VL] Add Scala 2.13 support #6326
Conversation
|
Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues? https://github.com/apache/incubator-gluten/issues Then could you also rename commit message and pull request title in the following format? See also: |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
@zhztheplayer Could you please review this PR? Thanks! |
|
Prior to Spark 4.0, Scala 2.13 was not the default Scala version, and I personally do not recommend supporting it for Spark 3.x. |
|
Thanks @LuciferYang, I think Scala 2.13 support is already added for CH backend, plan was to extend to VL backend as well, since the build fails when you enable FYI, the support for Scala 2.13 was added here already - #6311 cc: @zhztheplayer |
|
@Preetesh2110 please take a look at failing CI and maybe also add a CI option to build with Scala 2.13 |
Got |
|
Run Gluten Clickhouse CI |
|
thanks @surnaik. fixed the scala Style issues. |
|
Do we need to have a test for Scala 2.13 (with Spark 3.5 probably) ? https://github.com/apache/incubator-gluten/blob/main/.github/workflows/velox_docker.yml |
|
Run Gluten Clickhouse CI |
|
Thanks for the feedback. I have added a workflow @zhztheplayer. |
|
Run Gluten Clickhouse CI |
.github/workflows/velox_docker.yml
Outdated
| - name: Build and Run unit test for Spark 3.5.1 with Scala 2.13 (other tests) | ||
| run: | | ||
| cd $GITHUB_WORKSPACE/ | ||
| export SPARK_SCALA_VERSION=2.13 | ||
| $MVN_CMD clean install -Pspark-3.5 -Pscala-2.13 -Pbackends-velox -Pceleborn -Piceberg -Pdelta -Pspark-ut -DargLine="-Dspark.test.home=$GITHUB_WORKSPACE//shims/spark35/spark_home/" -DtagsToExclude=org.apache.spark.tags.ExtendedSQLTest,org.apache.gluten.tags.UDFTest,org.apache.gluten.tags.SkipTestTags && \ | ||
| $MVN_CMD test -Pspark-3.5 -Pscala-2.13 -Pbackends-velox -Piceberg -Pdelta -DtagsToExclude=None -DtagsToInclude=org.apache.gluten.tags.UDFTest |
There was a problem hiding this comment.
Done. Thanks @zhztheplayer
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
@Preetesh2110 Please fix the CI failures |
What changes were proposed in this pull request?
Add Scala 2.13 support
(Fixes: #6316)
How was this patch tested?
Build locally
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)