Java CI

[HUDI-8787] Improve compaction performance by reducing unnecessary disk access #37587

Triggered via pull request December 20, 2024 10:12

TheR1sing3un

opened #12531

TheR1sing3un:improve_compaction_by_reduce_disk_visit2

Status Success

Total duration 1h 8m 18s

Artifacts –

bot.yml

on: pull_request

validate-source

2m 42s

Matrix: build-flink-java17

Matrix: build-spark-java17

Matrix: docker-java17-test

Matrix: integration-tests

Matrix: test-flink

Matrix: test-hudi-hadoop-mr-and-hudi-java-client

Matrix: test-spark-java-tests

Matrix: test-spark-java11-17-java-tests

Matrix: test-spark-java11-17-scala-tests

Matrix: test-spark-java17-java-tests

Matrix: test-spark-java17-scala-tests

Matrix: test-spark-scala-tests

Matrix: validate-bundles

Annotations

10 errors and 281 warnings

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

Trying to access closed classloader. Please check if you store classloaders directly or indirectly in static fields. If the stacktrace suggests that the leak occurs in a third party library and cannot be fixed immediately, you can disable this check with the configuration 'classloader.check-leaked-classloader'.

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

test-spark-java17-java-tests (scala-2.12, spark3.3, hudi-spark-datasource/hudi-spark3.3.x)

Cannot resolve conflicts for overlapping writes

test-spark-java-tests (scala-2.12, spark3.3, hudi-spark-datasource/hudi-spark3.3.x)

Cannot resolve conflicts for overlapping writes

test-spark-java11-17-java-tests (scala-2.13, spark3.5, hudi-spark-datasource/hudi-spark3.5.x)

Cannot resolve conflicts for overlapping writes

test-spark-java-tests (scala-2.13, spark3.5, hudi-spark-datasource/hudi-spark3.5.x)

Cannot resolve conflicts for overlapping writes

test-spark-java17-java-tests (scala-2.12, spark3.4, hudi-spark-datasource/hudi-spark3.4.x)

Cannot resolve conflicts for overlapping writes

test-spark-java-tests (scala-2.12, spark3.4, hudi-spark-datasource/hudi-spark3.4.x)

Cannot resolve conflicts for overlapping writes

test-spark-java11-17-java-tests (scala-2.12, spark3.5, hudi-spark-datasource/hudi-spark3.5.x)

Cannot resolve conflicts for overlapping writes

validate-source

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

build-flink-java17 (scala-2.12, flink1.20, 1.11.3)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-flink (flink1.18, 1.11.1)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-flink (flink1.15, 1.10.0)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-flink (flink1.16, 1.11.1)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-flink (flink1.17, 1.11.1)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-flink (flink1.19, 1.11.1)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-spark-scala-tests (scala-2.12, spark3.5, hudi-spark-datasource/hudi-spark3.5.x)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-spark-java-tests (scala-2.12, spark3.5, hudi-spark-datasource/hudi-spark3.5.x)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

build-spark-java17 (scala-2.13, spark3.5, hudi-spark-datasource/hudi-spark3.5.x)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

build-spark-java17 (scala-2.12, spark3.3, hudi-spark-datasource/hudi-spark3.3.x)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

build-spark-java17 (scala-2.12, spark3.4, hudi-spark-datasource/hudi-spark3.4.x)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

build-spark-java17 (scala-2.12, spark3.5, hudi-spark-datasource/hudi-spark3.5.x)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

docker-java17-test (scala-2.13, flink1.20, spark3.5, spark3.5.0)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

docker-java17-test (scala-2.13, flink1.20, spark3.5, spark3.5.0)

docker_test_java17.sh Building Hudi with Java 8

docker-java17-test (scala-2.13, flink1.20, spark3.5, spark3.5.0)

docker_test_java17.sh Done building Hudi with Java 8

docker-java17-test (scala-2.13, flink1.20, spark3.5, spark3.5.0)

docker_test_java17.sh copying hadoop conf

docker-java17-test (scala-2.13, flink1.20, spark3.5, spark3.5.0)

docker_test_java17.sh starting hadoop hdfs

docker-java17-test (scala-2.13, flink1.20, spark3.5, spark3.5.0)

docker_test_java17.sh starting datanode:1

docker-java17-test (scala-2.13, flink1.20, spark3.5, spark3.5.0)

docker_test_java17.sh starting datanode:2

docker-java17-test (scala-2.13, flink1.20, spark3.5, spark3.5.0)

docker_test_java17.sh starting datanode:3

docker-java17-test (scala-2.13, flink1.20, spark3.5, spark3.5.0)

docker_test_java17.sh starting hadoop hdfs, hdfs report

docker-java17-test (scala-2.13, flink1.20, spark3.5, spark3.5.0)

docker_test_java17.sh Running tests with Java 17

docker-java17-test (scala-2.13, flink1.20, spark3.5, spark3.5.0)

docker_test_java17.sh run_docker_tests Running Hudi maven tests on Docker

docker-java17-test (scala-2.12, flink1.20, spark3.5, spark3.5.0)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

docker-java17-test (scala-2.12, flink1.20, spark3.5, spark3.5.0)

docker_test_java17.sh Building Hudi with Java 8

docker-java17-test (scala-2.12, flink1.20, spark3.5, spark3.5.0)

docker_test_java17.sh Done building Hudi with Java 8

docker-java17-test (scala-2.12, flink1.20, spark3.5, spark3.5.0)

docker_test_java17.sh copying hadoop conf

docker-java17-test (scala-2.12, flink1.20, spark3.5, spark3.5.0)

docker_test_java17.sh starting hadoop hdfs

docker-java17-test (scala-2.12, flink1.20, spark3.5, spark3.5.0)

docker_test_java17.sh starting datanode:1

docker-java17-test (scala-2.12, flink1.20, spark3.5, spark3.5.0)

docker_test_java17.sh starting datanode:2

docker-java17-test (scala-2.12, flink1.20, spark3.5, spark3.5.0)

docker_test_java17.sh starting datanode:3

docker-java17-test (scala-2.12, flink1.20, spark3.5, spark3.5.0)

docker_test_java17.sh starting hadoop hdfs, hdfs report

docker-java17-test (scala-2.12, flink1.20, spark3.5, spark3.5.0)

docker_test_java17.sh Running tests with Java 17

docker-java17-test (scala-2.12, flink1.20, spark3.5, spark3.5.0)

docker_test_java17.sh run_docker_tests Running Hudi maven tests on Docker

docker-java17-test (scala-2.12, flink1.20, spark3.4, spark3.4.0)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

docker-java17-test (scala-2.12, flink1.20, spark3.4, spark3.4.0)

docker_test_java17.sh Building Hudi with Java 8

docker-java17-test (scala-2.12, flink1.20, spark3.4, spark3.4.0)

docker_test_java17.sh Done building Hudi with Java 8

docker-java17-test (scala-2.12, flink1.20, spark3.4, spark3.4.0)

docker_test_java17.sh copying hadoop conf

docker-java17-test (scala-2.12, flink1.20, spark3.4, spark3.4.0)

docker_test_java17.sh starting hadoop hdfs

docker-java17-test (scala-2.12, flink1.20, spark3.4, spark3.4.0)

docker_test_java17.sh starting datanode:1

docker-java17-test (scala-2.12, flink1.20, spark3.4, spark3.4.0)

docker_test_java17.sh starting datanode:2

docker-java17-test (scala-2.12, flink1.20, spark3.4, spark3.4.0)

docker_test_java17.sh starting datanode:3

docker-java17-test (scala-2.12, flink1.20, spark3.4, spark3.4.0)

docker_test_java17.sh starting hadoop hdfs, hdfs report

docker-java17-test (scala-2.12, flink1.20, spark3.4, spark3.4.0)

docker_test_java17.sh Running tests with Java 17

docker-java17-test (scala-2.12, flink1.20, spark3.4, spark3.4.0)

docker_test_java17.sh run_docker_tests Running Hudi maven tests on Docker

test-hudi-hadoop-mr-and-hudi-java-client (scala-2.12, spark3.5, flink1.20)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-flink (flink1.14, 1.10.0)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating spark & hadoop-mr bundle

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh setting up hive metastore for spark & hadoop-mr bundles validation

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh Writing sample data via Spark DataSource and run Hive Sync...

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

Use default java runtime under /opt/java/openjdk

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh spark & hadoop-mr bundles validation was successful.

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh done validating spark & hadoop-mr bundle

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating utilities bundle

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh running deltastreamer

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh done with deltastreamer

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating deltastreamer in spark shell

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating spark & hadoop-mr bundle

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh setting up hive metastore for spark & hadoop-mr bundles validation

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh Writing sample data via Spark DataSource and run Hive Sync...

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

Use default java runtime under /opt/java/openjdk

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh spark & hadoop-mr bundles validation was successful.

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh done validating spark & hadoop-mr bundle

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating utilities bundle

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh running deltastreamer

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh done with deltastreamer

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating deltastreamer in spark shell

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating spark & hadoop-mr bundle

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh setting up hive metastore for spark & hadoop-mr bundles validation

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh Writing sample data via Spark DataSource and run Hive Sync...

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

Use default java runtime under /opt/java/openjdk

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh spark & hadoop-mr bundles validation was successful.

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh done validating spark & hadoop-mr bundle

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating utilities bundle

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh running deltastreamer

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh done with deltastreamer

validate-bundles (scala-2.13, flink1.19, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating deltastreamer in spark shell

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh validating spark & hadoop-mr bundle

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh setting up hive metastore for spark & hadoop-mr bundles validation

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh Writing sample data via Spark DataSource and run Hive Sync...

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

Use default java runtime under /opt/java/openjdk

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh spark & hadoop-mr bundles validation was successful.

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh done validating spark & hadoop-mr bundle

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh validating utilities bundle

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh running deltastreamer

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh done with deltastreamer

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh validating deltastreamer in spark shell

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh validating spark & hadoop-mr bundle

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh setting up hive metastore for spark & hadoop-mr bundles validation

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh Writing sample data via Spark DataSource and run Hive Sync...

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

Use default java runtime under /opt/java/openjdk

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh spark & hadoop-mr bundles validation was successful.

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh done validating spark & hadoop-mr bundle

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh validating utilities bundle

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh running deltastreamer

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh done with deltastreamer

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh validating deltastreamer in spark shell

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh validating spark & hadoop-mr bundle

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh setting up hive metastore for spark & hadoop-mr bundles validation

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh Writing sample data via Spark DataSource and run Hive Sync...

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

Use default java runtime under /opt/java/openjdk

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh spark & hadoop-mr bundles validation was successful.

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh done validating spark & hadoop-mr bundle

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh validating utilities bundle

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh running deltastreamer

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh done with deltastreamer

validate-bundles (scala-2.13, flink1.20, 1.11.3, spark3.5, spark3.5.1)

validate.sh validating deltastreamer in spark shell

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating spark & hadoop-mr bundle

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh setting up hive metastore for spark & hadoop-mr bundles validation

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh Writing sample data via Spark DataSource and run Hive Sync...

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

Use default java runtime under /opt/java/openjdk

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh spark & hadoop-mr bundles validation was successful.

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh done validating spark & hadoop-mr bundle

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating utilities bundle

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh running deltastreamer

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh done with deltastreamer

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating deltastreamer in spark shell

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating spark & hadoop-mr bundle

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh setting up hive metastore for spark & hadoop-mr bundles validation

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh Writing sample data via Spark DataSource and run Hive Sync...

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

Use default java runtime under /opt/java/openjdk

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh spark & hadoop-mr bundles validation was successful.

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh done validating spark & hadoop-mr bundle

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating utilities bundle

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh running deltastreamer

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh done with deltastreamer

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating deltastreamer in spark shell

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating spark & hadoop-mr bundle

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh setting up hive metastore for spark & hadoop-mr bundles validation

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh Writing sample data via Spark DataSource and run Hive Sync...

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

Use default java runtime under /opt/java/openjdk

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh spark & hadoop-mr bundles validation was successful.

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh done validating spark & hadoop-mr bundle

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating utilities bundle

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh running deltastreamer

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh done with deltastreamer

validate-bundles (scala-2.13, flink1.18, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating deltastreamer in spark shell

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh setting up hive metastore for spark & hadoop-mr bundles validation

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh Writing sample data via Spark DataSource and run Hive Sync...

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh Query and validate the results using Spark SQL

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh Query and validate the results using HiveQL

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

Use default java runtime under /opt/java/openjdk

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh spark & hadoop-mr bundles validation was successful.

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh done validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh skip validating utilities bundle for non-spark3.5 build

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh validating utilities slim bundle

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh setting up hive metastore for spark & hadoop-mr bundles validation

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh Writing sample data via Spark DataSource and run Hive Sync...

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh Query and validate the results using Spark SQL

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh Query and validate the results using HiveQL

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

Use default java runtime under /opt/java/openjdk

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh spark & hadoop-mr bundles validation was successful.

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh done validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh skip validating utilities bundle for non-spark3.5 build

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh validating utilities slim bundle

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh setting up hive metastore for spark & hadoop-mr bundles validation

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh Writing sample data via Spark DataSource and run Hive Sync...

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh Query and validate the results using Spark SQL

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh Query and validate the results using HiveQL

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

Use default java runtime under /opt/java/openjdk

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh spark & hadoop-mr bundles validation was successful.

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh done validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh skip validating utilities bundle for non-spark3.5 build

validate-bundles (scala-2.12, flink1.15, 1.10.0, spark3.3, spark3.3.4)

validate.sh validating utilities slim bundle

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh setting up hive metastore for spark & hadoop-mr bundles validation

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh Writing sample data via Spark DataSource and run Hive Sync...

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh Query and validate the results using Spark SQL

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh Query and validate the results using HiveQL

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

Use default java runtime under /opt/java/openjdk

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh spark & hadoop-mr bundles validation was successful.

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh done validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh skip validating utilities bundle for non-spark3.5 build

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh validating utilities slim bundle

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh setting up hive metastore for spark & hadoop-mr bundles validation

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh Writing sample data via Spark DataSource and run Hive Sync...

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh Query and validate the results using Spark SQL

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh Query and validate the results using HiveQL

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

Use default java runtime under /opt/java/openjdk

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh spark & hadoop-mr bundles validation was successful.

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh done validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh skip validating utilities bundle for non-spark3.5 build

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh validating utilities slim bundle

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh setting up hive metastore for spark & hadoop-mr bundles validation

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh Writing sample data via Spark DataSource and run Hive Sync...

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh Query and validate the results using Spark SQL

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh Query and validate the results using HiveQL

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

Use default java runtime under /opt/java/openjdk

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh spark & hadoop-mr bundles validation was successful.

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh done validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh skip validating utilities bundle for non-spark3.5 build

validate-bundles (scala-2.12, flink1.14, 1.10.0, spark3.3, spark3.3.4)

validate.sh validating utilities slim bundle

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh setting up hive metastore for spark & hadoop-mr bundles validation

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh Writing sample data via Spark DataSource and run Hive Sync...

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh Query and validate the results using Spark SQL

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh Query and validate the results using HiveQL

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

Use default java runtime under /opt/java/openjdk

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh spark & hadoop-mr bundles validation was successful.

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh done validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh skip validating utilities bundle for non-spark3.5 build

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh validating utilities slim bundle

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh setting up hive metastore for spark & hadoop-mr bundles validation

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh Writing sample data via Spark DataSource and run Hive Sync...

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh Query and validate the results using Spark SQL

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh Query and validate the results using HiveQL

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

Use default java runtime under /opt/java/openjdk

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh spark & hadoop-mr bundles validation was successful.

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh done validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh skip validating utilities bundle for non-spark3.5 build

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh validating utilities slim bundle

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh setting up hive metastore for spark & hadoop-mr bundles validation

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh Writing sample data via Spark DataSource and run Hive Sync...

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh Query and validate the results using Spark SQL

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh Query and validate the results using HiveQL

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

Use default java runtime under /opt/java/openjdk

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh spark & hadoop-mr bundles validation was successful.

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh done validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh skip validating utilities bundle for non-spark3.5 build

validate-bundles (scala-2.12, flink1.16, 1.11.1, spark3.4, spark3.4.3)

validate.sh validating utilities slim bundle

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh setting up hive metastore for spark & hadoop-mr bundles validation

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh Writing sample data via Spark DataSource and run Hive Sync...

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh Query and validate the results using Spark SQL

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh Query and validate the results using HiveQL

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

Use default java runtime under /opt/java/openjdk

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh spark & hadoop-mr bundles validation was successful.

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh done validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating utilities bundle

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh running deltastreamer

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh setting up hive metastore for spark & hadoop-mr bundles validation

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh Writing sample data via Spark DataSource and run Hive Sync...

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh Query and validate the results using Spark SQL

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh Query and validate the results using HiveQL

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

Use default java runtime under /opt/java/openjdk

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh spark & hadoop-mr bundles validation was successful.

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh done validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating utilities bundle

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh running deltastreamer

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh setting up hive metastore for spark & hadoop-mr bundles validation

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh Writing sample data via Spark DataSource and run Hive Sync...

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh Query and validate the results using Spark SQL

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh Query and validate the results using HiveQL

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

Use default java runtime under /opt/java/openjdk

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh spark & hadoop-mr bundles validation was successful.

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh done validating spark & hadoop-mr bundle

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh validating utilities bundle

validate-bundles (scala-2.12, flink1.17, 1.11.1, spark3.5, spark3.5.1)

validate.sh running deltastreamer

integration-tests (spark3.5, spark-3.5.3/spark-3.5.3-bin-hadoop3.tgz)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-spark-java17-scala-tests (scala-2.12, spark3.3, hudi-spark-datasource/hudi-spark3.3.x)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-spark-java17-scala-tests (scala-2.12, spark3.4, hudi-spark-datasource/hudi-spark3.4.x)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-flink (flink1.20, 1.11.3)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-spark-java11-17-scala-tests (scala-2.13, spark3.5, hudi-spark-datasource/hudi-spark3.5.x)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-spark-java11-17-scala-tests (scala-2.12, spark3.5, hudi-spark-datasource/hudi-spark3.5.x)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-spark-scala-tests (scala-2.13, spark3.5, hudi-spark-datasource/hudi-spark3.5.x)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-spark-scala-tests (scala-2.12, spark3.3, hudi-spark-datasource/hudi-spark3.3.x)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-spark-scala-tests (scala-2.12, spark3.4, hudi-spark-datasource/hudi-spark3.4.x)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-spark-java17-java-tests (scala-2.12, spark3.3, hudi-spark-datasource/hudi-spark3.3.x)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-spark-java-tests (scala-2.12, spark3.3, hudi-spark-datasource/hudi-spark3.3.x)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-spark-java11-17-java-tests (scala-2.13, spark3.5, hudi-spark-datasource/hudi-spark3.5.x)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-spark-java-tests (scala-2.13, spark3.5, hudi-spark-datasource/hudi-spark3.5.x)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-spark-java17-java-tests (scala-2.12, spark3.4, hudi-spark-datasource/hudi-spark3.4.x)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-spark-java-tests (scala-2.12, spark3.4, hudi-spark-datasource/hudi-spark3.4.x)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

test-spark-java11-17-java-tests (scala-2.12, spark3.5, hudi-spark-datasource/hudi-spark3.5.x)

ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[HUDI-8787] Improve compaction performance by reducing unnecessary disk access #37587

Summary