Skip to content

Commit 345e23f

Browse files
andygroveclaude
andauthored
build: optimize CI cache usage and add fast lint gate (#3251)
* build: optimize CI cache usage and add fast lint gate This PR addresses cache storage approaching its 10GB limit by: 1. Cache optimization (saves ~2+ GB): - Remove Java version from cargo cache key (Rust target is JDK-independent) - Use actions/cache/restore + actions/cache/save pattern - Only save cache on main branch, not on PRs 2. Reduce Rust test matrix: - Consolidate from 2 jobs (Java 11 + Java 17) to 1 job (Java 17) - Rust code is JDK-independent, so no coverage lost 3. Add fast lint gate (~30 seconds): - New lint job runs cargo fmt --check before expensive builds - build-native and linux-test-rust depend on lint passing - Fail fast on formatting errors instead of waiting 5-10 minutes - macOS lint runs on ubuntu-latest for cost efficiency Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add missing datafusion-datasource dependency The csv_scan.rs file added in #3044 uses datafusion_datasource but the dependency was not added to core/Cargo.toml. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * build: merge TPC-DS/TPC-H correctness tests into pr_build_linux These workflows verify that benchmark queries produce correct results (not actual performance benchmarks), so they can use the CI build profile and share the native library artifact from build-native. Changes: - Add verify-benchmark-results-tpch job to pr_build_linux - Add verify-benchmark-results-tpcds job to pr_build_linux (3 join strategies) - Delete standalone benchmark-tpcds.yml and benchmark-tpch.yml workflows - Jobs reuse native library artifact instead of rebuilding This eliminates 4+ redundant native builds per PR. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: compile test classes before generating TPC data The GenTPCHData and GenTPCDSData classes are test classes that need to be compiled before running exec:java. Added a build step to compile the project (including test classes) before data generation. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
1 parent c9af2c6 commit 345e23f

File tree

6 files changed

+230
-297
lines changed

6 files changed

+230
-297
lines changed

.github/actions/rust-test/action.yaml

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -21,11 +21,7 @@ description: "Run Rust tests"
2121
runs:
2222
using: "composite"
2323
steps:
24-
- name: Check Cargo fmt
25-
shell: bash
26-
run: |
27-
cd native
28-
cargo fmt --all -- --check --color=never
24+
# Note: cargo fmt check is now handled by the lint job that gates this workflow
2925

3026
- name: Check Cargo clippy
3127
shell: bash

.github/workflows/benchmark-tpcds.yml

Lines changed: 0 additions & 155 deletions
This file was deleted.

.github/workflows/benchmark-tpch.yml

Lines changed: 0 additions & 119 deletions
This file was deleted.

0 commit comments

Comments
 (0)