Skip to content

Use a shallow git clone on Jenkins when possible for main build#24201

Merged
apupier merged 1 commit into
apache:mainfrom
apupier:reduceWorkspaceSizeAndSpeedupCheckoutOnJenkins
Jun 24, 2026
Merged

Use a shallow git clone on Jenkins when possible for main build#24201
apupier merged 1 commit into
apache:mainfrom
apupier:reduceWorkspaceSizeAndSpeedupCheckoutOnJenkins

Conversation

@apupier

@apupier apupier commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

there are currently issues with disk space available on the Jenkins nodes.
Camel is one of the most consuming one. Each of its workspace for Camel core is taking 4.4G, it includes 1.4G for the .git folder. Using a shallow clone should reduce the 1.4G to 63M and gain few minutes to the build too.

Co-authored-by: IBM Bob IDE 1.0.3

Description

Target

  • I checked that the commit is targeting the correct branch (Camel 4 uses the main branch)

Tracking

  • If this is a large change, bug fix, or code improvement, I checked there is a JIRA issue filed for the change (usually before you start working on it).

Apache Camel coding standards and style

  • I checked that each commit in the pull request has a meaningful subject line and body.
  • I have run mvn clean install -DskipTests locally from root folder and I have committed all auto-generated changes.

@apupier

apupier commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

before merging I would like to test it on the CI by changing manually. I will likely do it tomorrow

@github-actions

Copy link
Copy Markdown
Contributor

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

  • First-time contributors require MANUAL approval for the GitHub Actions to run
  • You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
  • You can label PRs using skip-tests and test-dependents to fine-tune the checks executed by this PR.
  • Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

@github-actions github-actions Bot added the ci label Jun 23, 2026
@apupier apupier force-pushed the reduceWorkspaceSizeAndSpeedupCheckoutOnJenkins branch from 91844e9 to 8aa8c31 Compare June 23, 2026 14:46

@davsclaus davsclaus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this, @apupier — reducing the .git footprint from 1.4GB to ~63MB per workspace is a meaningful improvement for the Jenkins nodes.

A few observations from reviewing the diff:

Sonar/coverage case is correctly handled — the full clone exception for ubuntu-avx + jdk_21_latest ensures SonarCloud's SCM blame sensor (which uses git blame) will continue to work. Good catch.

No git-history-dependent Maven plugins found — I checked the root pom.xml for plugins like git-commit-id-plugin or buildnumber-maven-plugin and found none, so depth: 1 won't break the Maven build itself.

Minor observation on Jenkinsfile.jdk26 — this file unconditionally uses shallow clone for all platforms, which is safe since there's no Sonar/coverage stage. If Sonar analysis is ever added for JDK 26 builds, someone would need to add the full-clone exception there too. A brief comment noting this could help future maintainers.

Indentation in Jenkinsfile.jdk26 — the checkout([...]) block has slightly inconsistent indentation compared to the surrounding code and the main Jenkinsfile change. Very minor/cosmetic.

Overall the approach looks sound. I see you want to validate on Jenkins first before merging — makes sense given this is infrastructure.

This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.

Comment thread Jenkinsfile.jdk26
@apupier

apupier commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

tested with JDk 26, the exact git commands used on Jenkisn are:
git init
git fetch --no-tags --force --progress --depth=1 -- https://github.com/apache/camel.git +refs/heads/:refs/remotes/origin/

which is fecthing all branches, causing the .git to be at 376M, instead of 63M with a pure git clone --depth=1 https://github.com/apache/camel which is no downloading branches. To see if we can configure that safely. Can also be improved in another iteration.

remains the conditional to test too on the Camel main repo

there are currently issues with disk space available on the Jenkins
nodes.
Camel is one of the most consuming one. Each of its workspace for Camel
core is taking 4.4G, it includes 1.4G for the .git folder.
Using a shallow clone should reduce the 1.4G to 63M and gain few minutes
to the build too.

Co-authored-by: IBM Bob IDE 1.0.3
Signed-off-by: Aurélien Pupier <apupier@ibm.com>
@apupier apupier force-pushed the reduceWorkspaceSizeAndSpeedupCheckoutOnJenkins branch from 8aa8c31 to 224c0cc Compare June 24, 2026 07:40
@apupier

apupier commented Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

remains the conditional to test too on the Camel main repo

tested with https://ci-builds.apache.org/job/Camel/job/Camel%20Core%20(Build%20and%20test)/job/main/1943/

and validating that all .git in saved workspaces get down to 365M (no more 376M as I cleaned some old branches on apache/camel fork) vs 1.4G previously and for the remaining JDK 21/Ubuntu

I propose to merge this version and look to reduce even more to the 63M potential target in another iteration (CI is currently crowded so complicated to test too many things in //) it will surely be by using honorRefSpec parameter.
The current state already allows to save 1.1x (4 (matrix config) + 1 (JDK26)) x 5 (number of history build kept) = 27.5G of disk space

@apupier apupier marked this pull request as ready for review June 24, 2026 12:27
@apupier apupier requested a review from davsclaus June 24, 2026 12:27
@apupier apupier changed the title [DO NOT MERGE] Use a shallow git clone on Jenkins when possible for main build Use a shallow git clone on Jenkins when possible for main build Jun 24, 2026
@apupier apupier merged commit 9385377 into apache:main Jun 24, 2026
2 checks passed
@apupier apupier deleted the reduceWorkspaceSizeAndSpeedupCheckoutOnJenkins branch June 24, 2026 12:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants