Auto-detect gradle composite builds

[prabhu]

Composite builds are now auto-detected.

@malice00 Can you help test this PR?

[malice00]

Not right this minute, I have no access to a computer right now. But I can look it over in a couple of hours when I get back home.

[prabhu]

Merged so that this can be tested with a container image.

[malice00]

lgtm

[prabhu]

Thank you so much for checking! Is there a sample repo that we can use for repotests to demonstrate this?

[malice00]

I'd say elasticsearch, which is already in repotests -- we could probably add a run without the EnvVar for included build, or remove it altogether?

[prabhu]

Will test with elasticsearch tomorrow to see if the auto-detection is working correctly.

[malice00]

Will test with elasticsearch tomorrow to see if the auto-detection is working correctly.

With your changes I get the same results both with auto-detection and when explicitly setting GRADLE_INCLUDED_BUILDS!

[prabhu]

oh wow, thank you for checking! Next experiment, does the order of the included builds affect the precision of the generated sbom.

[nightm4re94]

@prabhu I believe the current approach might be a bit too eager. I have a bunch of commented-out lines in a settings.gradle file to toggle composite builds:

pluginManagement {
    repositories {
        // ...
    }
    resolutionStrategy {
        // ...
    }
    // toggle for a composite build with required plugins
    // includeBuild '../my-plugin1'
    // includeBuild '../my-plugin2'
}

// toggle for a composite build with required libraries
// includeBuild '../my-library1'
// includeBuild '../my-library2'

This results in the following output when running cdxgen on Jenkins (-> the required repositories are not checked out as siblings of the current project):

15:43:02  + cdxgen -o bom_my-project.json --author nightm4re --project-group de.gurkenlabs --project-name my-project --project-version latest -r true --deep --evidence
15:43:27  ╔═══════════════════════════════════════════════════════════════════════════════════════════════╗
15:43:27  ║                               ¤ Donate to the OWASP Foundation                                ║
15:43:27  ╟───────────────────────────────────────────────────────────────────────────────────────────────╢
15:43:27  ║ OWASP foundation relies on donations to fund our projects.                                    ║
15:43:27  ║ Donation link: https://owasp.org/donate/?reponame=www-project-cyclonedx&title=OWASP+CycloneDX ║
15:43:27  ╚═══════════════════════════════════════════════════════════════════════════════════════════════╝
15:43:27  
15:43:27  Scanning /var/jenkins_home/workspace/4_my-project_my-project_bom_main
15:43:27  Performing babel-based package usage analysis with source code at /var/jenkins_home/workspace/4_my-project_my-project_bom_main
15:43:35  Additional root projects: ://../my-plugin1 ://../my-plugin2 ://../my-library1 ://../my-library2.
15:43:35  Executing /var/jenkins_home/workspace/4_my-project_my-project_bom_main/gradlew with arguments --build-cache --console plain --no-parallel properties ://../my-plugin1:properties ://../my-plugin2:properties ://../my-library1:properties ://../my-library2:properties... in /var/jenkins_home/workspace/4_my-project_my-project_bom_main
15:46:06  *** GRADLE BUILD ERRORS ***
15:46:06    Downloading https://services.gradle.org/distributions/gradle-8.12-bin.zip
15:46:06    .....................10%......................20%......................30%......................40%......................50%......................60%......................70%......................80%......................90%......................100%
15:46:06    
15:46:06    Welcome to Gradle 8.12!
15:46:06    
15:46:06    Here are the highlights of this release:
15:46:06     - Enhanced error and warning reporting with the Problems API
15:46:06     - File-system watching support on Alpine Linux
15:46:06     - Build and test Swift 6 libraries and apps
15:46:06    
15:46:06    For more details see https://docs.gradle.org/8.12/release-notes.html
15:46:06    
15:46:06    Starting a Gradle Daemon (subsequent builds will be faster)
15:46:06    Configuration on demand is an incubating feature.
15:46:06    gradle build cache configuration (/var/jenkins_home/workspace/4_my-project_my-project_bom_main): local cache enabled: true, remote cache push enabled: false, remote cache enabled: true
15:46:06    > Task :buildSrc:extractPluginRequests FROM-CACHE
15:46:06    > Task :buildSrc:generatePluginAdapters FROM-CACHE
15:46:06    > Task :buildSrc:compileJava
15:46:06    > Task :buildSrc:compileGroovy
15:46:06    > Task :buildSrc:compileGroovyPlugins
15:46:06    > Task :buildSrc:pluginDescriptors
15:46:06    > Task :buildSrc:processResources
15:46:06    > Task :buildSrc:classes
15:46:06    > Task :buildSrc:jar
15:46:06    
15:46:06    [Incubating] Problems report is available at: file:///var/jenkins_home/workspace/4_my-project_my-project_bom_main/build/reports/problems/problems-report.html
15:46:06    
15:46:06    Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
15:46:06    
15:46:06    You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
15:46:06    
15:46:06    For more on this, please refer to https://docs.gradle.org/8.12/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
15:46:06    8 actionable tasks: 6 executed, 2 from cache
15:46:06     
15:46:06    FAILURE: Build failed with an exception.
15:46:06    
15:46:06    * What went wrong:
15:46:06    Cannot locate tasks that match '://../my-plugin1:properties' as project '//../my-plugin1' not found in root project 'my-project'.

[prabhu]

@nightm4re94 Good find! Could you contribute a PR?

cdxgen/lib/helpers/package_specific/gradleutils.js

Line 18 in 4b03f70

if (aline.includes("pluginManagement {")) {

Something that checks for comment prefix must work. Maybe we must check for /* blocks too.

if (aline.startsWith("//") {
    continue;
}

Tests could be added here. https://github.com/CycloneDX/cdxgen/blob/master/lib/helpers/package_specific/gradleutils.test.js

[nightm4re94]

I've opened PR #1731 to address this. Thanks for pointing me in the right direction!