CMake Dependency Validation & Reporting #5740

mdavis36 · 2025-12-26T21:28:00Z

Summary

This PR aims to aid new users in setting up and installing nvFuser successfully. This is done by providing user a comprehensive python report (based on #5609) as early as possible in the build process:

Clear nvFuser library dependencies & constraints
Minimum version (when applicable)
Optional vs Required dependencies
Actionable output to user on why a requirement is enforced and how to rectify it.

Differences (This PR vs #5609)

The outcome of the report is determined by CMake's evaluation of the constraints we place on requirements. The report has no effect on the ability to build nvFuser. All failure logic is define by the CMake system.

The report scripts are used to aid in formatting and printing pertinent information to the user. This is done by directly referencing CMake variables in python and allowing python to handle complicated string manipulation and formatting (which CMake is really bad at...).

The contents of the help messages largely remains the same as #5609. Giving user guidance based on their build platform.

CMake Changes

cmake/DependencyRequirements.cmake is the single source of truth for version requirements, components and the state of OPTIONAL for each dependency.
Option NVFUSER_ENABLE_DEPENDENCY_REPORT is by default ON. If this is set OFF then dependencies will be evaluated as "normal" in CMake and the build configuration will exit of the first failure.
Each requirements logic is defined in it's own cmake/deps/handle_<name>.cmake file for some organization/clarity.

Success Case

CMake dependency evaluation happens silently and is written to buffer.
Python report is generated as early as possible.
- On first run: CMake will always look for compilers for the LANGUAGES the project is built for first - this cant be skipped AFAIK.
- On subsequent runs: the python report is displayed immediately (compiler information is cached).
CMake output is dumped to the user for detailed reporting (this is the same as when running with NVFUSER_ENABLE_DEPENDECY_REPORT=Off)

Failure Case (example : pybind11 version too low)

Report fails with installation instructions for users.

Does not FATAL_ERROR when pybind11 mismatches.
CMake still dumps the output evaluating ALL dependencies
CMake exits after reporting detailed output.

…d basic constraint requirements.

github-actions · 2025-12-26T21:29:11Z

Review updated until commit a2c9053

Description

Implements comprehensive CMake dependency validation with Python-based reporting
Provides user-friendly dependency status with colored terminal output and installation guidance
Centralizes all dependency requirements in CMake with minimum versions and optional flags
Adds platform-specific installation instructions for missing dependencies (Python, CUDA, LLVM, etc.)
Enables early failure detection with actionable error messages for build prerequisites

Changes walkthrough

Relevant files

Enhancement

27 files

check_dependencies.py `Main Python dependency report generator with colored output`	+218/-0
__init__.py `Prerequisites package main module with exports`	+105/-0
exceptions.py `Custom exceptions for prerequisite validation`	+23/-0
platform.py `Platform detection utilities for OS and distribution`	+129/-0
requirement_utils.py `Version parsing utilities and URL generators`	+257/-0
__init__.py `Requirement classes registry and imports`	+31/-0
base.py `Base requirement classes with status formatting`	+252/-0
compiler.py `C++ compiler requirement with GNU/Clang support`	+139/-0
cuda_toolkit.py `CUDA Toolkit dependency requirement`	+75/-0
git_submodules.py `Git submodules initialization check`	+60/-0
llvm.py `LLVM dependency requirement with download URLs`	+121/-0
ninja.py `Ninja build system requirement`	+86/-0
pybind11.py `pybind11 dependency requirement`	+70/-0
python.py `Python interpreter requirement`	+96/-0
torch.py `PyTorch requirement with CUDA constraint validation`	+174/-0
DependencyRequirements.cmake `Centralized dependency metadata with minimum versions`	+39/-0
DependencyUtilities.cmake `Dependency status tracking and JSON export utilities`	+133/-0
LogCapture.cmake `CMake message capture and logging control`	+85/-0
handle_compiler.cmake `C++ compiler handler with version validation`	+25/-0
handle_cuda_toolkit.cmake `CUDA Toolkit dependency handler`	+18/-0
handle_git_submodules.cmake `Git submodules validation handler`	+29/-0
handle_llvm.cmake `LLVM dependency handler with component mapping`	+34/-0
handle_ninja.cmake `Ninja build system handler`	+25/-0
handle_pybind11.cmake `pybind11 dependency handler`	+18/-0
handle_python.cmake `Python dependency handler`	+18/-0
handle_torch.cmake `PyTorch handler with CUDA version constraint checking`	+98/-0
CMakeLists.txt `Main CMake file with integrated dependency validation system`	+70/-62

Additional files

1 files

FlatBuffers.cmake	+3/-0

PR Reviewer Guide

Here are some key observations to aid the review process:

🧪 PR contains tests
🔒 Security concerns No - The changes are primarily build system enhancements with no security implications. The JSON export includes proper escaping, and the Python script only reads JSON and prints formatted output without executing any external commands or processing untrusted input.
⚡ Recommended focus areas for review
CUDA Constraint Validation The CUDA version constraint validation between Torch and CUDAToolkit (lines 108-118) adds critical build-time validation but could potentially cause issues if PyTorch was built with a different CUDA version than the system toolkit. This is working as designed but reviewers should verify the constraint logic handles all edge cases correctly, especially for users with non-standard PyTorch installations. def is_failure(self) -> bool: """Check for both version failure and CUDA constraint failure.""" # Check base version requirement if super().is_failure(): return True # Check CUDA constraint if self.constraint_status == "mismatch": return True return False JSON Export Security The export_dependency_json function (lines 67-99) exports all CMake variables to JSON with basic escaping. While the escaping appears adequate for typical use cases, reviewers should verify that the string replacement operations (lines 82-86) handle all potential edge cases in variable values, especially for paths with special characters or unusual configurations. function(export_dependency_json output_file) # Get all CMake variables get_cmake_property(all_vars VARIABLES) # Write JSON file with flat variable dict file(WRITE "${output_file}" "{\n") file(APPEND "${output_file}" " \"cmake_vars\": {\n") # Export all variables (sorted for consistency) list(SORT all_vars) list(LENGTH all_vars var_count) set(var_index 0) foreach(var ${all_vars}) set(value "${${var}}") # Escape for JSON strings string(REPLACE "\\" "\\\\" value "${value}") string(REPLACE "\"" "\\\"" value "${value}") string(REPLACE "\n" "\\n" value "${value}") string(REPLACE "\t" "\\t" value "${value}") string(REPLACE "\r" "\\r" value "${value}") # Add comma if not last item math(EXPR var_index "${var_index} + 1") if(var_index LESS var_count) file(APPEND "${output_file}" " \"${var}\": \"${value}\",\n") else() file(APPEND "${output_file}" " \"${var}\": \"${value}\"\n") endif() endforeach() file(APPEND "${output_file}" " }\n") file(APPEND "${output_file}" "}\n") endfunction() Build Integration The integration of the new dependency system into the main CMakeLists.txt (lines 73-103) replaces the previous hardcoded dependency checks. Reviewers should verify that all previously required dependencies are still properly validated and that the new system doesn't introduce any regressions in dependency handling logic. if(NVFUSER_ENABLE_DEPENDENCY_REPORT) start_capture() endif() # Handle each dependency explicitly handle_compiler() handle_ninja() handle_cuda_toolkit() handle_python() handle_torch() # Must come AFTER python and cudatoolkit. handle_pybind11() handle_llvm() if(NVFUSER_ENABLE_DEPENDENCY_REPORT) stop_capture(DEP_LOGS) # Show the dependency report (Python-based) report_dependencies() # Dump detailed logs message("") message(STATUS "****** Detailed Dependency Output ****") dump_captured_logs("${DEP_LOGS}") message(STATUS "**** End Dependency Output ******") message("") endif() # Check if any required dependencies failed if(NOT NVFUSER_DEPENDENCIES_OK) message(FATAL_ERROR "Please install or upgrade the required dependencies listed above.") endif()

greptile-apps · 2025-12-26T21:36:36Z

Greptile Summary

This PR implements a comprehensive CMake dependency validation and reporting system that transforms cryptic build errors into actionable guidance for users. The implementation is well-architected with clear separation of concerns: CMake handles validation logic while Python provides user-friendly formatting and platform-specific installation instructions.

Key Improvements

Single Source of Truth: DependencyRequirements.cmake centralizes all version constraints
Enhanced User Experience: Python-based reporter with colored output and installation instructions replaces cryptic CMake errors
Log Buffering: Clever message override system captures CMake output for delayed display after Python report
Comprehensive Validation: Validates 8 dependencies including special CUDA constraint (PyTorch CUDA must match system CUDAToolkit)
Optional Dependencies: Proper handling of optional vs required dependencies with clear visual indicators

Implementation Quality

The code demonstrates strong engineering practices:

Clean OOP design in Python with base requirement classes
Proper error handling and fallback mechanisms
Consistent status tracking across all handlers
Well-documented with clear variable naming
Platform-specific help generation for Linux/macOS

Previous Review Comments

All previously identified issues have been addressed or acknowledged by the development team. The main feedback points were style-related (JSON escaping completeness, variable scope in compiler.py, etc.) and have either been resolved or deemed acceptable trade-offs.

Confidence Score: 5/5

This PR is safe to merge with minimal risk - well-tested dependency validation framework with clear benefits
Score reflects thorough implementation with proper separation of concerns, comprehensive error handling, and non-breaking design. The validation logic is sound, Python fallback mechanisms prevent failures, and the feature can be disabled via NVFUSER_ENABLE_DEPENDENCY_REPORT=OFF. Previous review feedback has been addressed.
No files require special attention - all implementations are solid and previous concerns have been addressed

Important Files Changed

Filename	Overview
CMakeLists.txt	Added dependency validation framework with log capture, Python reporting integration, and explicit dependency handlers - clean integration with existing build system
cmake/DependencyRequirements.cmake	Central source of truth for all dependency version requirements and constraints - well-organized and maintainable
cmake/DependencyUtilities.cmake	Provides status tracking and JSON export utilities for dependency reporting - solid implementation with proper escaping
cmake/LogCapture.cmake	Message capture system using function override and global properties - clever implementation for buffering CMake output
cmake/deps/handle_compiler.cmake	Compiler validation handler for GNU/Clang - correctly sets version requirements based on compiler ID
cmake/deps/handle_torch.cmake	PyTorch handler with CUDA version constraint validation - properly checks Torch CUDA matches system CUDAToolkit version
python/tools/check_dependencies.py	Python-based dependency reporter that formats CMake JSON data into user-friendly colored output with installation instructions
python/tools/prereqs/requirements/base.py	Base classes for requirement types with OOP design - clean abstractions for version and boolean requirements
python/tools/prereqs/requirements/compiler.py	Compiler requirement class with platform-specific installation help - stores min versions properly as instance attributes
python/tools/prereqs/requirements/torch.py	Torch requirement with CUDA constraint checking - properly validates and reports CUDA version mismatches

Sequence Diagram

sequenceDiagram
    participant User
    participant CMake as CMake Build System
    participant LogCapture as LogCapture.cmake
    participant Handlers as Dependency Handlers
    participant Utils as DependencyUtilities.cmake
    participant Python as check_dependencies.py
    participant Requirements as Requirement Classes
    
    User->>CMake: Run cmake ..
    CMake->>CMake: Load DependencyRequirements.cmake (version constraints)
    CMake->>CMake: Load DependencyUtilities.cmake (status tracking)
    CMake->>LogCapture: Load LogCapture.cmake (message override)
    CMake->>CMake: Include all handler files
    
    alt NVFUSER_ENABLE_DEPENDENCY_REPORT=ON
        CMake->>LogCapture: start_capture()
        LogCapture->>LogCapture: Set LOG_CAPTURE_MODE=TRUE
        
        CMake->>Handlers: handle_compiler()
        Handlers->>CMake: Check compiler version vs requirements
        Handlers->>Utils: set_dependency_report_status(Compiler)
        Utils->>Utils: Compare version, set status variables
        Note over Handlers,LogCapture: Messages buffered, not printed
        
        CMake->>Handlers: handle_python(), handle_cuda_toolkit()...
        Handlers->>Utils: set_dependency_report_status() for each
        Utils->>Utils: Update NVFUSER_DEPENDENCIES_OK flag
        
        CMake->>Handlers: handle_torch()
        Handlers->>Handlers: Find PyTorch via Python
        Handlers->>Utils: set_dependency_report_status(Torch)
        Handlers->>Handlers: Query torch.version.cuda
        Handlers->>Handlers: Compare Torch CUDA vs CUDAToolkit version
        Handlers->>Utils: Set Torch_CUDA_constraint_status
        
        CMake->>LogCapture: stop_capture(DEP_LOGS)
        LogCapture->>LogCapture: Set LOG_CAPTURE_MODE=FALSE
        LogCapture->>CMake: Return buffered logs
        
        CMake->>Utils: export_dependency_json()
        Utils->>Utils: Write all CMake vars to JSON
        
        CMake->>Python: Execute check_dependencies.py <json_file>
        Python->>Python: Load cmake_vars from JSON
        Python->>Requirements: Create requirement objects
        Requirements->>Requirements: Extract status/version from cmake_vars
        Python->>User: Print colored dependency report
        
        alt Any failures found
            Requirements->>Requirements: is_failure() checks
            Requirements->>User: generate_help() with install instructions
        end
        
        Python->>CMake: Exit (status 0, doesn't fail build)
        
        CMake->>LogCapture: dump_captured_logs(DEP_LOGS)
        LogCapture->>User: Print detailed CMake output
        
        alt NVFUSER_DEPENDENCIES_OK=FALSE
            CMake->>User: FATAL_ERROR with instructions
        end
        
    else NVFUSER_ENABLE_DEPENDENCY_REPORT=OFF
        CMake->>Handlers: handle_compiler() etc.
        Handlers->>CMake: find_package(...REQUIRED)
        Note over CMake,User: Fails immediately on first error
    end

greptile-apps

_{27 files reviewed, 3 comments}

_{Edit Code Review Agent Settings | Greptile}

cmake/deps/handle_torch.cmake

CMakeLists.txt

greptile-apps

_{27 files reviewed, 11 comments}

_{Edit Code Review Agent Settings | Greptile}

cmake/deps/handle_compiler.cmake

python/tools/prereqs/requirements/compiler.py

cmake/deps/handle_torch.cmake

python/tools/prereqs/requirements/torch.py

python/tools/prereqs/requirements/ninja.py

cmake/deps/handle_ninja.cmake

cmake/deps/handle_git_submodules.cmake

python/tools/check_dependencies.py

cmake/LogCapture.cmake

greptile-apps

_{27 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

python/tools/prereqs/requirements/compiler.py

greptile-apps

_{27 files reviewed, 7 comments}

_{Edit Code Review Agent Settings | Greptile}

python/tools/prereqs/requirements/git_submodules.py

python/tools/prereqs/requirements/cuda_toolkit.py

python/tools/prereqs/__init__.py

cmake/DependencyRequirements.cmake

python/tools/prereqs/platform.py

cmake/LogCapture.cmake

…init

greptile-apps

_{27 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

python/tools/prereqs/requirements/compiler.py

greptile-apps

_{27 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

cmake/deps/handle_compiler.cmake

greptile-apps

_{27 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

python/tools/check_dependencies.py

mdavis36 · 2025-12-27T21:58:55Z

!build

mdavis36 · 2026-01-02T23:50:37Z

CMakeLists.txt

 # All rights reserved.
 # SPDX-License-Identifier: BSD-3-Clause
-cmake_minimum_required(VERSION 3.18 FATAL_ERROR)
+cmake_minimum_required(VERSION 3.18)


FATAL_ERROR here has been ignored since v3.6, this is always a hard requirement

mdavis36 · 2026-01-02T23:53:37Z

CMakeLists.txt

-# We try to update which C++ standard we use together in lockstep across all
-# built libraries, and these variables control which that is. Generally we are
-# on C++20, but we still support a version of CUDA (11) that does not recognize
-# C++20 and so we drop back to 17 there. Also, we allow all of these to be
-# overridden by the user.


@xwang233 is this comment still correct? is there a version of CUDA we support that does not recognize c++20 still?

greptile-apps

_{28 files reviewed, 4 comments}

_{Edit Code Review Agent Settings | Greptile}

cmake/LogCapture.cmake

cmake/DependencyUtilities.cmake

greptile-apps · 2026-01-03T01:12:11Z

cmake/deps/handle_torch.cmake

+  set(cuda_toolkit_version "${CUDAToolkit_VERSION_MAJOR}.${CUDAToolkit_VERSION_MINOR}")
+
+  # Extract major.minor from Torch CUDA version
+  string(REGEX MATCH "^([0-9]+\\.[0-9]+)" torch_cuda_major_minor "${torch_cuda_version}")


logic: If torch_cuda_version has unexpected format (e.g., no dots), torch_cuda_major_minor will be empty, causing silent comparison failure.

Add validation:

if(NOT torch_cuda_major_minor) set(Torch_CUDA_constraint_status "not_available") return() endif()

greptile-apps · 2026-01-03T01:12:12Z

cmake/deps/handle_compiler.cmake

+  if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
+    set(NVFUSER_REQUIREMENT_Compiler_VERSION_MIN ${NVFUSER_REQUIREMENT_GNU_VERSION_MIN})
+  elseif(CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+    set(NVFUSER_REQUIREMENT_Compiler_VERSION_MIN ${NVFUSER_REQUIREMENT_Clang_VERSION_MIN})
+  endif()


style: Compilers other than GNU/Clang (e.g., MSVC, Intel) will have unset NVFUSER_REQUIREMENT_Compiler_VERSION_MIN, causing version check to pass with SUCCESS status.

The OPTIONAL=TRUE on line 23 makes this non-fatal, but users might not notice they're using an unsupported compiler. Consider adding an explicit check for supported compiler IDs.

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

mdavis36 · 2026-01-03T18:19:36Z

!build

mdavis36 added 21 commits December 22, 2025 20:14

Working example dependency output buffering.

8ddd465

Reporting basic status of CUDAToolkit & pybind11

f84b2d0

pretty colors.

a173f30

Alligning the report text.

7ccfce0

On failure return report before exiting

4a16da4

Remove duplicate install logic

2925d82

Setting up infrastructure for validating find_package dependencies an…

dfa35bf

…d basic constraint requirements.

Use helper functions to aid in readability of report function

6f38b6b

generate a json dependency file with the results from CMake

7617ae3

dep python report.

afaaba3

CMake execute python report script.

1042a65

Remove CMake base reporting.

8b47e83

Creating prereq package.

3b99e75

pulling in error/help messages from #5609

53fa541

compiler, ninja & submodule reports.

6c234bb

compiler, ninja and submodule checks.

90ee1ec

simplify dep failure check.

564ab2c

organize python req into modules.

ae99893

Fail after CMake dump.

3a890b6

re-order

9ea74d9

Move validation functions into their own cmake modules.

108dabf

Drop unused requirements vars.

a0f5f12

mdavis36 added 6 commits December 26, 2025 14:02

Reducing logical overhead in repoting scripts with class structures.

e85dc11

Treat torch.cuda == cudatoolkit as a boolean req.

dcc0032

Remove abstraction between python and CMake, just infer CMake variables

85967a9

Simplify json output from CMake, let python do what it's good at.

3be18cb

explicit arguments for constructors.

3cb8098

Drop dependency name list.

0f2ad5a

greptile-apps bot reviewed Dec 27, 2025

View reviewed changes

cmake/deps/handle_torch.cmake Show resolved Hide resolved

cmake/deps/handle_torch.cmake Show resolved Hide resolved

CMakeLists.txt Outdated Show resolved Hide resolved

mdavis36 added 4 commits December 27, 2025 10:21

Disable submodules, this needs further consideration.

1048742

remove include.

eaad1ed

user var for components.

e818dfe

user set_dependency_report_status in torch logic pre-checks.

7486630

greptile-apps bot reviewed Dec 27, 2025

View reviewed changes

mdavis36 added 2 commits December 27, 2025 10:58

Fix location report

7c5c597

report colors & format.

4331d77

mdavis36 force-pushed the md/cmake-deps-5609-integration branch from 4bbd913 to 4331d77 Compare December 27, 2025 20:12

greptile-apps bot reviewed Dec 27, 2025

View reviewed changes

python/tools/prereqs/requirements/compiler.py Outdated Show resolved Hide resolved

doc update

8b286f1

greptile-apps bot reviewed Dec 27, 2025

View reviewed changes

add platform info to report; remove unused functions in requirements …

adcac39

…init

greptile-apps bot reviewed Dec 27, 2025

View reviewed changes

python/tools/prereqs/requirements/compiler.py Outdated Show resolved Hide resolved

mdavis36 added 2 commits December 27, 2025 13:14

clean up imports

f8f6677

check cmake variables are correct in comments.

59fe38a

greptile-apps bot reviewed Dec 27, 2025

View reviewed changes

cmake/deps/handle_compiler.cmake Show resolved Hide resolved

Remove unused log functions.

11fb7ff

greptile-apps bot reviewed Dec 27, 2025

View reviewed changes

python/tools/check_dependencies.py Outdated Show resolved Hide resolved

mdavis36 added 3 commits December 27, 2025 13:37

var scope.

281adf9

print help on non-success optional requirments too.

3918639

Ninja configuration instructions.

19da6b5

Addressing greptile comments.

1b7fc18

mdavis36 commented Jan 3, 2026

View reviewed changes

mdavis36 added 2 commits January 2, 2026 17:06

lintrunner changes.

4353690

Add license headers

476d48d

greptile-apps bot reviewed Jan 3, 2026

View reviewed changes

lint

a2c9053

CMake Dependency Validation & Reporting #5740

Are you sure you want to change the base?

CMake Dependency Validation & Reporting #5740

Uh oh!

Conversation

mdavis36 commented Dec 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Differences (This PR vs #5609)

CMake Changes

Success Case

Failure Case (example : pybind11 version too low)

Uh oh!

github-actions bot commented Dec 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Changes walkthrough

PR Reviewer Guide

Uh oh!

greptile-apps bot commented Dec 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Key Improvements

Implementation Quality

Previous Review Comments

Confidence Score: 5/5

Important Files Changed

Sequence Diagram

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mdavis36 commented Dec 27, 2025

Uh oh!

mdavis36 Jan 2, 2026

Choose a reason for hiding this comment

Uh oh!

mdavis36 Jan 2, 2026

mdavis36 commented Dec 26, 2025 •

edited

Loading

github-actions bot commented Dec 26, 2025 •

edited

Loading

greptile-apps bot commented Dec 26, 2025 •

edited

Loading