Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pre-commit run --all-files fails on dirs that don't contain terragrunt.hcl #732

Open
sophie-warner-annalise-ai opened this issue Dec 27, 2024 · 2 comments

Comments

@sophie-warner-annalise-ai

Describe the bug

Not a major breaking issue but confused me for a bit.

If you're testing your pre-commit hook using run --all-files then per_dir_hook_unique_part is run instead of run_hook_on_whole_repo. This causes it to fail on dirs that don't have terragrunt.hcl but root.hcl for example.

function per_dir_hook_unique_part {
local -r args="$1"
# shellcheck disable=SC2034 # Unused var.
local -r dir_path="$2"
# pass the arguments to hook
# shellcheck disable=SC2068 # hook fails when quoting is used ("$arg[@]")
terragrunt validate ${args[@]}
# return exit code to common::per_dir_hook
local exit_code=$?
return $exit_code
}
#######################################################################
# Unique part of `common::per_dir_hook`. The function is executed one time
# in the root git repo
# Arguments:
# args (string with array) arguments that configure wrapped tool behavior
#######################################################################
function run_hook_on_whole_repo {
local -r args="$1"
# pass the arguments to hook
# shellcheck disable=SC2068 # hook fails when quoting is used ("$arg[@]")
terragrunt run-all validate ${args[@]}
# return exit code to common::per_dir_hook
local exit_code=$?
return $exit_code
}

How can we reproduce it?

❯ pre-commit run --all-files
Terragrunt fmt...........................................................Passed
Terragrunt validate......................................................Failed
- hook id: terragrunt_validate
- exit code: 1

18:16:11.781 ERROR  stat ./terragrunt.hcl: no such file or directory
18:16:11.781 ERROR  Unable to determine underlying exit code, so Terragrunt will exit with error code 1
❯ pre-commit try-repo /Users/sophie/repos/infrastructure-live/ terragrunt_validate
===============================================================================
Using config:
===============================================================================
repos:
-   repo: .
    rev: a6401255219ba1a2d7b93bcf899532ba17a2b443
    hooks:
    -   id: terragrunt_validate
===============================================================================
[INFO] Initializing environment for ..
An error has occurred: InvalidManifestError: 
=====> /var/folders/tg/zs1s0gr56vvggt4zly9ws__40000gr/T/tmph42b5wu5/repoj8tsfnee/.pre-commit-hooks.yaml is not a file
Check the log at /Users/sophie/.cache/pre-commit/pre-commit.log
/Users/sophie/.cache/pre-commit/pre-commit.log

version information

pre-commit version: 4.0.1
git --version: git version 2.39.5 (Apple Git-154)
sys.version:
    3.13.1 (main, Dec  3 2024, 17:59:52) [Clang 16.0.0 (clang-1600.0.26.4)]
sys.executable: /opt/homebrew/Cellar/pre-commit/4.0.1/libexec/bin/python
os.name: posix
sys.platform: darwin

error information

An error has occurred: InvalidManifestError: 
=====> /var/folders/tg/zs1s0gr56vvggt4zly9ws__40000gr/T/tmpja0ieavh/repoo10rh3nz/.pre-commit-hooks.yaml is not a file
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/pre-commit/4.0.1/libexec/lib/python3.13/site-packages/pre_commit/error_handler.py", line 73, in error_handler
    yield
  File "/opt/homebrew/Cellar/pre-commit/4.0.1/libexec/lib/python3.13/site-packages/pre_commit/main.py", line 421, in main
    return try_repo(args)
  File "/opt/homebrew/Cellar/pre-commit/4.0.1/libexec/lib/python3.13/site-packages/pre_commit/commands/try_repo.py", line 77, in try_repo
    return run(config_filename, store, args)
  File "/opt/homebrew/Cellar/pre-commit/4.0.1/libexec/lib/python3.13/site-packages/pre_commit/commands/run.py", line 425, in run
    for hook in all_hooks(config, store)
                ~~~~~~~~~^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pre-commit/4.0.1/libexec/lib/python3.13/site-packages/pre_commit/repository.py", line 233, in all_hooks
    return tuple(
        hook
        for repo in root_config['repos']
        for hook in _repository_hooks(repo, store, root_config)
    )
  File "/opt/homebrew/Cellar/pre-commit/4.0.1/libexec/lib/python3.13/site-packages/pre_commit/repository.py", line 236, in <genexpr>
    for hook in _repository_hooks(repo, store, root_config)
                ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pre-commit/4.0.1/libexec/lib/python3.13/site-packages/pre_commit/repository.py", line 211, in _repository_hooks
    return _cloned_repository_hooks(repo_config, store, root_config)
  File "/opt/homebrew/Cellar/pre-commit/4.0.1/libexec/lib/python3.13/site-packages/pre_commit/repository.py", line 178, in _cloned_repository_hooks
    by_id = {hook['id']: hook for hook in load_manifest(manifest_path)}
                                          ~~~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pre-commit/4.0.1/libexec/lib/python3.13/site-packages/cfgv.py", line 399, in load_from_filename
    with reraise_as(exc_tp):
         ~~~~~~~~~~^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/contextlib.py", line 162, in __exit__
    self.gen.throw(value)
    ~~~~~~~~~~~~~~^^^^^^^
  File "/opt/homebrew/Cellar/pre-commit/4.0.1/libexec/lib/python3.13/site-packages/cfgv.py", line 45, in reraise_as
    raise tp(e).with_traceback(tb) from None
  File "/opt/homebrew/Cellar/pre-commit/4.0.1/libexec/lib/python3.13/site-packages/cfgv.py", line 42, in reraise_as
    yield
  File "/opt/homebrew/Cellar/pre-commit/4.0.1/libexec/lib/python3.13/site-packages/cfgv.py", line 401, in load_from_filename
    raise ValidationError(f'{display_filename} is not a file')
pre_commit.clientlib.InvalidManifestError: 
=====> /var/folders/tg/zs1s0gr56vvggt4zly9ws__40000gr/T/tmpja0ieavh/repoo10rh3nz/.pre-commit-hooks.yaml is not a file

Environment information

  • OS: MacOS 15.2 (24C101)
  • uname -a and/or systeminfo | Select-String "^OS" output:
Darwin Mac.fritz.box 24.2.0 Darwin Kernel Version 24.2.0: Fri Dec  6 19:02:41 PST 2024; root:xnu-11215.61.5~2/RELEASE_ARM64_T6030 arm64
  • Tools availability and versions:
GNU bash, version 3.2.57(1)-release (arm64-apple-darwin24)
pre-commit 4.0.1
OpenTofu v1.8.8
Terraform v1.6.6
python SKIPPED
Python 3.13.1
checkov SKIPPED
infracost SKIPPED
terraform-docs version v0.19.0 darwin/arm64
terragrunt version v0.71.1
terrascan SKIPPED
TFLint version 0.54.0
+ ruleset.terraform (0.10.0-bundled)
tfsec SKIPPED
trivy SKIPPED
tfupdate SKIPPED
hcledit SKIPPED
  • .pre-commit-config.yaml:
file content
repos:
  - repo: https://github.com/antonbabenko/pre-commit-terraform
    rev: "v1.96.3"
    hooks:
      - id: terragrunt_fmt
      - id: terragrunt_validate
@MaxymVlasov
Copy link
Collaborator

This causes it to fail on dirs that don't have terragrunt.hcl but root.hcl for example.

According to https://terragrunt.gruntwork.io/docs/getting-started/configuration/

Terragrunt figures out the path to its config file according to the following rules:

1. The value of the --terragrunt-config command-line option, if specified.
2. The value of the TERRAGRUNT_CONFIG environment variable, if defined.
3. A terragrunt.hcl file in the current working directory, if it exists.
4. A terragrunt.hcl.json file in the current working directory, if it exists.
5. If none of these are found, exit with an error.

So we parse all hcl files

- id: terragrunt_validate
name: Terragrunt validate
description: Validates all Terragrunt configuration files.
entry: hooks/terragrunt_validate.sh
language: script
files: (\.hcl)$
exclude: \.terraform\/.*$

If you face that if you will run from CLI

terragrunt run-all validate "${args[@]}"

result will be different than provided by pre-commit - please let us know


  1. pre-commit try-repo /Users/sophie/repos/infrastructure-live/ terragrunt_validate
    2.1. It does_ not run hook on all files - you missed --all
    2.2. You should run it to repo with hook, not your infra repo - https://pre-commit.com/#developing-hooks-interactively. And note that this will ignore all your settings in .pre-commit-config.yaml if you have any

  1. I don't see there any issue yet, 1. - about terragrunt tool usage, and 2. - about pre-commit tool usage

Please run PRE_COMMIT_COLOR=never PCT_LOG=trace pre-commit run -a terragrunt_validate and attach full output here, if you still think that there issues with hook

@MaxymVlasov MaxymVlasov added question and removed bug Something isn't working labels Dec 30, 2024
@MaxymVlasov
Copy link
Collaborator

MaxymVlasov commented Dec 30, 2024

Btw, 1. looks like duplicate of #191

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants