Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(misconf): High memory usage (> 10 GB) on some repos #6959

Closed
2 tasks
nikpivkin opened this issue Jun 18, 2024 Discussed in #6958 · 0 comments · Fixed by #6968
Closed
2 tasks

perf(misconf): High memory usage (> 10 GB) on some repos #6959

nikpivkin opened this issue Jun 18, 2024 Discussed in #6958 · 0 comments · Fixed by #6968
Assignees
Labels
kind/bug Categorizes issue or PR as related to a bug. scan/misconfiguration Issues relating to misconfiguration scanning
Milestone

Comments

@nikpivkin
Copy link
Contributor

Discussed in #6958

Originally posted by david-nascimento-form3 June 18, 2024

Description

Seems related to:

We noticed high memory usage when using the misconfig scanner terraform, it easily reaches the 24GB of memory when a high number of terraform resources are present [>10000].

Screenshot 2024-06-18 at 14 37 45

[!Note]

This profiling information was not from the code sample below

Desired Behavior

To have a steady memory usage, if possible close to tfsec.

Actual Behavior

A high memory usage that leads to a container being killed.

Reproduction Steps

This is an issue with a high number of resources, the following sample easily reach 20GB of memory:


locals {
  team_repos = [ for i in range(1000): "repo-${i}"]
  teams = [ for i in range(10): "team-${i}"]
  repositories = merge([for team_id in local.teams : { for repo in local.team_repos : "${team_id}-${repo}" => team_id}]...)
}

resource "aws_ecr_repository" "ecr-repository" {
  for_each = local.repositories

  name                 = each.key
  image_tag_mutability = "IMMUTABLE"
  tags = {
    "Team" : each.value
  }
}


To note that when analysed with `tfsec` the memory did not exceed `1GB`.

Target

Filesystem

Scanner

Misconfiguration

Output Format

JSON

Mode

Standalone

Debug Output

$ trivy config --misconfig-scanners terraform test-tf --debug

2024-06-18T16:46:54+01:00       DEBUG   Parsed severities       severities=[UNKNOWN LOW MEDIUM HIGH CRITICAL]
2024-06-18T16:46:54+01:00       DEBUG   Cache dir       dir="/Users/***********/Library/Caches/trivy"
2024-06-18T16:46:54+01:00       INFO    Misconfiguration scanning is enabled
2024-06-18T16:46:54+01:00       DEBUG   Policies successfully loaded from disk
2024-06-18T16:46:54+01:00       DEBUG   Enabling misconfiguration scanners      scanners=[terraform]
2024-06-18T16:46:54+01:00       DEBUG   [nuget] The nuget packages directory couldn't be found. License search disabled
2024-06-18T16:46:54+01:00       DEBUG   Scanning files for misconfigurations... scanner="Terraform"
2024-06-18T16:46:54+01:00       DEBUG   [misconf] 46:54.073583000 terraform.scanner                Scanning [&{%!s(*mapfs.file=&{ [] {. 256 2147484096 {13948924422273498744 333823334 0x10d8f81a0} <nil>} {{{0 0} {[] {} 0x14002f1dcc0} map[test.tf:0x140027e5bf0] 0}}}) test-tf}] at '.'...
2024-06-18T16:46:54+01:00       DEBUG   [misconf] 46:54.075550000 terraform.scanner.rego           Overriding filesystem for checks!
2024-06-18T16:46:54+01:00       DEBUG   [misconf] 46:54.076185000 terraform.scanner.rego           Loaded 3 embedded libraries.
2024-06-18T16:46:54+01:00       DEBUG   [misconf] 46:54.105253000 terraform.scanner.rego           Loaded 191 embedded policies.
2024-06-18T16:46:54+01:00       DEBUG   [misconf] 46:54.146610000 terraform.scanner.rego           Loaded 194 checks from disk.
2024-06-18T16:46:54+01:00       DEBUG   [misconf] 46:54.146888000 terraform.scanner.rego           Overriding filesystem for data!
2024-06-18T16:46:54+01:00       DEBUG   [misconf] 46:54.346109000 terraform.parser.<root>          Setting project/module root to '.'
2024-06-18T16:46:54+01:00       DEBUG   [misconf] 46:54.346135000 terraform.parser.<root>          Parsing FS from '.'
2024-06-18T16:46:54+01:00       DEBUG   [misconf] 46:54.346163000 terraform.parser.<root>          Parsing 'test.tf'...
2024-06-18T16:46:54+01:00       DEBUG   [misconf] 46:54.346434000 terraform.parser.<root>          Added file test.tf.
2024-06-18T16:46:54+01:00       DEBUG   [misconf] 46:54.346517000 terraform.scanner                Scanning root module '.'...
2024-06-18T16:46:54+01:00       DEBUG   [misconf] 46:54.346521000 terraform.parser.<root>          Setting project/module root to '.'
2024-06-18T16:46:54+01:00       DEBUG   [misconf] 46:54.346523000 terraform.parser.<root>          Parsing FS from '.'
2024-06-18T16:46:54+01:00       DEBUG   [misconf] 46:54.346529000 terraform.parser.<root>          Parsing 'test.tf'...
2024-06-18T16:46:54+01:00       DEBUG   [misconf] 46:54.346605000 terraform.parser.<root>          Added file test.tf.
2024-06-18T16:46:54+01:00       DEBUG   [misconf] 46:54.346613000 terraform.parser.<root>          Evaluating module...
2024-06-18T16:46:54+01:00       DEBUG   [misconf] 46:54.346645000 terraform.parser.<root>          Read 2 block(s) and 0 ignore(s) for module 'root' (1 file[s])...
2024-06-18T16:46:54+01:00       DEBUG   [misconf] 46:54.346651000 terraform.parser.<root>          Added 0 variables from tfvars.
2024-06-18T16:46:54+01:00       DEBUG   [misconf] 46:54.346686000 terraform.parser.<root>          Working directory for module evaluation is "/Users/***********/trivy"
2024-06-18T16:46:54+01:00       DEBUG   [misconf] 46:54.346725000 terraform.parser.<root>.evaluator Filesystem key is '**********************************************'
2024-06-18T16:46:54+01:00       DEBUG   [misconf] 46:54.346728000 terraform.parser.<root>.evaluator Starting module evaluation...
2024-06-18T16:49:52+01:00       DEBUG   [misconf] 49:52.125958000 terraform.parser.<root>.evaluator Expanded block 'aws_ecr_repository.ecr-repository' into 10000 clones via 'for_each' attribute.
2024-06-18T16:49:52+01:00       DEBUG   [misconf] 49:52.131119000 terraform.parser.<root>.evaluator Starting submodule evaluation...
2024-06-18T16:49:52+01:00       DEBUG   [misconf] 49:52.131450000 terraform.parser.<root>.evaluator All submodules are evaluated at i=0
2024-06-18T16:49:52+01:00       DEBUG   [misconf] 49:52.131456000 terraform.parser.<root>.evaluator Starting post-submodule evaluation...
2024-06-18T17:01:01+01:00       DEBUG   [misconf] 01:01.298179000 terraform.parser.<root>.evaluator Finished processing 0 submodule(s).
2024-06-18T17:01:01+01:00       DEBUG   [misconf] 01:01.299317000 terraform.parser.<root>.evaluator Module evaluation complete.
2024-06-18T17:01:01+01:00       DEBUG   [misconf] 01:01.329185000 terraform.parser.<root>          Finished parsing module 'root'.
2024-06-18T17:01:01+01:00       DEBUG   [misconf] 01:01.329385000 terraform.executor               Adapting modules...
2024-06-18T17:01:01+01:00       DEBUG   [misconf] 01:01.413986000 terraform.executor               Adapted 1 module(s) into defsec state data.
2024-06-18T17:01:01+01:00       DEBUG   [misconf] 01:01.414046000 terraform.executor               Using max routines of 9
2024-06-18T17:01:01+01:00       DEBUG   [misconf] 01:01.414613000 terraform.executor               Initialized 486 rule(s).
2024-06-18T17:01:01+01:00       DEBUG   [misconf] 01:01.414619000 terraform.executor               Created pool with 9 worker(s) to apply rules.
2024-06-18T17:01:01+01:00       DEBUG   [misconf] 01:01.851145000 terraform.scanner.rego           Scanning 1 inputs...
2024-06-18T17:01:02+01:00       DEBUG   [misconf] 01:02.970171000 terraform.executor               Finished applying rules.
2024-06-18T17:01:02+01:00       DEBUG   [misconf] 01:02.970229000 terraform.executor               Applying ignores...
2024-06-18T17:01:05+01:00       DEBUG   OS is not detected.
2024-06-18T17:01:05+01:00       INFO    Detected config files   num=2
2024-06-18T17:01:05+01:00       DEBUG   Scanned config file     path="."
2024-06-18T17:01:05+01:00       DEBUG   Scanned config file     path="test.tf"

Operating System

Sonoma 14.5, [Container image linux/amd64]

Version

Version: 0.52.2

Also tested with latest version of master:

Version: dev
Vulnerability DB:
  Version: 2
  UpdatedAt: 2022-05-26 12:06:58.288892667 +0000 UTC
  NextUpdate: 2022-05-26 18:06:58.288892267 +0000 UTC
  DownloadedAt: 2022-05-26 13:22:42.722024668 +0000 UTC
Check Bundle:
  Digest: sha256:cfb65621a1f55d9d099c4c28931b252716fcda8bba5081eb43f1001668e79d85
  DownloadedAt: 2024-06-18 14:19:01.638403 +0000 UTC

Checklist

@nikpivkin nikpivkin added kind/bug Categorizes issue or PR as related to a bug. scan/misconfiguration Issues relating to misconfiguration scanning labels Jun 18, 2024
@nikpivkin nikpivkin self-assigned this Jun 19, 2024
@simar7 simar7 added this to the v0.55.0 milestone Aug 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug. scan/misconfiguration Issues relating to misconfiguration scanning
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants