Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(loki.secretfilter): Fix partial masking for short secrets and support multiple allowlists per rule #2320

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

romain-gaillard
Copy link
Contributor

PR Description

  • Fixes the issue reported in Panic: runtime error utilizing secretfilter component #2288 where the component was crashing when the secret to mask was shorter than the partial_mask value.
  • Adds (partial) support for the new [[rules.allowlists]] format added in Gitleaks v8.21.0
  • Updates the documentation to be clearer on the fact that the component doesn't support all features of the Gitleaks configuration format.

Which issue(s) this PR fixes

Notes to the Reviewer

PR Checklist

  • CHANGELOG.md updated
  • Documentation added
  • Tests updated
  • Config converters updated

@romain-gaillard romain-gaillard requested a review from a team December 31, 2024 12:07
Copy link
Contributor

github-actions bot commented Dec 31, 2024

@romain-gaillard romain-gaillard self-assigned this Dec 31, 2024
@romain-gaillard romain-gaillard marked this pull request as ready for review January 6, 2025 12:00
Regexes []string
}
Rules []struct {
ID string
Description string
Description string // Not used
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these unused variables still here so the format processes? What is the harm in removing them?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, indeed they could be removed. I initially added them to keep track of features we don't currently support but might in the future.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case I would rather remote/comment out the whole line so they cant mistakenly be used.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 I removed it

Copy link
Member

@mostafa mostafa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@clayton-cornell clayton-cornell added the type/docs Docs Squad label across all Grafana Labs repos label Jan 6, 2025

The `gitleaks_config` argument is the path to the custom `gitleaks.toml` file.
The Gitleaks configuration file embedded in the component is used if you don't provide the path to a custom configuration file.

The `types` argument is a map of secret types to look for. The values are used as prefixes for the secret types in the Gitleaks configuration. If you don't provide this argument, all types are used.
{{< admonition type="note" >}}
This component does not support all the features of the Gitleaks configuration file. Currently, it only supports the regex-based rules, `secretGroup`, and allowlist regexes (`regexTarget` only supports the default value `secret`). Other features such as `keywords`, `entropy`, `paths`, and `stopwords` are not supported. The `extend` feature is also not supported, meaning that a custom configuration file must contain all the rules to use.
Copy link
Contributor

@clayton-cornell clayton-cornell Jan 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This component does not support all the features of the Gitleaks configuration file. Currently, it only supports the regex-based rules, `secretGroup`, and allowlist regexes (`regexTarget` only supports the default value `secret`). Other features such as `keywords`, `entropy`, `paths`, and `stopwords` are not supported. The `extend` feature is also not supported, meaning that a custom configuration file must contain all the rules to use.
This component doesn't support all the features of the Gitleaks configuration file. It only supports regular expression-based rules, `secretGroup`, and allowlist regular expressions. `regexTarget` only supports the default value `secret`. Other features such as `keywords`, `entropy`, `paths`, and `stopwords` are not supported. The `extend` feature is not supported. If you use a custom configuration file, you must include all the rules you want to use within the configuration file.

It's not clear to me here... is it regex-based rules AND secretGroup AND allowlist? Or just secretGroup and allowlist (which are regex-based rules)? I assume it's the first one... and the suggestion reflects that assumption.

The "other features"... are they actually features? or arguments?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, indeed, it's the first one.

I think it's still worth keeping a mention to the Gitleaks configuration file? Because people would read the documentation on the Gitleaks configuration file (or import it from a different place they use it) and maybe expect some features to work in this component too. That's why I wanted to list what's supported and what's not in the Gitleaks configuration file format

Copy link
Contributor Author

@romain-gaillard romain-gaillard Jan 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, for the feature/argument part, it's probably both. The feature (e.g. entropy, stopwords, ...) is configured via arguments. The feature is currently not implemented and the related arguments are therefore ignored.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the suggestion to include the Gitleaks part. We can leave the "other features" part as-is :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/docs Docs Squad label across all Grafana Labs repos
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants