Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add: standalone plugin for evaluating dependencies with a graph #774

Draft
wants to merge 18 commits into
base: main
Choose a base branch
from

Conversation

NiklasHargarter
Copy link
Contributor

@NiklasHargarter NiklasHargarter commented Dec 11, 2024

What

Adds a standalone plugin for evaluating script dependencies with a directed networkx graph.

checks for:

  • duplicate dependencies (a script declaring dependency on other script multiple times)
  • checking for cyclic dependencies
  • checking for missing dependencies
  • checking for cross feed dependencies (community script dependence on enterprise script). It is differentiated between dependencies that are behind a enterprise feed gate and those that are not.
  • category order
  • dependency on deprecated script

included functionality of normal plugins

  • dependencies (not included the subdirectory placement warning)
  • dependency_category_order (missing ACT_SCANNER error)
  • deprecated_dependency (i use the helper pattern regex not the one from the plugin)

Output

python logging levels for system information (error, warning, info)
normal additive verbosity up to -vv for result output.

Feed options

  • 21.04 (21.04 + common)
  • 22.04 (22.04 + common)
  • common
  • full (21.0 + 22.04 + common)

example call:
poetry run troubadix-dependency-graph ~/gb/vulnerability-tests/nasl --feed full --log info -vv

Execution Time

locally ~13 seconds

Why

When checking dependencies, it makes sense to analyse the whole feed, rather than just working on changed scripts. And working on the whole feed is easier with a standalone plugin that doesn't have to adhere to the Troubadix structure.

References

Checklist

  • Tests

Copy link

github-actions bot commented Dec 11, 2024

Conventional Commits Report

Type Number
Changed 11
Added 5

🚀 Conventional commits found.

Copy link

codecov bot commented Dec 11, 2024

Codecov Report

Attention: Patch coverage is 84.87395% with 36 lines in your changes missing coverage. Please review.

Project coverage is 80.38%. Comparing base (9bf6d58) to head (3a529e5).

Files with missing lines Patch % Lines
...alone_plugins/dependency_graph/dependency_graph.py 77.87% 17 Missing and 8 partials ⚠️
...oubadix/standalone_plugins/dependency_graph/cli.py 68.00% 7 Missing and 1 partial ⚠️
...adix/standalone_plugins/dependency_graph/checks.py 95.55% 1 Missing and 1 partial ⚠️
...adix/standalone_plugins/dependency_graph/models.py 96.55% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #774      +/-   ##
==========================================
+ Coverage   79.81%   80.38%   +0.57%     
==========================================
  Files          87       91       +4     
  Lines        3022     3212     +190     
  Branches      589      606      +17     
==========================================
+ Hits         2412     2582     +170     
- Misses        462      478      +16     
- Partials      148      152       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

github-actions bot commented Dec 12, 2024

Dependency Review

The following issues were found:
  • ✅ 0 vulnerable package(s)
  • ✅ 0 package(s) with incompatible licenses
  • ✅ 0 package(s) with invalid SPDX license definitions
  • ⚠️ 2 package(s) with unknown licenses.
See the Details below.

Snapshot Warnings

⚠️: No snapshots were found for the head SHA 3a529e5.
Ensure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice.

License Issues

poetry.lock

PackageVersionLicenseIssue Type
networkx3.4.2NullUnknown License

pyproject.toml

PackageVersionLicenseIssue Type
networkx^ 3.4.2NullUnknown License
Allowed Licenses: 0BSD, AGPL-3.0-or-later, Apache-2.0, BlueOak-1.0.0, BSD-2-Clause, BSD-3-Clause-Clear, BSD-3-Clause, BSL-1.0, CAL-1.0, CC-BY-3.0, CC-BY-4.0, CC-BY-SA-4.0, CC0-1.0, EPL-2.0, GPL-2.0-only, GPL-2.0-or-later, GPL-2.0, GPL-3.0-or-later, ISC, LGPL-2.0-only, LGPL-2.0-or-later, LGPL-2.1-only, LGPL-2.1-or-later, LGPL-2.1, LGPL-3.0-only, LGPL-3.0, LGPL-3.0-or-later, MIT, MIT-CMU, MPL-1.1, MPL-2.0, OFL-1.1, PSF-2.0, Python-2.0, Python-2.0.1, Unicode-DFS-2016, Unlicense

OpenSSF Scorecard

PackageVersionScoreDetails
pip/networkx 3.4.2 🟢 4.9
Details
CheckScoreReason
Code-Review🟢 9Found 26/27 approved changesets -- score normalized to 9
Maintained🟢 1030 commit(s) and 16 issue activity found in the last 90 days -- score normalized to 10
Dangerous-Workflow⚠️ 0dangerous workflow patterns detected
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
License🟢 9license file detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Vulnerabilities🟢 100 existing vulnerabilities detected
Binary-Artifacts🟢 10no binaries found in the repo
Signed-Releases⚠️ -1no releases found
Branch-Protection⚠️ 0branch protection not enabled on development/release branches
Packaging🟢 10packaging workflow detected
Security-Policy⚠️ 0security policy file not detected
Pinned-Dependencies⚠️ 0dependency not pinned by hash detected -- score normalized to 0
Fuzzing🟢 10project is fuzzed
SAST⚠️ 0SAST tool is not run on all commits -- score normalized to 0
pip/networkx ^ 3.4.2 🟢 4.9
Details
CheckScoreReason
Code-Review🟢 9Found 26/27 approved changesets -- score normalized to 9
Maintained🟢 1030 commit(s) and 16 issue activity found in the last 90 days -- score normalized to 10
Dangerous-Workflow⚠️ 0dangerous workflow patterns detected
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
License🟢 9license file detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Vulnerabilities🟢 100 existing vulnerabilities detected
Binary-Artifacts🟢 10no binaries found in the repo
Signed-Releases⚠️ -1no releases found
Branch-Protection⚠️ 0branch protection not enabled on development/release branches
Packaging🟢 10packaging workflow detected
Security-Policy⚠️ 0security policy file not detected
Pinned-Dependencies⚠️ 0dependency not pinned by hash detected -- score normalized to 0
Fuzzing🟢 10project is fuzzed
SAST⚠️ 0SAST tool is not run on all commits -- score normalized to 0

Scanned Files

  • poetry.lock
  • pyproject.toml

troubadix/standalone_plugins/dependency_graph.py Outdated Show resolved Hide resolved
troubadix/standalone_plugins/dependency_graph.py Outdated Show resolved Hide resolved
troubadix/standalone_plugins/dependency_graph.py Outdated Show resolved Hide resolved
troubadix/standalone_plugins/dependency_graph.py Outdated Show resolved Hide resolved
troubadix/standalone_plugins/dependency_graph.py Outdated Show resolved Hide resolved
troubadix/standalone_plugins/dependency_graph.py Outdated Show resolved Hide resolved
troubadix/standalone_plugins/dependency_graph.py Outdated Show resolved Hide resolved
troubadix/standalone_plugins/dependency_graph.py Outdated Show resolved Hide resolved
checks for a script depending on a script multiple times
"""
warnings = []
for script in scripts:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to come back to this, reminder to myself



def cross_feed_dependencies(graph, gated_status: bool) -> list[tuple[str, str]]:
"""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to come back to this

Copy link
Contributor

@amy-gb amy-gb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried it locally and it works great! Left a couple of small comments but I will check back when Max has finished his review

troubadix/standalone_plugins/dependency_graph.py Outdated Show resolved Hide resolved
troubadix/standalone_plugins/dependency_graph.py Outdated Show resolved Hide resolved
troubadix/standalone_plugins/dependency_graph.py Outdated Show resolved Hide resolved
troubadix/standalone_plugins/dependency_graph.py Outdated Show resolved Hide resolved
@mbrinkhoff mbrinkhoff mentioned this pull request Jan 29, 2025
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants