Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Go, NPM and Yarn detection #451

Merged
merged 18 commits into from
Oct 3, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions scripts/dev_scripts/integration_tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,56 @@ $RUN_MACARON analyze -rp https://github.com/timyarkov/docker_test -b main -d 404

python $COMPARE_JSON_OUT $JSON_RESULT $JSON_EXPECTED || log_fail

echo -e "\n----------------------------------------------------------------------------------"
echo "uiv-lib/uiv: Analysing the repo path, the branch name and the commit digest for an npm project,"
echo "skipping dependency resolution."
echo -e "----------------------------------------------------------------------------------\n"
JSON_EXPECTED=$WORKSPACE/tests/e2e/expected_results/uiv/uiv.json
JSON_RESULT=$WORKSPACE/output/reports/github_com/uiv-lib/uiv/uiv.json
$RUN_MACARON analyze -rp https://github.com/uiv-lib/uiv -b dev -d 057b25b4db0913edab4cf728c306085e6fc20d49 --skip-deps || log_fail

python $COMPARE_JSON_OUT $JSON_RESULT $JSON_EXPECTED || log_fail

echo -e "\n----------------------------------------------------------------------------------"
echo "onu-ui/onu-ui: Analysing the repo path, the branch name and the commit digest for a pnpm project,"
echo "skipping dependency resolution."
echo -e "----------------------------------------------------------------------------------\n"
JSON_EXPECTED=$WORKSPACE/tests/e2e/expected_results/onu-ui/onu-ui.json
JSON_RESULT=$WORKSPACE/output/reports/github_com/onu-ui/onu-ui/onu-ui.json
$RUN_MACARON analyze -rp https://github.com/onu-ui/onu-ui -b main -d e3f2825c3940002a920d65476116a64684b3d95e --skip-deps || log_fail

python $COMPARE_JSON_OUT $JSON_RESULT $JSON_EXPECTED || log_fail

echo -e "\n----------------------------------------------------------------------------------"
echo "facebook/yoga: Analysing the repo path, the branch name and the commit digest for a Yarn classic"
echo "project, skipping dependency resolution."
echo -e "----------------------------------------------------------------------------------\n"
JSON_EXPECTED=$WORKSPACE/tests/e2e/expected_results/yoga/yoga.json
JSON_RESULT=$WORKSPACE/output/reports/github_com/facebook/yoga/yoga.json
$RUN_MACARON analyze -rp https://github.com/facebook/yoga -b main -d f8e2bc0875c145c429d0e865c9b83a40f65b3070 --skip-deps || log_fail

python $COMPARE_JSON_OUT $JSON_RESULT $JSON_EXPECTED || log_fail

echo -e "\n----------------------------------------------------------------------------------"
echo "wojtekmaj/react-pdf: Analysing the repo path, the branch name and the commit digest for a Yarn modern"
echo "project, skipping dependency resolution."
echo -e "----------------------------------------------------------------------------------\n"
JSON_EXPECTED=$WORKSPACE/tests/e2e/expected_results/react-pdf/react-pdf.json
JSON_RESULT=$WORKSPACE/output/reports/github_com/wojtekmaj/react-pdf/react-pdf.json
$RUN_MACARON analyze -rp https://github.com/wojtekmaj/react-pdf -b main -d be18436b7be827eb993b2e1e4bd9230dd835a9a3 --skip-deps || log_fail

python $COMPARE_JSON_OUT $JSON_RESULT $JSON_EXPECTED || log_fail

echo -e "\n----------------------------------------------------------------------------------"
echo "sigstore/sget: Analysing the repo path, the branch name and the"
echo "commit digest for a Go project, skipping dependency resolution."
echo -e "----------------------------------------------------------------------------------\n"
JSON_EXPECTED=$WORKSPACE/tests/e2e/expected_results/sget/sget.json
JSON_RESULT=$WORKSPACE/output/reports/github_com/sigstore/sget/sget.json
$RUN_MACARON analyze -rp https://github.com/sigstore/sget -b main -d 99e7b91204d391ccc76507f7079b6d2a7957489e --skip-deps || log_fail

python $COMPARE_JSON_OUT $JSON_RESULT $JSON_EXPECTED || log_fail

echo -e "\n----------------------------------------------------------------------------------"
echo "apache/maven: Analyzing with PURL and repository path without dependency resolution."
echo -e "----------------------------------------------------------------------------------\n"
Expand Down
65 changes: 65 additions & 0 deletions src/macaron/config/defaults.ini
Original file line number Diff line number Diff line change
Expand Up @@ -318,6 +318,71 @@ deploy_arg =
github_actions =
docker/build-push-action

# This is the spec for trusted NPM build tool usages.
# This also includes the spec for pnpm, since for the purposes of this program
# they work more or less the same.
[builder.npm]
entry_conf =
.npmrc
build_configs =
package.json
package_lock =
package-lock.json
pnpm-lock.yaml
builder =
npm
pnpm
# Build args not defined since npm build is just a plumbing command https://docs.npmjs.com/cli/v6/commands/npm-build
# and SLSA v1.0 removes the scripted build requirement https://slsa.dev/spec/v1.0/requirements
build_arg =
deploy_arg =
publish
[builder.npm.ci.deploy]
github_actions =
JS-DevTools/npm-publish

# This is the spec for trusted Yarn build tool usages.
# The entries need to cover both Yarn classic and Yarn modern; namely .yarnrc vs .yarnrc.yml
# for the entry configs, publish vs npm publish for the deploy args, and yarn.lock for
# Yarn classic's package lock.
# See https://yarnpkg.com/migration/guide and https://classic.yarnpkg.com/lang/en/docs/yarn-lock/
# Currently npm publish is not supported here due to the multiple arguments; see issue #493.
[builder.yarn]
entry_conf =
.yarnrc
.yarnrc.yml
build_configs =
package.json
package_lock =
package-lock.json
yarn.lock
builder =
yarn
# Build args not defined for similar reasons to npm
build_arg =
deploy_arg =
publish

# This is the spec for trusted Go build tool usages.
[builder.go]
entry_conf =
build_configs =
go.mod
go.sum
builder =
go
build_arg =
build
install
# Goreleaser will be taken as publisher as per https://github.com/oracle/macaron/milestone/14
publisher =
goreleaser
deploy_arg =
release
[builder.go.ci.deploy]
github_actions =
goreleaser/goreleaser-action

# This is the spec for GitHub Actions CI.
[ci.github_actions]
entry_conf =
Expand Down
5 changes: 4 additions & 1 deletion src/macaron/slsa_analyzer/build_tool/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,14 @@

from .base_build_tool import BaseBuildTool
from .docker import Docker
from .go import Go
from .gradle import Gradle
from .maven import Maven
from .npm import NPM
from .pip import Pip
from .poetry import Poetry
from .yarn import Yarn

# The list of supported build tools. The order of the list determine the order
# in which each build tool is checked against the target repository.
BUILD_TOOLS: list[BaseBuildTool] = [Gradle(), Maven(), Poetry(), Pip(), Docker()]
BUILD_TOOLS: list[BaseBuildTool] = [Gradle(), Maven(), Poetry(), Pip(), Docker(), NPM(), Yarn(), Go()]
85 changes: 85 additions & 0 deletions src/macaron/slsa_analyzer/build_tool/go.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# Copyright (c) 2023 - 2023, Oracle and/or its affiliates. All rights reserved.
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/.

"""This module contains the Go class which inherits BaseBuildTool.

This module is used to work with repositories that have Go.
"""

from macaron.config.defaults import defaults
from macaron.dependency_analyzer.dependency_resolver import DependencyAnalyzer, NoneDependencyAnalyzer
from macaron.slsa_analyzer.build_tool.base_build_tool import BaseBuildTool, file_exists


class Go(BaseBuildTool):
"""This class contains the information of the Go build tool."""

def __init__(self) -> None:
super().__init__(name="go")

def load_defaults(self) -> None:
"""Load the default values from defaults.ini."""
if "builder.go" in defaults:
for item in defaults["builder.go"]:
if hasattr(self, item):
setattr(self, item, defaults.get_list("builder.go", item))

if "builder.go.ci.deploy" in defaults:
for item in defaults["builder.go.ci.deploy"]:
if item in self.ci_deploy_kws:
self.ci_deploy_kws[item] = defaults.get_list("builder.go.ci.deploy", item)

def is_detected(self, repo_path: str) -> bool:
"""Return True if this build tool is used in the target repo.

Parameters
----------
repo_path : str
The path to the target repo.

Returns
-------
bool
True if this build tool is detected, else False.
"""
go_config_files = self.build_configs + self.entry_conf
for file in go_config_files:
if file_exists(repo_path, file):
return True

return False

def prepare_config_files(self, wrapper_path: str, build_dir: str) -> bool:
"""Prepare the necessary wrapper files for running the build.

Go doesn't require preparation, so return true.

Parameters
----------
wrapper_path : str
The path where all necessary wrapper files are located.
build_dir : str
The path of the build dir. This is where all files are copied to.

Returns
-------
bool
True if succeed else False.
"""
return True

def get_dep_analyzer(self, repo_path: str) -> DependencyAnalyzer:
"""Create a DependencyAnalyzer for the build tool.

Parameters
----------
repo_path: str
The path to the target repo.

Returns
-------
DependencyAnalyzer
The DependencyAnalyzer object.
"""
# TODO: Implement this method.
return NoneDependencyAnalyzer()
89 changes: 89 additions & 0 deletions src/macaron/slsa_analyzer/build_tool/npm.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Copyright (c) 2023 - 2023, Oracle and/or its affiliates. All rights reserved.
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/.

"""This module contains the NPM class which inherits BaseBuildTool.

This module is used to work with repositories that use npm/pnpm as its
build tool.
"""

from macaron.config.defaults import defaults
from macaron.dependency_analyzer.dependency_resolver import DependencyAnalyzer, NoneDependencyAnalyzer
from macaron.slsa_analyzer.build_tool.base_build_tool import BaseBuildTool, file_exists


class NPM(BaseBuildTool):
"""This class contains the information of the npm/pnpm build tool."""

def __init__(self) -> None:
super().__init__(name="npm")

def load_defaults(self) -> None:
"""Load the default values from defaults.ini."""
if "builder.npm" in defaults:
for item in defaults["builder.npm"]:
if hasattr(self, item):
setattr(self, item, defaults.get_list("builder.npm", item))

if "builder.npm.ci.deploy" in defaults:
for item in defaults["builder.npm.ci.deploy"]:
if item in self.ci_deploy_kws:
self.ci_deploy_kws[item] = defaults.get_list("builder.npm.ci.deploy", item)

def is_detected(self, repo_path: str) -> bool:
"""Return True if this build tool is used in the target repo.

Parameters
----------
repo_path : str
The path to the target repo.

Returns
-------
bool
True if this build tool is detected, else False.
"""
# TODO: When more complex build detection is being implemented, consider
# cases like .npmrc existing but not package-lock.json and whether
# they would still count as "detected"
npm_config_files = self.build_configs + self.package_lock + self.entry_conf
for file in npm_config_files:
if file_exists(repo_path, file):
return True

return False

def prepare_config_files(self, wrapper_path: str, build_dir: str) -> bool:
"""Prepare the necessary wrapper files for running the build.

npm/pnpm doesn't require preparation, so return true.

Parameters
----------
wrapper_path : str
The path where all necessary wrapper files are located.
build_dir : str
The path of the build dir. This is where all files are copied to.

Returns
-------
bool
True if succeed else False.
"""
return True

def get_dep_analyzer(self, repo_path: str) -> DependencyAnalyzer:
"""Create a DependencyAnalyzer for the build tool.

Parameters
----------
repo_path: str
The path to the target repo.

Returns
-------
DependencyAnalyzer
The DependencyAnalyzer object.
"""
# TODO: Implement this method.
return NoneDependencyAnalyzer()
Loading