Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new unified fields to Mend and Blackduck SARIF #4635

Closed
wants to merge 58 commits into from

Conversation

andrew-kireev
Copy link
Member

@andrew-kireev andrew-kireev commented Oct 17, 2023

Changes

  • Tests
  • Documentation

@andrew-kireev andrew-kireev requested a review from a team as a code owner October 17, 2023 08:35
andrew-kireev and others added 29 commits October 17, 2023 11:04
* add comments with examples to methods

* a bit refactoring and cleanup

* actionsURL

* GetBuildStatus

* GetBuildID, GetChangeSet, GetPipelineStartTime

* GetStageName and GetBuildReason

* refactor fetching jobs

* GetJobName and GetJobURL

* chnage GetBuildURL

* refactor actionsURL

* fix guessCurrentJob bug

* unit tests for all

* refactor GetLog

* refactor and fix tests

* change GetBuildURL to use env vars

* fix issues

* leftover

* add comment

* fix according to review comments

---------

Co-authored-by: Gulom Alimov <[email protected]>
Co-authored-by: Jordi van Liempt <[email protected]>
* fix(deps): update module github.com/google/uuid to v1.3.1

* run go mod tidy

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Muhammadali Nazarov <[email protected]>
Co-authored-by: Jordi van Liempt <[email protected]>
…4541)

* feat(logging): print out commit sha of code used to build the binary

* Update piper.go
* Initial in progress

* compiling but not yet functional

* Missed file

* updated checkmarxone step

* Working up to fetching a project then breaks

* Missed file

* Breaks when retrieving projects+proxy set

* Create project & run scan working, now polling

* Fixed polling

* added back the zipfile remove command

* Fixed polling again

* Generates and downloads PDF report

* Updated and working, prep for refactor

* Added compliance steps

* Cleanup, reporting, added groovy connector

* fixed groovy file

* checkmarxone to checkmarxOne

* checkmarxone to checkmarxOne

* split credentials (id+secret, apikey), renamed pullrequestname to branch, groovy fix

* Fixed filenames & yaml

* missed the metadata_generated.go

* added json to sarif conversion

* fix:type in new checkmarxone package

* fix:type in new checkmarxone package

* removed test logs, added temp error log for creds

* extra debugging to fix crash

* improved auth logging, fixed query parse issue

* fixed bug with group fetch when using oauth user

* CWE can be -1 if not defined, can't be uint

* Query also had CweID

* Disabled predicates-fetch in sarif generation

* Removing leftover info log message

* Better error handling

* fixed default preset configuration

* removing .bat files - sorry

* Cleanup per initial review

* refactoring per Gist, fixed project find, add apps

* small fix - sorry for commit noise while testing

* Fixing issues with incremental scans.

* removing maxretries

* Updated per PR feedback, further changes todo toda

* JSON Report changes and reporting cleanup

* removing .bat (again?)

* adding docs, groovy unit test, linter fixes

* Started adding tests maybe 15% covered

* fix(checkmarxOne): test cases for pkg and reporting

* fix(checkmarxOne):fix formatting

* feat(checkmarxone): update interface with missing method

* feat(checkmarxone):change runStep signature to be able to inject dependency

* feat(checkmarxone): add tests for step (wip)

* Adding a bit more coverage

* feat(checkmarxOne): fix code review

* feat(checkmarxOne): fix code review

* feat(checkmarxOne): fix code review

* feat(checkmarxOne): fix integration test PR

* adding scan-summary bug workaround, reportgen fail

* enforceThresholds fix when no results passed in

* fixed gap when preset empty in yaml & project conf

* fixed another gap in preset selection

* fix 0-result panic

* fail when no preset is set anywhere

* removed comment

* initial project-under-app support

* fixing sarif reportgen

* some cleanup of error messages

* post-merge test fixes

* revert previous upstream merge

* fix:formatting

* fix(checkmarxOne):yamllint too many blank lines

* fix(checkmarxOne):unit test

* fix(checkmarxOne):generated code

---------

Co-authored-by: thtri <[email protected]>
Co-authored-by: Thanh-Hai Trinh <[email protected]>
…ity] (#4427)

* fix(deps): update module github.com/hashicorp/vault to v1.13.5 [security]

* fix(deps): update module github.com/Azure/azure-sdk-for-go/tree/sdk/storage/azblob to v0.4.1

* fix(deps): update module github.com/hashicorp/vault/sdk to v0.9.2
fix(deps): update module oras.land/oras-go to v1.2.3

* fix(deps): update module github.com/hashicorp/vault/sdk to v0.9.2-0.20230530190758-08ee474850e0
fix(deps): update module github.com/hashicorp/vault/sdk to v0.9.2-0.20230530190758-08ee474850e0

* replacing deprecated function

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Muhammadali Nazarov <[email protected]>
Co-authored-by: Jordi van Liempt <[email protected]>
* Add cnbBuild to buildExecute

* Error message change

* Make if check simpler

Co-authored-by: Alexander Link <[email protected]>

* Switch order of check

---------

Co-authored-by: Linda Siebert <[email protected]>
Co-authored-by: Linda Siebert <[email protected]>
Co-authored-by: Alexander Link <[email protected]>
Add option to opt out from helm template parsing

Co-authored-by: Linda Siebert <[email protected]>
Co-authored-by: Alexander Link <[email protected]>
)

* fix(docs): fixing documentation for violating markdownlint rules.

* correct indent

* Apply suggestions from code review

* correct list

---------

Co-authored-by: Christopher Fenner <[email protected]>
* encrypt CPE - init

* fix

* disable encrypt on Jenkins

* get PIPER_pipelineEnv_SECRET from vault

* reuse artifactPrepareVersionOptions

* encrypt only with orchestrator.GitHubActions

* Workaround: orchestrators expect json

* add encryptedCPE flag

* remove JSON workaround

* throw error if stepConfigPassword is empty

* fix log messages

---------

Co-authored-by: Egor Balakin <[email protected]>
* Enable wdi5 autologin

By also providing the basic auth credential on the env vars wdi5_username and wdi5_password we enable the wdi5 autologin feature, where the user does not have to remap the credentials in their wdi5 configuration. See https://ui5-community.github.io/wdi5/#/authentication?id=credentials

* Add documentation

* Add wdi5 parameter

* Add tests
…sage of go >1.21 (#4558)

* chore: ⬆️ upgrade to golang.org/x/mod/modfile v0.12.0 to enable the usage of go >1.21

* chore: run

---------

Co-authored-by: Jk1484 <[email protected]>
…ion issue (#4565)

* removed quotation for detect8

* changed order of args

* chagned order
* Use new Golang implementation of tmsUpload step by default
…ons (#4557)

* Allow running as different user on Kubernetes

Co-authored-by: Ralf Pannemans <[email protected]>
Co-authored-by: Johannes Dillmann <[email protected]>
Co-authored-by: Pavel Busko <[email protected]>

* infer securityContext from dockerOptions

Co-authored-by: Ralf Pannemans <[email protected]>
Co-authored-by: Pavel Busko <[email protected]>

* verify --user flag value

---------

Co-authored-by: Johannes Dillmann <[email protected]>
Co-authored-by: Ralf Pannemans <[email protected]>
Co-authored-by: Anil Keshav <[email protected]>
* made detect8 default

* amended tests to reflect detect 8

* amended

* amend

* amend

* amend tests

* 1

* 1

* tests-with-temp-changes-for-transition

* removed auto unmapping for detect7

* added-old-parameters-as-deprecated
renovate bot and others added 23 commits October 17, 2023 11:04
* fix(deps): update module golang.org/x/oauth2 to v0.12.0

* run go mod tidy

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: jliempt <>
* vault retry check function

---------

Co-authored-by: I557621 <[email protected]>
…ntials config (#4600)

* added logging if unauthorized for github

* refactored

* fixed log message & added logging github response

* deleted extra log

* refactored log message
… commit to trust enterprise github instances (#4602)

* downloading ca cert bundle when added as config

* adding logging statements

* allowing bats test to handle ca cert

* adding info message

* hard coding file names

* including correct http client util bundle

* removing logging message not needed

* adding cert bundle to commit and push

* improving the condition to add ca cert in commit and push

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test
* add containerDockerfilePath param to multipleImages

* rename ContainerDockerfilePath param to DockerfilePath

* Fix trailing spaces

---------

Co-authored-by: Egor Balakin <[email protected]>
Co-authored-by: Vyacheslav Starostin <[email protected]>
…yer (#4604)

forward sourceRepositoryCredentialsId from groovy to go layer in the same way how this is done for the targetRepositoryCredentialsId
…anges (#4605)

* exporting generateConfig function and applying minor changes

* Added setConfigOptions to set configOptions variable.
Added possibility to set format output, json or yaml for now.

* Correcting mistake on cmd/getDefaults.go

Co-authored-by: Jordi van Liempt <[email protected]>

---------

Co-authored-by: Jordi van Liempt <[email protected]>
…4595)

* added logic of fetching golang private packages for whitesource step and detectExecuteScan step

* changed logic of checking by config.PrivateModulesGitToken

* moved func prepareGolangPrivatePackages to golangBuild.go

* fix (gitOpsUpdateDeployment) add CA bundle options to plain clone and commit to trust enterprise github instances (#4602)

* downloading ca cert bundle when added as config

* adding logging statements

* allowing bats test to handle ca cert

* adding info message

* hard coding file names

* including correct http client util bundle

* removing logging message not needed

* adding cert bundle to commit and push

* improving the condition to add ca cert in commit and push

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test

* feat(kanikoExecute): add dockerfilePath param to multipleImages (#4569)

* add containerDockerfilePath param to multipleImages

* rename ContainerDockerfilePath param to DockerfilePath

* Fix trailing spaces

---------

Co-authored-by: Egor Balakin <[email protected]>
Co-authored-by: Vyacheslav Starostin <[email protected]>

* fix(helm): forward sourceRepositoryCredentialsId from groovy to go layer (#4604)

forward sourceRepositoryCredentialsId from groovy to go layer in the same way how this is done for the targetRepositoryCredentialsId

* feat(config): exporting generateConfig function and applying minor changes (#4605)

* exporting generateConfig function and applying minor changes

* Added setConfigOptions to set configOptions variable.
Added possibility to set format output, json or yaml for now.

* Correcting mistake on cmd/getDefaults.go

Co-authored-by: Jordi van Liempt <[email protected]>

---------

Co-authored-by: Jordi van Liempt <[email protected]>

* moved func prepareGolangPrivatePackages to pkg/golang

---------

Co-authored-by: Akramdzhon Azamov <[email protected]>
Co-authored-by: Andrei Kireev <[email protected]>
Co-authored-by: Anil Keshav <[email protected]>
Co-authored-by: Egor Balakin <[email protected]>
Co-authored-by: Egor Balakin <[email protected]>
Co-authored-by: Vyacheslav Starostin <[email protected]>
Co-authored-by: Marcus Holl <[email protected]>
Co-authored-by: Jk1484 <[email protected]>
Co-authored-by: Jordi van Liempt <[email protected]>
#4579)

* enabling publish to only publish sub packages

* changing directory and then coming back to original after the publish runs

* searching the glob tar and npmrc in the current directory

* excluding build descriptor check and addtional target tool check

* changing the npm pack before publish to run only in sub packages

* removing commented code clean up

* adding the correct npm pack

* improve logging

* fix error handling and a bit style fix

* fix unit tests

* remove commented lines

* respecting build descriptor list when provided

* improve docu for the step param

* fixing linting issues

* improve docu

---------

Co-authored-by: Gulom Alimov <[email protected]>
Co-authored-by: Jordi van Liempt <[email protected]>
…be stashed/unstashed (#4573)

* Add stash param to mavenBuild

* go generate

* Delete default stash group

---------

Co-authored-by: Anil Keshav <[email protected]>
* Add CertificateDownload func to certutils package

* Add customTlsCertificateLinks param for artifactPrepareVersion

* Add the possibility to provide custom certs for artifactPrepareVersion

* Update tests

* Return back build flags

* Return back build flags

* Update pkg/certutils/certutils.go

Apply suggestion from code review

Co-authored-by: Christopher Fenner <[email protected]>

---------

Co-authored-by: Christopher Fenner <[email protected]>
* structure vault logs

* Update client.go
* Update default_pipeline_environment.yml

Add filterRegex parameter to jmeter configuration. Set it to an empty string as generated from pipelinesyntax reference in jenkins

* Update testsPublishResults.groovy

Add the filterRegex param to the publishJmeterReport plugin in testPublishResults

* Update documentation to include filterRegex

---------

Co-authored-by: Anil Keshav <[email protected]>
* Adapting unit test to run for packages with namespace

* Handling classes with namespace

* support namespace obj for ATC
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.15.0 to 0.17.0.
- [Commits](golang/net@v0.15.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…pload and export to node (#4624)

* Provide additional log message on successful upload and export to node

---------

Co-authored-by: Oliver Feldmann <[email protected]>
…ity] (#4550)

* fix(deps): update module github.com/hashicorp/vault to v1.14.3 [security]

* update to v1.14.1 instead due to Go version

* github.com/hashicorp/[email protected]

* go mod tidy

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: jliempt <>
Co-authored-by: Jordi van Liempt <[email protected]>
* fix(cxone): 1st scan failed due to 403 forbiden

* fix(cxone): 1st scan failed due to 403 forbiden

* fix(cxone): 1st scan failed due to 403 forbiden
…nks parameter (#4631)

* Add conditions for customTlsCertificateLinks

* go generate
@andrew-kireev andrew-kireev force-pushed the andrew-kireev/add-new-fields-to-sarif branch from 5584068 to c069796 Compare October 17, 2023 09:06
@andrew-kireev andrew-kireev requested review from inf2381 and a team as code owners October 17, 2023 09:06
@sonarcloud
Copy link

sonarcloud bot commented Oct 17, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@andrew-kireev andrew-kireev deleted the andrew-kireev/add-new-fields-to-sarif branch October 17, 2023 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.