Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable lifecycle release bundle distribute #2185

Merged
merged 4 commits into from
Sep 13, 2023
Merged

Enable lifecycle release bundle distribute #2185

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c3f774b
Enable lifecycle release bundle distribute
RobiNino Sep 5, 2023
9546fa7
Merge branch 'dev' of https://github.com/jfrog/jfrog-cli into rbv2-di…
RobiNino Sep 13, 2023
db422fa
update go mod
RobiNino Sep 13, 2023
c2ace4f
Merge branch 'dev' of https://github.com/jfrog/jfrog-cli into rbv2-di…
RobiNino Sep 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions documentation/CLI-for-JFrog-Lifecycle.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,3 +121,69 @@ Use signing key pair "myKeyPair" and overwrite in case of conflicts.
```
jf rbp --signing-key=myKeyPair --project=project0 --overwrite=true --sync=true myApp 1.0.0 PROD
```

### Distributing a release bundle

This command distributes a Release Bundle v2 to an edge node.

| | |
|------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Command-name | release-bundle-distribute |
| Abbreviation | rbd |
| Command options | |
| --create-repo | \[Default: false\]<br><br>Set to true to create the repository on the edge if it does not exist. |
| --dry-run | \[Default: false\]<br><br>Set to true to disable communication with JFrog Distribution. |
| --dist-rules | \[Optional\]<br><br>Path to a file, which includes the Distribution Rules in a JSON format. See [structure](#distribution-rules-structure) bellow. |
| --site | \[Default: *\]<br><br>Wildcard filter for site name. |
| --city | \[Default: *\]<br><br>Wildcard filter for site city name. |
| --country-codes | \[Default: *\]<br><br>Semicolon-separated list of wildcard filters for site country codes. |
| --insecure-tls | \[Default: false\]<br><br>Set to true to skip TLS certificates verification. |
| --mapping-pattern | \[Optional\]<br><br>Specify along with 'mapping-target' to distribute artifacts to a different path on the edge node. You can use wildcards to specify multiple artifacts. |
| --mapping-target | \[Optional\]<br><br>The target path for distributed artifacts on the edge node. If not specified, the artifacts will have the same path and name on the edge node, as on the source Artifactory server. For flexibility in specifying the distribution path, you can include [placeholders](https://www.jfrog.com/confluence/display/CLI/CLI+for+JFrog+Artifactory#CLIforJFrogArtifactory-UsingPlaceholders) in the form of {1}, {2} which are replaced by corresponding tokens in the pattern path that are enclosed in parenthesis. |
| Command arguments | |
| release bundle name | Name of the Release Bundle to distribute. |
| release bundle version | Version of the Release Bundle to distribute. |

#### Distribution Rules Structure:
```json
{
"distribution_rules": [
{
"site_name": "DC-1",
"city_name": "New-York",
"country_codes": ["1"]
},
{
"site_name": "DC-2",
"city_name": "Tel-Aviv",
"country_codes": ["972"]
}
]
}
```
The Distribution Rules format also supports wildcards. For example:
```json
{
"distribution_rules": [
{
"site_name": "",
"city_name": "",
"country_codes": ["*"]
}
]
}
```

#### Examples

#### Example 1
Distribute the Release Bundle v2 named myApp with version 1.0.0. Use the distribution rules defined in the specified file.

jf rbd --dist-rules=/path/to/dist-rules.json myApp 1.0.0

#### Example 2

Distribute the Release Bundle v2 named myApp with version 1.0.0 using the default distribution rules.
Map files under the 'source' directory to be placed under the 'target' directory.

jf rbd --dist-rules=/path/to/dist-rules.json --mapping-pattern="(*)/source/(*)" --mapping-target="{1}/target/{2}" myApp 1.0.0
10 changes: 10 additions & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,3 +123,13 @@ require (
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)

// replace github.com/jfrog/build-info-go => github.com/jfrog/build-info-go v1.8.9-0.20230828134416-f0db33dd9344

// replace github.com/jfrog/jfrog-cli-core/v2 => github.com/jfrog/jfrog-cli-core/v2 v2.31.1-0.20230904170814-03a7938bae94

// replace github.com/jfrog/gofrog => github.com/jfrog/gofrog v1.2.6-0.20230418122323-2bf299dd6d27

replace github.com/jfrog/jfrog-client-go => github.com/jfrog/jfrog-client-go v1.28.1-0.20230913164821-c396cb9a4e06

// replace github.com/jfrog/build-info-go => github.com/jfrog/build-info-go v1.8.9-0.20230831151231-e5e7bd035ddc
4 changes: 2 additions & 2 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -241,8 +241,8 @@ github.com/jfrog/gofrog v1.3.0 h1:o4zgsBZE4QyDbz2M7D4K6fXPTBJht+8lE87mS9bw7Gk=
github.com/jfrog/gofrog v1.3.0/go.mod h1:IFMc+V/yf7rA5WZ74CSbXe+Lgf0iApEQLxRZVzKRUR0=
github.com/jfrog/jfrog-cli-core/v2 v2.43.0 h1:euo1CjZcpMdWkFUQ3zffRPfCR1zXhLD6TE/lfexV99o=
github.com/jfrog/jfrog-cli-core/v2 v2.43.0/go.mod h1:NWqT0ZnAvEdjaXGp64POvRV35TJ2R/c0W45UmrXQonk=
github.com/jfrog/jfrog-client-go v1.32.2 h1:t0ceWCtFri+xsa0D2ESqD/itcovlxBXCky1A1MJ4P2I=
github.com/jfrog/jfrog-client-go v1.32.2/go.mod h1:UewnwkIf/77HzBgwCPzOHZCK6V/Nw5/JwdzN/tRb4aU=
github.com/jfrog/jfrog-client-go v1.28.1-0.20230913164821-c396cb9a4e06 h1:QcREjpuUJmLsy7aSXTcMomAIVFgOsdUHGi7lcqVfg8M=
github.com/jfrog/jfrog-client-go v1.28.1-0.20230913164821-c396cb9a4e06/go.mod h1:UewnwkIf/77HzBgwCPzOHZCK6V/Nw5/JwdzN/tRb4aU=
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
github.com/jszwec/csvutil v1.8.0 h1:G7vS2LGdpZZDH1HmHeNbxOaJ/ZnJlpwGFvOkTkJzzNk=
Expand Down
3 changes: 1 addition & 2 deletions lifecycle/cli.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,6 @@ func GetCommands() []cli.Command {
ArgsUsage: common.CreateEnvVars(),
BashComplete: coreCommon.CreateBashCompletionFunc(),
Category: lcCategory,
Hidden: true,
Action: distribute,
},
})
Expand Down Expand Up @@ -170,7 +169,7 @@ func assertSigningKeyProvided(c *cli.Context) error {
}

func createLifecycleDetailsByFlags(c *cli.Context) (*coreConfig.ServerDetails, error) {
lcDetails, err := cliutils.CreateServerDetailsWithConfigOffer(c, false, cliutils.Platform)
lcDetails, err := cliutils.CreateServerDetailsWithConfigOffer(c, true, cliutils.Platform)
if err != nil {
return nil, err
}
Expand Down
33 changes: 27 additions & 6 deletions lifecycle_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,14 @@ import (
"github.com/jfrog/jfrog-client-go/lifecycle"
"github.com/jfrog/jfrog-client-go/lifecycle/services"
clientUtils "github.com/jfrog/jfrog-client-go/utils"
"github.com/jfrog/jfrog-client-go/utils/distribution"
"github.com/jfrog/jfrog-client-go/utils/errorutils"
"github.com/stretchr/testify/assert"
"net/http"
"os"
"path/filepath"
"testing"
"time"
)

const (
Expand Down Expand Up @@ -64,14 +66,17 @@ func TestLifecycle(t *testing.T) {
promoteRb(t, lcManager, number3)

// Verify the artifacts of both the initial release bundles made it to the prod repo.
searchSpec, err := tests.CreateSpec(tests.SearchAllProdRepo)
searchProdSpec, err := tests.CreateSpec(tests.SearchAllProdRepo)
assert.NoError(t, err)
inttestutils.VerifyExistInArtifactory(tests.GetExpectedLifecycleArtifacts(), searchSpec, serverDetails, t)
inttestutils.VerifyExistInArtifactory(tests.GetExpectedLifecycleArtifacts(), searchProdSpec, serverDetails, t)

distributeRb(t)
defer remoteDeleteReleaseBundle(t, lcManager, tests.LcRbName3, number3)

// Verify the artifacts were distributed correctly by the provided path mappings.
expected := append(tests.GetExpectedLifecycleArtifacts(), tests.GetExpectedLifecycleMappingArtifacts()...)
inttestutils.VerifyExistInArtifactory(expected, searchSpec, serverDetails, t)
searchDevSpec, err := tests.CreateSpec(tests.SearchAllDevRepo)
assert.NoError(t, err)
inttestutils.VerifyExistInArtifactory(tests.GetExpectedLifecycleDistributedArtifacts(), searchDevSpec, serverDetails, t)
}

func uploadBuilds(t *testing.T) func() {
Expand Down Expand Up @@ -107,9 +112,11 @@ func distributeRb(t *testing.T) {
assert.NoError(t, lcCli.Exec(
"rbd", tests.LcRbName3, number3,
getOption(cliutils.DistRules, distributionRulesPath),
getOption(cliutils.PathMappingPattern, tests.RtProdRepo+"/(*)"),
getOption(cliutils.PathMappingTarget, tests.RtProdRepo+"/target/{1}"),
getOption(cliutils.PathMappingPattern, "(*)/(*)"),
getOption(cliutils.PathMappingTarget, "{1}/target/{2}"),
))
// Wait after distribution before asserting. Can be removed once distribute supports sync.
time.Sleep(5 * time.Second)
}

func getOption(option, value string) string {
Expand Down Expand Up @@ -187,6 +194,20 @@ func deleteReleaseBundle(t *testing.T, lcManager *lifecycle.LifecycleServicesMan
assert.NoError(t, lcManager.DeleteReleaseBundle(rbDetails, services.ReleaseBundleQueryParams{Async: false}))
}

func remoteDeleteReleaseBundle(t *testing.T, lcManager *lifecycle.LifecycleServicesManager, rbName, rbVersion string) {
params := distribution.NewDistributeReleaseBundleParams(rbName, rbVersion)
rules := &distribution.DistributionCommonParams{
SiteName: "*",
CityName: "*",
CountryCodes: []string{"*"},
}
params.DistributionRules = append(params.DistributionRules, rules)

assert.NoError(t, lcManager.RemoteDeleteReleaseBundle(params, false))
// Wait after remote deleting. Can be removed once remote deleting supports sync.
time.Sleep(5 * time.Second)
}

func uploadBuild(t *testing.T, specFileName, buildName, buildNumber string) {
specFile, err := tests.CreateSpec(specFileName)
assert.NoError(t, err)
Expand Down
7 changes: 7 additions & 0 deletions testdata/filespecs/search_all_dev_repo.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"files": [
{
"pattern": "${DEV_REPO}/*"
}
]
}
32 changes: 21 additions & 11 deletions utils/tests/consts.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ const (
SearchAllMaven = "search_all_maven.json"
SearchAllNpm = "search_all_npm.json"
SearchAllRepo1 = "search_all_repo1.json"
SearchAllDevRepo = "search_all_dev_repo.json"
SearchAllProdRepo = "search_all_prod_repo.json"
SearchDistRepoByInSuffix = "search_dist_repo_by_in_suffix.json"
SearchRepo1ByInSuffix = "search_repo1_by_in_suffix.json"
Expand Down Expand Up @@ -2099,17 +2100,26 @@ func GetExpectedLifecycleArtifacts() []string {
}
}

func GetExpectedLifecycleMappingArtifacts() []string {
return []string{
RtProdRepo + "/target/a1.in",
RtProdRepo + "/target/a2.in",
RtProdRepo + "/target/a3.in",
RtProdRepo + "/target/b1.in",
RtProdRepo + "/target/b2.in",
RtProdRepo + "/target/b3.in",
RtProdRepo + "/target/c1.in",
RtProdRepo + "/target/c2.in",
RtProdRepo + "/target/c3.in",
func GetExpectedLifecycleDistributedArtifacts() []string {
return []string{
RtDevRepo + "/a1.in",
RtDevRepo + "/a2.in",
RtDevRepo + "/a3.in",
RtDevRepo + "/b1.in",
RtDevRepo + "/b2.in",
RtDevRepo + "/b3.in",
RtDevRepo + "/c1.in",
RtDevRepo + "/c2.in",
RtDevRepo + "/c3.in",
RtDevRepo + "/target/a1.in",
RtDevRepo + "/target/a2.in",
RtDevRepo + "/target/a3.in",
RtDevRepo + "/target/b1.in",
RtDevRepo + "/target/b2.in",
RtDevRepo + "/target/b3.in",
RtDevRepo + "/target/c1.in",
RtDevRepo + "/target/c2.in",
RtDevRepo + "/target/c3.in",
}
}

Expand Down
Loading