Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DEVHAS-418]add dockerfile as an alternate name for Dockerfile #382

Merged
merged 2 commits into from
Aug 22, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions cdq-analysis/pkg/detect.go
Original file line number Diff line number Diff line change
Expand Up @@ -122,30 +122,31 @@
}
}
}
} else if f.Name() == DockerfileName {
// Check for Dockerfile
} else if f.Name() == DockerfileName || f.Name() == AlternateDockerfileName {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small suggestion: Rather than checking for alternate types of Dockerfile names, what if we convert the name to lower case and check against "dockerfile"?

Copy link
Contributor Author

@yangcao77 yangcao77 Aug 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

then any other mix-case names are also allowed? i.e. DoCKerFiLE?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess so? I don’t see the harm in allowing it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we still need to check the exact allowed name, due to

for _, path := range validDockerfileLocations {
dockerfilePath := dir + "/" + path
dockerfileBytes, err = DownloadFile(dockerfilePath)
if err == nil {
// if we get a 200, return
return dockerfileBytes, path, err
}

When looking for the files using endpoint, we still need to check for supported names.

But in the cdq analysis here, when we have the repo downloaded on disk, I can convert to lower case and check against that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

// Check for Dockerfile or dockerfile
// NOTE: if a Dockerfile is named differently, for example, Dockerfile.jvm;
// thats ok. As we finish iterating through all the files in the localpath
// we will read the devfile to ensure a Dockerfile has been referenced.
// However, if a Dockerfile is named differently and not referenced in the devfile
// it will go undetected
dockerfileContextMapFromRepo[context] = DockerfileName
dockerfileContextMapFromRepo[context] = f.Name()
isDockerfilePresent = true
} else if f.Name() == ContainerfileName {
// Check for Containerfile
dockerfileContextMapFromRepo[context] = ContainerfileName
isDockerfilePresent = true
} else if f.IsDir() && (f.Name() == DockerDir || f.Name() == HiddenDockerDir || f.Name() == BuildDir) {
// Check for docker/Dockerfile, .docker/Dockerfile and build/Dockerfile
// OR docker/dockerfile, .docker/dockerfile and build/dockerfile
// OR docker/Containerfile, .docker/Containerfile and build/Containerfile
dirName := f.Name()
dirPath := path.Join(curPath, dirName)
files, err := os.ReadDir(dirPath)
if err != nil {
return nil, nil, nil, nil, err
}

Check warning on line 147 in cdq-analysis/pkg/detect.go

View check run for this annotation

Codecov / codecov/patch

cdq-analysis/pkg/detect.go#L146-L147

Added lines #L146 - L147 were not covered by tests
for _, f := range files {
if f.Name() == DockerfileName || f.Name() == ContainerfileName {
if f.Name() == DockerfileName || f.Name() == AlternateDockerfileName || f.Name() == ContainerfileName {
dockerfileContextMapFromRepo[context] = path.Join(dirName, f.Name())
isDockerfilePresent = true
}
Expand All @@ -170,8 +171,8 @@
}

if len(devfilesURLMapFromRepo) == 0 && len(devfileMapFromRepo) == 0 && len(dockerfileContextMapFromRepo) == 0 {
// if we didnt find any devfile or Dockerfile we should return an err
log.Info(fmt.Sprintf("no devfile or Dockerfile found in the specified location %s", localpath))

Check warning on line 175 in cdq-analysis/pkg/detect.go

View check run for this annotation

Codecov / codecov/patch

cdq-analysis/pkg/detect.go#L174-L175

Added lines #L174 - L175 were not covered by tests
}

return devfileMapFromRepo, devfilesURLMapFromRepo, dockerfileContextMapFromRepo, componentPortsMapFromRepo, err
Expand Down Expand Up @@ -256,9 +257,9 @@
if detectedPorts != nil && !reflect.DeepEqual(detectedPorts, []int{}) {
componentPortsMapFromRepo[context] = detectedPorts
}
} else {
log.Info(fmt.Sprintf("failed to detect port from context: %v, error: %v", context, err))
}

Check warning on line 262 in cdq-analysis/pkg/detect.go

View check run for this annotation

Codecov / codecov/patch

cdq-analysis/pkg/detect.go#L260-L262

Added lines #L260 - L262 were not covered by tests
}
return nil
}
Expand Down
29 changes: 17 additions & 12 deletions cdq-analysis/pkg/devfile.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,24 +34,29 @@
)

const (
DevfileName = "devfile.yaml"
HiddenDevfileName = ".devfile.yaml"
HiddenDevfileDir = ".devfile"
DockerfileName = "Dockerfile"
ContainerfileName = "Containerfile"
HiddenDockerDir = ".docker"
DockerDir = "docker"
BuildDir = "build"
DevfileName = "devfile.yaml"
HiddenDevfileName = ".devfile.yaml"
HiddenDevfileDir = ".devfile"
DockerfileName = "Dockerfile"
AlternateDockerfileName = "dockerfile"
ContainerfileName = "Containerfile"
HiddenDockerDir = ".docker"
DockerDir = "docker"
BuildDir = "build"

Devfile = DevfileName // devfile.yaml
HiddenDevfile = HiddenDevfileName // .devfile.yaml
HiddenDirDevfile = HiddenDevfileDir + "/" + DevfileName // .devfile/devfile.yaml
HiddenDirHiddenDevfile = HiddenDevfileDir + "/" + HiddenDevfileName // .devfile/.devfile.yaml

Dockerfile = DockerfileName // Dockerfile
HiddenDirDockerfile = HiddenDockerDir + "/" + DockerfileName // .docker/Dockerfile
DockerDirDockerfile = DockerDir + "/" + DockerfileName // docker/Dockerfile
BuildDirDockerfile = BuildDir + "/" + DockerfileName // build/Dockerfile
Dockerfile = DockerfileName // Dockerfile
HiddenDirDockerfile = HiddenDockerDir + "/" + DockerfileName // .docker/Dockerfile
DockerDirDockerfile = DockerDir + "/" + DockerfileName // docker/Dockerfile
BuildDirDockerfile = BuildDir + "/" + DockerfileName // build/Dockerfile
AlternateDockerfile = AlternateDockerfileName // dockerfile
HiddenDirAlternateDockerfile = HiddenDockerDir + "/" + AlternateDockerfileName // .docker/dockerfile
DockerDirAlternateDockerfile = DockerDir + "/" + AlternateDockerfileName // docker/dockerfile
BuildDirAlternateDockerfile = BuildDir + "/" + AlternateDockerfileName // build/dockerfile

Containerfile = ContainerfileName // Containerfile
HiddenDirContainerfile = HiddenDockerDir + "/" + ContainerfileName // .docker/Containerfile
Expand Down Expand Up @@ -104,8 +109,8 @@
switch merr.Errors[i].(type) {
case *devfileValidation.MissingDefaultCmdWarning:
log.Info(fmt.Sprintf("devfile is missing default command, found a warning: %v", merr.Errors[i]))
default:
newErr = multierror.Append(newErr, merr.Errors[i])

Check warning on line 113 in cdq-analysis/pkg/devfile.go

View check run for this annotation

Codecov / codecov/patch

cdq-analysis/pkg/devfile.go#L112-L113

Added lines #L112 - L113 were not covered by tests
}
}
} else {
Expand All @@ -120,13 +125,13 @@
}
deployCompMap, err := parser.GetDeployComponents(devfileData)
if err != nil {
log.Error(err, fmt.Sprintf("failed to get deploy components from %s", URL))
return shouldIgnoreDevfile, nil, fmt.Errorf(fmt.Sprintf("err: %v, failed to get deploy components from %s", err, URL))
}

Check warning on line 130 in cdq-analysis/pkg/devfile.go

View check run for this annotation

Codecov / codecov/patch

cdq-analysis/pkg/devfile.go#L128-L130

Added lines #L128 - L130 were not covered by tests
devfileBytes, err = yaml.Marshal(devfileData)
if err != nil {
return shouldIgnoreDevfile, nil, err
}

Check warning on line 134 in cdq-analysis/pkg/devfile.go

View check run for this annotation

Codecov / codecov/patch

cdq-analysis/pkg/devfile.go#L133-L134

Added lines #L133 - L134 were not covered by tests
kubeCompFilter := common.DevfileOptions{
ComponentOptions: common.ComponentOptions{
ComponentType: v1alpha2.KubernetesComponentType,
Expand All @@ -134,10 +139,10 @@
}
kubeComp, err := devfileData.GetComponents(kubeCompFilter)
if err != nil {
log.Error(err, fmt.Sprintf("failed to get kubernetes component from %s", URL))
shouldIgnoreDevfile = true
return shouldIgnoreDevfile, nil, nil
}

Check warning on line 145 in cdq-analysis/pkg/devfile.go

View check run for this annotation

Codecov / codecov/patch

cdq-analysis/pkg/devfile.go#L142-L145

Added lines #L142 - L145 were not covered by tests
if len(kubeComp) == 0 {
log.Info(fmt.Sprintf("Found 0 kubernetes components being defined in devfile from %s, it is not a valid outerloop definition, the devfile will be ignored. A devfile will be matched from registry...", URL))
shouldIgnoreDevfile = true
Expand All @@ -147,8 +152,8 @@
found := false
for _, component := range kubeComp {
if _, ok := deployCompMap[component.Name]; ok {
found = true
break

Check warning on line 156 in cdq-analysis/pkg/devfile.go

View check run for this annotation

Codecov / codecov/patch

cdq-analysis/pkg/devfile.go#L155-L156

Added lines #L155 - L156 were not covered by tests
}
}
if !found {
Expand All @@ -166,9 +171,9 @@
}
imageComp, err := devfileData.GetComponents(imageCompFilter)
if err != nil {
log.Error(err, fmt.Sprintf("failed to get image component from %s", URL))
return shouldIgnoreDevfile, nil, fmt.Errorf(fmt.Sprintf("err: %v, failed to get image component from %s", err, URL))
}

Check warning on line 176 in cdq-analysis/pkg/devfile.go

View check run for this annotation

Codecov / codecov/patch

cdq-analysis/pkg/devfile.go#L174-L176

Added lines #L174 - L176 were not covered by tests
if len(imageComp) == 0 {
log.Info(fmt.Sprintf("Found 0 image components being defined in devfile from %s, it is not a valid outerloop definition, the devfile will be ignored. A devfile will be matched from registry...", URL))
shouldIgnoreDevfile = true
Expand All @@ -181,21 +186,21 @@
dockerfileURI := component.Image.Dockerfile.DockerfileSrc.Uri
absoluteURI := strings.HasPrefix(dockerfileURI, "http://") || strings.HasPrefix(dockerfileURI, "https://")
if absoluteURI {
// image uri
_, err = CurlEndpoint(dockerfileURI)

Check warning on line 190 in cdq-analysis/pkg/devfile.go

View check run for this annotation

Codecov / codecov/patch

cdq-analysis/pkg/devfile.go#L189-L190

Added lines #L189 - L190 were not covered by tests
} else {
if devfileSrc.Path != "" {
// local devfile src with relative Dockerfile uri
dockerfileURI = path.Join(path.Dir(URL), dockerfileURI)
err = parserUtil.ValidateFile(dockerfileURI)

Check warning on line 195 in cdq-analysis/pkg/devfile.go

View check run for this annotation

Codecov / codecov/patch

cdq-analysis/pkg/devfile.go#L193-L195

Added lines #L193 - L195 were not covered by tests
} else {
// remote devfile src with relative Dockerfile uri
var u *url.URL
u, err = url.Parse(URL)
if err != nil {
log.Error(err, fmt.Sprintf("failed to parse URL from %s", URL))
return shouldIgnoreDevfile, nil, fmt.Errorf(fmt.Sprintf("failed to parse URL from %s", URL))
}

Check warning on line 203 in cdq-analysis/pkg/devfile.go

View check run for this annotation

Codecov / codecov/patch

cdq-analysis/pkg/devfile.go#L201-L203

Added lines #L201 - L203 were not covered by tests
u.Path = path.Join(u.Path, dockerfileURI)
dockerfileURI = u.String()
_, err = CurlEndpoint(dockerfileURI)
Expand All @@ -206,16 +211,16 @@
return shouldIgnoreDevfile, nil, fmt.Errorf(fmt.Sprintf("failed to get Dockerfile from the URI %s, invalid image component: %s", URL, component.Name))
}
}
if _, ok := deployCompMap[component.Name]; ok {
found = true
break

Check warning on line 216 in cdq-analysis/pkg/devfile.go

View check run for this annotation

Codecov / codecov/patch

cdq-analysis/pkg/devfile.go#L214-L216

Added lines #L214 - L216 were not covered by tests
}
}
if !found {
err = fmt.Errorf("found more than one image components, but no deploy command associated with any being defined in the devfile from %s", URL)
log.Error(err, "failed to validate devfile")
return shouldIgnoreDevfile, nil, err
}

Check warning on line 223 in cdq-analysis/pkg/devfile.go

View check run for this annotation

Codecov / codecov/patch

cdq-analysis/pkg/devfile.go#L219-L223

Added lines #L219 - L223 were not covered by tests
}
// TODO: if only one image component, should return a warning that no apply command being defined
}
Expand Down Expand Up @@ -248,8 +253,8 @@
} else if src.Path != "" {
parserArgs.Path = src.Path
} else {
return nil, fmt.Errorf("cannot parse devfile without a src")
}

Check warning on line 257 in cdq-analysis/pkg/devfile.go

View check run for this annotation

Codecov / codecov/patch

cdq-analysis/pkg/devfile.go#L256-L257

Added lines #L256 - L257 were not covered by tests
devfileObj, _, err := devfilePkg.ParseDevfileAndValidate(parserArgs)
return devfileObj.Data, err
}
2 changes: 1 addition & 1 deletion controllers/componentdetectionquery_controller_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1443,7 +1443,7 @@ var _ = Describe("Component Detection Query controller", func() {
Expect(component.ComponentStub.Source.GitSource.Revision).Should(Equal("main"))
}

// Make sure the devfiles are detected
// Make sure the components are detected
Expect(len(createdHasCompDetectionQuery.Status.ComponentDetected)).Should(Equal(3)) // mocked, not accurate. check unit test for accurate detection that uses the alizer client instead of the mock client.
for _, componentDesc := range createdHasCompDetectionQuery.Status.ComponentDetected {
Expect(componentDesc.ComponentStub.Source.GitSource).ShouldNot(BeNil())
Expand Down
1 change: 1 addition & 0 deletions pkg/devfile/devfile.go
Original file line number Diff line number Diff line change
Expand Up @@ -95,8 +95,8 @@

ingressEndpoint, err := GetIngressFromEndpoint(endpoint.Name, compName, fmt.Sprintf("%d", endpoint.TargetPort), endpoint.Path, isSecure, endpoint.Annotations, hostname)
if err != nil {
return parser.KubernetesResources{}, err
}

Check warning on line 99 in pkg/devfile/devfile.go

View check run for this annotation

Codecov / codecov/patch

pkg/devfile/devfile.go#L98-L99

Added lines #L98 - L99 were not covered by tests
endpointIngresses = append(endpointIngresses, ingressEndpoint)

endpointRoutes = append(endpointRoutes, GetRouteFromEndpoint(endpoint.Name, compName, fmt.Sprintf("%d", endpoint.TargetPort), endpoint.Path, isSecure, endpoint.Annotations))
Expand Down Expand Up @@ -441,8 +441,8 @@

portNumber, err := strconv.ParseInt(port, 10, 32)
if err != nil {
return networkingv1.Ingress{}, err
}

Check warning on line 445 in pkg/devfile/devfile.go

View check run for this annotation

Codecov / codecov/patch

pkg/devfile/devfile.go#L444-L445

Added lines #L444 - L445 were not covered by tests

ingress := networkingv1.Ingress{
TypeMeta: generator.GetTypeMeta("Ingress", "networking.k8s.io/v1"),
Expand Down Expand Up @@ -744,6 +744,7 @@
var err error
// Containerfile is an alternate name for Dockerfile
validDockerfileLocations := []string{cdqanalysis.Dockerfile, cdqanalysis.DockerDirDockerfile, cdqanalysis.HiddenDirDockerfile, cdqanalysis.BuildDirDockerfile,
cdqanalysis.AlternateDockerfile, cdqanalysis.DockerDirAlternateDockerfile, cdqanalysis.HiddenDirAlternateDockerfile, cdqanalysis.BuildDirAlternateDockerfile,
cdqanalysis.Containerfile, cdqanalysis.DockerDirContainerfile, cdqanalysis.HiddenDirContainerfile, cdqanalysis.BuildDirContainerfile}

for _, path := range validDockerfileLocations {
Expand Down
5 changes: 5 additions & 0 deletions pkg/devfile/devfile_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -288,6 +288,11 @@ func TestFindAndDownloadDockerfile(t *testing.T) {
url: "https://raw.githubusercontent.com/yangcao77/dockerfile-priority/main/case8",
wantDockerfileContext: "build/Containerfile",
},
{
name: "Curl dockerfile",
url: "https://raw.githubusercontent.com/yangcao77/dockerfile-priority/main/case9",
wantDockerfileContext: "dockerfile",
},
{
name: "Cannot curl for a Dockerfile or a Containerfile",
url: "https://github.com/octocat/Hello-World",
Expand Down
Loading