Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: remove string checks and add robust checks for fileType #658

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Jougan-0
Copy link
Contributor

@Jougan-0 Jougan-0 commented Jan 29, 2025

Description

This PR fixes issue of fragile introspecing logic for file types.

Tested locally for all the file types.

image
Notes for Reviewers

Signed commits

  • Yes, I signed my commits.

@Jougan-0
Copy link
Contributor Author

Tested for these 4 files
Compose file


services:

  unit:
    image: golang:1.11
    volumes:
      - .:/usr/src/myapp
      - /tmp/go:/go
    working_dir: /usr/src/myapp
    command: bash -c "pwd && go get -d -v && go test --cover -v ./... && go build -v -o go-demo"

  staging-dep:
    extends:
      file: docker-compose.yml
      service: app
    ports:
      - 8080:8080
    depends_on:
      - db

  staging:
    extends:
      service: unit
    environment:
      - HOST_IP=localhost:8080
    network_mode: host
    command: bash -c "cd /usr/src/myapp && go get -d -v && go test --tags integration -v"

  production:
    extends:
      service: unit
    environment:
      - HOST_IP=${HOST_IP}
    command: bash -c "cd /usr/src/myapp && go get -d -v && go test --tags integration -v"

  db:
    extends:
      file: docker-compose.yml
      service: db```
      

@Jougan-0
Copy link
Contributor Author

Meshery Design

id: 00000000-0000-0000-0000-000000000000
components:
- capabilities: []
  component:
    kind: APIGroup
    schema: ""
    version: v1
  configuration:
    name: ""
    serverAddressByClientCIDRs: []
    versions: []
  description: ""
  displayName: api-group-pq
  format: JSON
  id: 873cfc1d-9d9c-4c23-9be1-1fac1b8d093a
  metadata:
    genealogy: ""
    isAnnotation: false
    isNamespaced: false
    published: false
    instanceDetails: null
    additionalproperties:
      source_uri: git://github.com/kubernetes/kubernetes/master/api/openapi-spec/v3
  model:
    category:
      id: 00000000-0000-0000-0000-000000000000
      name: Orchestration & Management
    displayName: Kubernetes
    id: bba54c44-a440-773a-a524-0114ebde5397
    metadata:
      capabilities: null
      isAnnotation: false
      primaryColor: '#326CE5'
      secondaryColor: '#7aa1f0'
      svgColor: ui/public/static/img/meshmodels/kubernetes/color/kubernetes-color.svg
      svgComplete: ""
      svgWhite: ui/public/static/img/meshmodels/kubernetes/white/kubernetes-white.svg
    model:
      version: v1.32.0-alpha.3
    name: kubernetes
    registrant:
      created_at: 2025-01-27T22:22:14.321642004Z
      credential_id: 00000000-0000-0000-0000-000000000000
      deleted_at: 0001-01-01T00:00:00Z
      id: 9e04779c-94b6-6a03-575c-66f4c57541eb
      kind: github
      name: Github
      status: registered
      sub_type: ""
      type: registry
      updated_at: 2025-01-27T22:22:14.321642004Z
      user_id: 00000000-0000-0000-0000-000000000000
    connection_id: 9e04779c-94b6-6a03-575c-66f4c57541eb
    schemaVersion: models.meshery.io/v1beta1
    status: enabled
    subCategory: Scheduling & Orchestration
    version: v1.0.0
    components: null
    relationships: null
    components_count: 0
    relationships_count: 0
  schemaVersion: components.meshery.io/v1beta1
  status: enabled
  styles:
    background-opacity: 1
    body-text: ""
    body-text-color: '#808080'
    body-text-font-size: 12
    body-text-font-weight: "400"
    body-text-horizontal-align: center
    body-text-vertical-align: center
    border-width: 0
    height: 24
    opacity: 1
    padding: 6
    position:
      x: 269
      "y": 386
    primaryColor: '#326CE5'
    secondaryColor: '#7aa1f0'
    shape: circle
    svgColor: ui/public/static/img/meshmodels/kubernetes/color/apigroup-color.svg
    svgWhite: ui/public/static/img/meshmodels/kubernetes/white/apigroup-white.svg
    width: 24
    z-index: 0
  version: v1.0.0
name: Untitled Design
metadata: {}
relationships: []
schemaVersion: designs.meshery.io/v1beta1
version: 0.0.1```

@Jougan-0
Copy link
Contributor Author

K8s manifest

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-deployment
spec:
  replicas: 2
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
        - name: myapp
          image: myapp:latest
          ports:
            - containerPort: 8000
          env:
            - name: DATABASE_URL
              value: "mongodb://mongo-service:27017/mydatabase"

---
apiVersion: v1
kind: Service
metadata:
  name: myapp-service
spec:
  selector:
    app: myapp
  ports:
    - protocol: TCP
      port: 8000
      targetPort: 8000
  type: ClusterIP

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongo-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mongo
  template:
    metadata:
      labels:
        app: mongo
    spec:
      containers:
        - name: mongo
          image: mongo:latest
          ports:
            - containerPort: 27017
          volumeMounts:
            - name: mongo-storage
              mountPath: /data/db
      volumes:
        - name: mongo-storage
          emptyDir: {}

---
apiVersion: v1
kind: Service
metadata:
  name: mongo-service
spec:
  selector:
    app: mongo
  ports:
    - protocol: TCP
      port: 27017
      targetPort: 27017
  type: ClusterIP

@Jougan-0
Copy link
Contributor Author

Helm chart of meshery meshery-v0.7.157.tgz

@Jougan-0 Jougan-0 requested a review from leecalcote January 29, 2025 20:55
@Jougan-0
Copy link
Contributor Author

Used this script locally to check these

package main

import (
	"fmt"
	"io/ioutil"
	"os"
	"path/filepath"

	"github.com/layer5io/meshkit/utils"
)

func main() {
	// List of files to check
	files := []string{
		"design.yaml",
		"dockercompose.yaml",
		"k8smanifest.yaml",
		"meshery-v0.7.157.tgz",
	}

	// Get the current working directory
	dir, err := os.Getwd()
	if err != nil {
		fmt.Printf("Error getting current directory: %v\n", err)
		os.Exit(1)
	}

	// Iterate over each file
	for _, file := range files {
		filePath := filepath.Join(dir, file)

		// Check if the file exists
		if _, err := os.Stat(filePath); os.IsNotExist(err) {
			fmt.Printf("File %s does not exist in the directory.\n", file)
			continue
		}

		// Read file contents
		data, err := ioutil.ReadFile(filePath)
		if err != nil {
			fmt.Printf("Error reading file %s: %v\n", file, err)
			continue
		}

		// Identify the file type
		fileType, err := utils.IdentifyInputType(data)
		if err != nil {
			fmt.Printf("File type for %s could not be identified: %v\n", file, err)
		} else {
			fmt.Printf("Identified file type for %s: %s\n", file, fileType)
		}
	}
}

Here is the result

Identified file type for design.yaml: Meshery Design
Identified file type for dockercompose.yaml: Docker Compose
Identified file type for k8smanifest.yaml: Kubernetes Manifest
Identified file type for meshery-v0.7.157.tgz: Helm Chart```

@Jougan-0
Copy link
Contributor Author

I looked at the failing tests these are due to 3 packages

  1. Nats
  2. Opa
  3. Ors
    They have deprecated some code. I'll take a look once I get time in few days.

The unit tests for codecov they are failing as they have fragile test cases written.
package_test.go:31: got https://charts.bitnami.com/bitnami/oci://registry-1.docker.io/bitnamicharts/consul:11.4.3, want https://charts.bitnami.com/bitnami/consul

Copy link
Member

@leecalcote leecalcote left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoa! Very cool. 😎 👏 @Jougan-0, this is quite timely as this particular area is being actively discussed - #657

// @aabidsofi19

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants