Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] MeshSync E2E test #286

Closed

Conversation

KiptoonKipkurui
Copy link
Member

Description
This PR introduces a new workflow e2etests.yaml that adds end to end test for MeshSync. By design, MeshSync discovers resources within a Kubernetes cluster and watches them for events [ADDED, MODIFIED and DELETED] events, these events are published via a NATS to a topic (meshery.meshsync.core). The strategy therefore for this test, is to run MeshSync against a Kubernetes cluster, and subscribe to the topic that it publishes its discovered Kubernetes object events' at, then within a 3 minutes(Subject to discussion) it should have discovered some resources otherwise marked as a failure.
This PR fixes #285

Notes for Reviewers

Signed commits

  • Yes, I signed my commits.

Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
@KiptoonKipkurui KiptoonKipkurui added area/ci Continuous integration | Build and release area/devops labels Dec 17, 2023
@KiptoonKipkurui KiptoonKipkurui self-assigned this Dec 17, 2023
@KiptoonKipkurui KiptoonKipkurui added the component/meshsync MeshSync related label Dec 17, 2023
@KiptoonKipkurui
Copy link
Member Author

Once on track, we can discuss the trigger event, currently just at on [push]

Signed-off-by: Daniel Kiptoon <[email protected]>
Copy link

codecov bot commented Dec 17, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 9.37%. Comparing base (a653c60) to head (5e45525).
Report is 22 commits behind head on master.

Additional details and impacted files
@@          Coverage Diff           @@
##           master    #286   +/-   ##
======================================
  Coverage    9.37%   9.37%           
======================================
  Files          10      10           
  Lines         651     651           
======================================
  Hits           61      61           
  Misses        583     583           
  Partials        7       7           
Flag Coverage Δ
unittests 9.37% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
Signed-off-by: Daniel Kiptoon <[email protected]>
…ach time

we add a resource to be tracked

Signed-off-by: Daniel Kiptoon <[email protected]>
@KiptoonKipkurui
Copy link
Member Author

@MUzairS15 @MarioArriaga92 I have replaced all sleep statements with appropriate kubectl commands and so far here's the final product.

Some things to note, with multiple Kubernetes versions to test for in the matrix, random Kubernetes versions may fail at different times in different tests for example

My observation over several runs is that the tests largely fail at the following stages

  1. Failure to get the token even though other steps correctly identify it hence mesheryctl system check command fails
  2. Random failure to install Minikube
  3. Failing to discover the resource (from the logs, one can see that Meshery fails to connect to the Broker)

You may have good pointers here, so let me know if there's anything I can do to make this test consistent

Copy link

stale bot commented Apr 14, 2024

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the issue/stale Issue has not had any activity for an extended period of time label Apr 14, 2024
Copy link

stale bot commented Apr 28, 2024

This issue is being automatically closed due to inactivity. However, you may choose to reopen this issue.

@stale stale bot closed this Apr 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/ci Continuous integration | Build and release area/devops component/meshsync MeshSync related issue/stale Issue has not had any activity for an extended period of time
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CI] E2E tests MeshSync
2 participants