Enhancement: Using Discovery API to discover Kubernetes Resources #284

KiptoonKipkurui · 2023-12-13T13:52:25Z

Description
To discover all resources in a cluster, this PR uses the Discovery API to list all resources and avoid hardcoding the resources to track and also supports using both whitelist and blacklisted resources in watch-list ConfigMap
This PR fixes #274

Notes for Reviewers

Signed commits

Yes, I signed my commits.

Signed-off-by: Daniel Kiptoon <[email protected]>

…rui/default_resources Signed-off-by: Daniel Kiptoon <[email protected]>

Signed-off-by: Daniel Kiptoon <[email protected]>

codecov · 2023-12-13T14:11:14Z

Codecov Report

Attention: Patch coverage is 80.32787% with 12 lines in your changes are missing coverage. Please review.

Project coverage is 11.95%. Comparing base (a8e2fb7) to head (927e036).
Report is 29 commits behind head on master.

Files	Patch %	Lines
internal/config/default_config.go	77.55%	7 Missing and 4 partials ⚠️
internal/config/crd_config.go	91.66%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #284      +/-   ##
==========================================
+ Coverage    9.37%   11.95%   +2.58%     
==========================================
  Files          10       11       +1     
  Lines         651      669      +18     
==========================================
+ Hits           61       80      +19     
+ Misses        583      582       -1     
  Partials        7        7

Flag	Coverage Δ
unittests	`11.95% <80.32%> (+2.58%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

leecalcote · 2023-12-13T17:35:02Z

This might be an ideal time for a set of standard tests to be run. What sort of CI coverage do we have currently? Only compilation of golang, but no functional or integration tests, right?

Time to invest in one, I suggest. We already have a number of them that can be reused from meshery/meshery....

Signed-off-by: Daniel Kiptoon <[email protected]>

KiptoonKipkurui · 2023-12-13T20:06:47Z

This might be an ideal time for a set of standard tests to be run. What sort of CI coverage do we have currently? Only compilation of golang, but no functional or integration tests, right?

Time to invest in one, I suggest. We already have a number of them that can be reused from meshery/meshery....

There are many of them, I shall have fun reusing some of them here

leecalcote · 2023-12-15T17:20:44Z

@VihasMakwana, your 👀s on this would be great.

internal/config/crd_config.go

VihasMakwana

LGTM overall. a couple of nits.
Great step towards the discovery!

stale · 2024-02-13T04:32:54Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

leecalcote · 2024-02-13T11:42:34Z

merge conflicts, @KiptoonKipkurui

Signed-off-by: Daniel Kiptoon <[email protected]>

KiptoonKipkurui · 2024-02-14T08:10:08Z

@leecalcote @VihasMakwana @MUzairS15 I have resolved all conflicts and reviews

MUzairS15 · 2024-03-15T06:43:21Z

internal/config/default_config.go

@@ -134,5 +40,66 @@ var (
 		},
 	}

-	DefaultEvents = []string{"ADD", "UPDATE", "DELETE"}
+	DefaultEvents     = []string{"ADD", "UPDATE", "DELETE"}
+	ExemptedResources = []string{"selfsubjectrulesreviews", "localsubjectaccessreviews", "bindings", "selfsubjectaccessreviews", "subjectaccessreviews", "tokenreviews", "componentstatuses", "flowschemas", "prioritylevelconfigurations"}


A perfect example of the use of deny selector in the CR config.

MUzairS15 · 2024-03-15T06:47:57Z

internal/config/default_config.go

+
+	for _, group := range groupList {
+		for _, resource := range group.APIResources {
+			if strings.Contains(resource.Name, "/") {


What is this for?
Does this mean resources under apps/v1 , networking.k8s.io/v1 ... not to be watched for changes inside the cluster?

MUzairS15 · 2024-03-15T06:49:41Z

internal/config/default_config_test.go

+		t.Error("local resources not discovered, expected 1")
+	}
+
+	idx := slices.IndexFunc(Pipelines["global"], func(c PipelineConfig) bool { return c.Name == "namespaces.v1." })


Is there any specific reason for treating namespace special? Is it just because all other namespaced resources are in the local config and non-namespaced resources are in the global config?

I see it's the test file, but even then adding a comment will help.

stale · 2024-05-01T10:50:52Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale · 2024-05-12T17:51:02Z

This issue is being automatically closed due to inactivity. However, you may choose to reopen this issue.

KiptoonKipkurui added 6 commits December 13, 2023 16:46

support whitelisting and blacklisting resources

3590e17

Signed-off-by: Daniel Kiptoon <[email protected]>

augment resources with those from crd

60843f4

Signed-off-by: Daniel Kiptoon <[email protected]>

refactor unit tests

282ead1

Signed-off-by: Daniel Kiptoon <[email protected]>

add functionality to autodiscover resources

4dabf9b

Signed-off-by: Daniel Kiptoon <[email protected]>

Merge remote-tracking branch 'upstream/master' into feat/kiptoonkipku…

b3f6a7e

…rui/default_resources Signed-off-by: Daniel Kiptoon <[email protected]>

correct ineffectual assignment

da2f9cb

Signed-off-by: Daniel Kiptoon <[email protected]>

KiptoonKipkurui changed the title ~~Feat/kiptoonkipkurui/default resources~~ Enhancement: Using Discovery API to determine resources Dec 13, 2023

KiptoonKipkurui changed the title ~~Enhancement: Using Discovery API to determine resources~~ Enhancement: Using Discovery API to discover Kubernetes Resources Dec 13, 2023

KiptoonKipkurui requested review from MUzairS15 and leecalcote December 13, 2023 14:08

leecalcote requested a review from Aisuko December 13, 2023 17:31

leecalcote added component/meshsync MeshSync related kind/feature New major feature or request labels Dec 13, 2023

KiptoonKipkurui added 2 commits December 13, 2023 22:17

add unit tests for getting resources from kubernetes Discovery

b7b50d7

Signed-off-by: Daniel Kiptoon <[email protected]>

pass the discovery client

49c524b

Signed-off-by: Daniel Kiptoon <[email protected]>

leecalcote requested a review from VihasMakwana December 15, 2023 17:20

VihasMakwana reviewed Dec 27, 2023

View reviewed changes

internal/config/crd_config.go Show resolved Hide resolved

VihasMakwana suggested changes Dec 27, 2023

View reviewed changes

stale bot added the issue/stale Issue has not had any activity for an extended period of time label Feb 13, 2024

stale bot removed the issue/stale Issue has not had any activity for an extended period of time label Feb 13, 2024

KiptoonKipkurui added 3 commits February 13, 2024 15:16

Merge branch 'master' into feat/kiptoonkipkurui/default_resources

160d99e

Signed-off-by: Daniel Kiptoon <[email protected]>

fix import issues

a0e590f

Signed-off-by: Daniel Kiptoon <[email protected]>

simplify steps

927e036

Signed-off-by: Daniel Kiptoon <[email protected]>

MUzairS15 reviewed Mar 15, 2024

View reviewed changes

stale bot added the issue/stale Issue has not had any activity for an extended period of time label May 1, 2024

stale bot closed this May 12, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enhancement: Using Discovery API to discover Kubernetes Resources #284

Enhancement: Using Discovery API to discover Kubernetes Resources #284

KiptoonKipkurui commented Dec 13, 2023

codecov bot commented Dec 13, 2023 •

edited

Loading

leecalcote commented Dec 13, 2023

KiptoonKipkurui commented Dec 13, 2023

leecalcote commented Dec 15, 2023

VihasMakwana left a comment

stale bot commented Feb 13, 2024

leecalcote commented Feb 13, 2024

KiptoonKipkurui commented Feb 14, 2024

MUzairS15 Mar 15, 2024

MUzairS15 Mar 15, 2024

MUzairS15 Mar 15, 2024

MUzairS15 Mar 15, 2024

stale bot commented May 1, 2024

stale bot commented May 12, 2024

Enhancement: Using Discovery API to discover Kubernetes Resources #284

Enhancement: Using Discovery API to discover Kubernetes Resources #284

Conversation

KiptoonKipkurui commented Dec 13, 2023

codecov bot commented Dec 13, 2023 • edited Loading

Codecov Report

leecalcote commented Dec 13, 2023

KiptoonKipkurui commented Dec 13, 2023

leecalcote commented Dec 15, 2023

VihasMakwana left a comment

Choose a reason for hiding this comment

stale bot commented Feb 13, 2024

leecalcote commented Feb 13, 2024

KiptoonKipkurui commented Feb 14, 2024

MUzairS15 Mar 15, 2024

Choose a reason for hiding this comment

MUzairS15 Mar 15, 2024

Choose a reason for hiding this comment

MUzairS15 Mar 15, 2024

Choose a reason for hiding this comment

MUzairS15 Mar 15, 2024

Choose a reason for hiding this comment

stale bot commented May 1, 2024

stale bot commented May 12, 2024

codecov bot commented Dec 13, 2023 •

edited

Loading