OCPBUGS-41587: E2E: Adjust performance_update suite for hypershift #1157

SargunNarula · 2024-09-04T12:48:47Z

Updated tests to skip MCP resource verification on Hypershift, where MCP concept does not apply.
Profile applying is made consistent
Removed deprecated API client.
Added test suite run in makefile

openshift-ci-robot · 2024-09-04T12:48:50Z

@SargunNarula: This pull request references CNF-13487 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.18.0" version, but no target version was set.

In response to this:

Updated tests to skip MCP resource verification on Hypershift, where MCP concept does not apply.

Profile applying is made consistent

Removed deprecated API client.

Added test suite run in makefile

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2024-09-10T08:22:20Z

@SargunNarula: This pull request references Jira Issue OCPBUGS-41587, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.18.0) matches configured target version for branch (4.18.0)
bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @gsr-shanks

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Updated tests to skip MCP resource verification on Hypershift, where MCP concept does not apply.

Profile applying is made consistent

Removed deprecated API client.

Added test suite run in makefile

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Tal-or · 2024-09-18T13:55:29Z

Makefile

@@ -274,7 +274,7 @@ pao-functests-mixedcpus: $(BINDATA)
 pao-functests-hypershift: $(BINDATA)
 	@echo "Cluster Version"
 	hack/show-cluster-version.sh
-	hack/run-test.sh -t "./test/e2e/performanceprofile/functests/0_config ./test/e2e/performanceprofile/functests/1_performance ./test/e2e/performanceprofile/functests/3_performance_status ./test/e2e/performanceprofile/functests/6_mustgather_testing" -p "-vv --label-filter="!openshift" -r --fail-fast --flake-attempts=2 --timeout=2h --junit-report=report.xml" -m "Running Functional Tests over Hypershift"
+	hack/run-test.sh -t "./test/e2e/performanceprofile/functests/0_config ./test/e2e/performanceprofile/functests/1_performance ./test/e2e/performanceprofile/functests/2_performance_update ./test/e2e/performanceprofile/functests/3_performance_status ./test/e2e/performanceprofile/functests/6_mustgather_testing" -p "-vv --label-filter="!openshift" -r --fail-fast --flake-attempts=2 --timeout=2h --junit-report=report.xml" -m "Running Functional Tests over Hypershift"


We need a separate make target for that see #1168

Revised with the new target.

Tal-or · 2024-09-18T13:55:52Z

test/e2e/performanceprofile/functests/2_performance_update/updating_profile.go

 )

 type checkFunction func(context.Context, *corev1.Node) (string, error)

 var _ = Describe("[rfe_id:28761][performance] Updating parameters in performance profile", func() {
 	var workerRTNodes []corev1.Node
 	var profile, initialProfile *performancev2.PerformanceProfile
-	var performanceMCP string
+	var resourcePool string


let's use poolName instead

Revised, Thanks.

Tal-or · 2024-09-18T13:59:58Z

test/e2e/performanceprofile/functests/2_performance_update/updating_profile.go


-			By("Waiting when mcp finishes updates")
-			mcps.WaitForCondition(performanceMCP, machineconfigv1.MachineConfigPoolUpdated, corev1.ConditionTrue)
+			testlog.Infof("Waiting when %s finishes updates", resourcePool)


Why did we replace By with testlog.Infof?

@SargunNarula can you replace this with By(fmt.Sprintf instead of testlog.Info

The motivation was to keep log statements consistent. Revised with By().

Tal-or · 2024-09-18T14:00:22Z

Good work, left some comments

Tal-or · 2024-09-23T12:00:11Z

please add this test-suite under the pao-functests-update-only-hypershift makefile target and run:
/test e2e-pao-updating-profile-hypershift

openshift-ci · 2024-09-23T12:00:16Z

@Tal-or: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test e2e-aws-operator
/test e2e-aws-ovn
/test e2e-aws-ovn-techpreview
/test e2e-gcp-pao
/test e2e-gcp-pao-updating-profile
/test e2e-gcp-pao-workloadhints
/test e2e-hypershift
/test e2e-hypershift-pao
/test e2e-no-cluster
/test e2e-upgrade
/test images
/test unit
/test verify
/test vet

The following commands are available to trigger optional jobs:

/test e2e-telco5g-cnftests
/test lint
/test okd-scos-images

Use /test all to run the following jobs that were automatically triggered:

pull-ci-openshift-cluster-node-tuning-operator-master-e2e-aws-operator
pull-ci-openshift-cluster-node-tuning-operator-master-e2e-aws-ovn
pull-ci-openshift-cluster-node-tuning-operator-master-e2e-aws-ovn-techpreview
pull-ci-openshift-cluster-node-tuning-operator-master-e2e-gcp-pao
pull-ci-openshift-cluster-node-tuning-operator-master-e2e-gcp-pao-updating-profile
pull-ci-openshift-cluster-node-tuning-operator-master-e2e-gcp-pao-workloadhints
pull-ci-openshift-cluster-node-tuning-operator-master-e2e-hypershift
pull-ci-openshift-cluster-node-tuning-operator-master-e2e-hypershift-pao
pull-ci-openshift-cluster-node-tuning-operator-master-e2e-no-cluster
pull-ci-openshift-cluster-node-tuning-operator-master-e2e-upgrade
pull-ci-openshift-cluster-node-tuning-operator-master-images
pull-ci-openshift-cluster-node-tuning-operator-master-lint
pull-ci-openshift-cluster-node-tuning-operator-master-unit
pull-ci-openshift-cluster-node-tuning-operator-master-verify
pull-ci-openshift-cluster-node-tuning-operator-master-vet

In response to this:

please add this test-suite under the pao-functests-update-only-hypershift makefile target and run:
/test e2e-pao-updating-profile-hypershift

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Tal-or · 2024-09-24T13:32:20Z

/test help

openshift-ci · 2024-09-24T13:32:25Z

@Tal-or: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test e2e-aws-operator
/test e2e-aws-ovn
/test e2e-aws-ovn-techpreview
/test e2e-gcp-pao
/test e2e-gcp-pao-updating-profile
/test e2e-gcp-pao-workloadhints
/test e2e-hypershift
/test e2e-hypershift-pao
/test e2e-no-cluster
/test e2e-pao-updating-profile-hypershift
/test e2e-upgrade
/test images
/test unit
/test verify
/test vet

The following commands are available to trigger optional jobs:

/test e2e-telco5g-cnftests
/test lint
/test okd-scos-images

Use /test all to run the following jobs that were automatically triggered:

pull-ci-openshift-cluster-node-tuning-operator-master-e2e-aws-operator
pull-ci-openshift-cluster-node-tuning-operator-master-e2e-aws-ovn
pull-ci-openshift-cluster-node-tuning-operator-master-e2e-aws-ovn-techpreview
pull-ci-openshift-cluster-node-tuning-operator-master-e2e-gcp-pao
pull-ci-openshift-cluster-node-tuning-operator-master-e2e-gcp-pao-updating-profile
pull-ci-openshift-cluster-node-tuning-operator-master-e2e-gcp-pao-workloadhints
pull-ci-openshift-cluster-node-tuning-operator-master-e2e-hypershift
pull-ci-openshift-cluster-node-tuning-operator-master-e2e-hypershift-pao
pull-ci-openshift-cluster-node-tuning-operator-master-e2e-no-cluster
pull-ci-openshift-cluster-node-tuning-operator-master-e2e-upgrade
pull-ci-openshift-cluster-node-tuning-operator-master-images
pull-ci-openshift-cluster-node-tuning-operator-master-lint
pull-ci-openshift-cluster-node-tuning-operator-master-unit
pull-ci-openshift-cluster-node-tuning-operator-master-verify
pull-ci-openshift-cluster-node-tuning-operator-master-vet

In response to this:

/test help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Tal-or · 2024-09-24T13:32:53Z

/test e2e-pao-updating-profile-hypershift

Tal-or · 2024-10-01T13:27:04Z

/retest
The failure related to:
openshift/release#57238

Tal-or · 2024-10-01T13:28:22Z

/retest

SargunNarula · 2024-10-04T13:18:37Z

/retest

Tal-or · 2024-10-06T09:09:42Z

{"component":"entrypoint","file":"sigs.k8s.io/prow/pkg/entrypoint/run.go:169","func":"sigs.k8s.io/prow/pkg/entrypoint.Options.ExecuteProcess","level":"error","msg":"Process did not finish before 2h0m0s timeout","severity":"error","time":"2024-10-04T16:08:19Z"}

We need to increase the timeout of the this lane

Tal-or · 2024-10-06T09:24:51Z

openshift/release#57500

SargunNarula · 2024-10-07T07:28:52Z

/retest

Tal-or · 2024-10-08T12:23:00Z

/lgtm

Tal-or · 2024-11-06T16:33:08Z

/test e2e-pao-updating-profile-hypershift

SargunNarula · 2024-11-07T09:40:51Z

/test e2e-pao-updating-profile-hypershift

SargunNarula · 2024-11-12T10:57:52Z

/test e2e-pao-updating-profile-hypershift

SargunNarula · 2024-11-13T10:11:01Z

/test e2e-pao-updating-profile-hypershift

SargunNarula · 2024-11-14T15:28:30Z

/retest

SargunNarula · 2024-11-15T04:40:06Z

/test e2e-pao-updating-profile-hypershift

test/e2e/performanceprofile/functests/2_performance_update/updating_profile.go

SargunNarula · 2024-11-17T06:36:13Z

/retest

Tal-or · 2024-11-18T13:06:39Z

/test e2e-pao-updating-profile-hypershift

test/e2e/performanceprofile/functests/2_performance_update/updating_profile.go

SargunNarula · 2024-11-20T11:22:04Z

/test e2e-pao-updating-profile-hypershift

Tal-or · 2024-11-20T14:35:09Z

You're trying to get MCP on Hypershift. MCP CRD are not exist on Hypershift platform

  [FAILED] in [BeforeAll] - /go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/performanceprofile/functests/2_performance_update/updating_profile.go:1161 @ 11/20/24 13:29:42.748
• [FAILED] [20.126 seconds]
[rfe_id:28761][performance] Updating parameters in performance profile ContainerRuntimeConfig [BeforeAll] when is not given should run high-performance runtimes class with runc as container-runtime [tier-2, openshift]
  [BeforeAll] /go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/performanceprofile/functests/2_performance_update/updating_profile.go:1157
  [It] /go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/performanceprofile/functests/2_performance_update/updating_profile.go:1168
  [FAILED] cannot get MCP "clusters/52cc3a83cf2bb1b34dfb-us-east-1a"
  Unexpected error:
      <*meta.NoKindMatchError | 0xc0008187c0>: 
      no matches for kind "MachineConfigPool" in version "machineconfiguration.openshift.io/v1"
      {
          GroupKind: {
              Group: "machineconfiguration.openshift.io",
              Kind: "MachineConfigPool",
          },
          SearchedVersions: ["v1"],
      }
  occurred
  In [BeforeAll] at: /go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/performanceprofile/functests/2_performance_update/updating_profile.go:1161 @ 11/20/24 13:29:42.748

SargunNarula · 2024-11-20T16:53:00Z

/test e2e-pao-updating-profile-hypershift

Tal-or · 2024-11-21T14:17:48Z

This is the failure from the the last run on the ci/prow/e2e-gcp-pao-updating-profile

  [FAILED] in [BeforeEach] - /go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/performanceprofile/functests/2_performance_update/updating_profile.go:98 @ 11/21/24 11:05:12.562
• [FAILED] [0.794 seconds]
[rfe_id:28761][performance] Updating parameters in performance profile [BeforeEach] Updating of nodeSelector parameter and node labels [test_id:27484]Verifies that node is reverted to plain worker when the extra labels are removed [tier-2, openshift]
  [BeforeEach] /go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/performanceprofile/functests/2_performance_update/updating_profile.go:91
  [It] /go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/performanceprofile/functests/2_performance_update/updating_profile.go:473
  [FAILED] Unexpected error:
      <*errors.errorString | 0xc000592690>: 
      failed to find performance profile with specified node selector map[node-role.kubernetes.io/worker-test:]
      {
          s: "failed to find performance profile with specified node selector map[node-role.kubernetes.io/worker-test:]",
      }
  occurred
  In [BeforeEach] at: /go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/performanceprofile/functests/2_performance_update/updating_profile.go:98 @ 11/21/24 11:05:12.562

- Updated tests to skip MCP resource verification on Hypershift, where MCP concept does not apply. - Profile applying is made consistent - Removed deprecated API client. - Added test suite run in makefile Signed-off-by: Sargun Narula <[email protected]> Added label filter on update only hypershift target - The pao-functests-update-only-hypershift lane is designed to run the performance update suite exclusively on the HyperShift platform, requiring tests to be filtered for HyperShift. Signed-off-by: Sargun Narula <[email protected]>

SargunNarula · 2024-11-21T20:18:51Z

/retest

mrniranjan · 2024-11-22T00:49:28Z

/retest

mrniranjan · 2024-11-22T00:49:37Z

/retest-required

SargunNarula · 2024-11-22T05:09:17Z

/test e2e-pao-updating-profile-hypershift

Tal-or · 2024-11-22T06:39:58Z

/lgtm

openshift-ci · 2024-11-22T07:43:30Z

@SargunNarula: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

shajmakh · 2024-11-22T07:59:52Z

this is tests adjustments PR and doesn't risk breaking the product
/label acknowledge-critical-fixes-only

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Sep 4, 2024

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 4, 2024

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 4, 2024

openshift-ci bot requested review from swatisehgal and yanirq September 4, 2024 12:57

SargunNarula force-pushed the hypershift_update_tests branch from ce177c7 to 3aa1cc3 Compare September 4, 2024 13:08

openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 4, 2024

SargunNarula force-pushed the hypershift_update_tests branch from 3aa1cc3 to 4045e84 Compare September 4, 2024 13:22

SargunNarula changed the title ~~WIP CNF-13487: Adjust performance_update suite for hypershift~~ CNF-13487: Adjust performance_update suite for hypershift Sep 10, 2024

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 10, 2024

SargunNarula changed the title ~~CNF-13487: Adjust performance_update suite for hypershift~~ OCPBUGS-41587: E2E: Adjust performance_update suite for hypershift Sep 10, 2024

openshift-ci-robot added the jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. label Sep 10, 2024

openshift-ci bot requested a review from gsr-shanks September 10, 2024 08:22

Tal-or reviewed Sep 18, 2024

View reviewed changes

SargunNarula force-pushed the hypershift_update_tests branch from 4045e84 to a54d484 Compare September 24, 2024 13:08

SargunNarula force-pushed the hypershift_update_tests branch from 5aaba0c to 92fba8e Compare November 14, 2024 11:10

mrniranjan reviewed Nov 15, 2024

View reviewed changes

test/e2e/performanceprofile/functests/2_performance_update/updating_profile.go Outdated Show resolved Hide resolved

SargunNarula force-pushed the hypershift_update_tests branch from b94cf86 to 08c0202 Compare November 15, 2024 14:31

Tal-or reviewed Nov 19, 2024

View reviewed changes

test/e2e/performanceprofile/functests/2_performance_update/updating_profile.go Outdated Show resolved Hide resolved

test/e2e/performanceprofile/functests/2_performance_update/updating_profile.go Outdated Show resolved Hide resolved

SargunNarula force-pushed the hypershift_update_tests branch from 08c0202 to 6da62db Compare November 20, 2024 10:34

SargunNarula force-pushed the hypershift_update_tests branch from f5fd715 to 5c4ad45 Compare November 20, 2024 14:48

SargunNarula force-pushed the hypershift_update_tests branch from 5c4ad45 to 5d5d55f Compare November 21, 2024 09:17

SargunNarula force-pushed the hypershift_update_tests branch from 5d5d55f to 7214d74 Compare November 21, 2024 15:18

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Nov 22, 2024

openshift-ci bot added the acknowledge-critical-fixes-only Indicates if the issuer of the label is OK with the policy. label Nov 22, 2024

OCPBUGS-41587: E2E: Adjust performance_update suite for hypershift #1157

Are you sure you want to change the base?

OCPBUGS-41587: E2E: Adjust performance_update suite for hypershift #1157

Conversation

SargunNarula commented Sep 4, 2024

openshift-ci-robot commented Sep 4, 2024 • edited by openshift-ci bot Loading

openshift-ci-robot commented Sep 10, 2024

Tal-or Sep 18, 2024

Choose a reason for hiding this comment

SargunNarula Sep 24, 2024

Choose a reason for hiding this comment

Tal-or Sep 18, 2024

Choose a reason for hiding this comment

SargunNarula Sep 24, 2024

Choose a reason for hiding this comment

Tal-or Sep 18, 2024

Choose a reason for hiding this comment

mrniranjan Sep 23, 2024

Choose a reason for hiding this comment

SargunNarula Sep 24, 2024

Choose a reason for hiding this comment

Tal-or commented Sep 18, 2024

Tal-or commented Sep 23, 2024

openshift-ci bot commented Sep 23, 2024

Tal-or commented Sep 24, 2024

openshift-ci bot commented Sep 24, 2024

Tal-or commented Sep 24, 2024

Tal-or commented Oct 1, 2024

Tal-or commented Oct 1, 2024

SargunNarula commented Oct 4, 2024

Tal-or commented Oct 6, 2024

Tal-or commented Oct 6, 2024

SargunNarula commented Oct 7, 2024

Tal-or commented Oct 8, 2024

Tal-or commented Nov 6, 2024

SargunNarula commented Nov 7, 2024

SargunNarula commented Nov 12, 2024

SargunNarula commented Nov 13, 2024

SargunNarula commented Nov 14, 2024

SargunNarula commented Nov 15, 2024

SargunNarula commented Nov 17, 2024

Tal-or commented Nov 18, 2024

SargunNarula commented Nov 20, 2024

Tal-or commented Nov 20, 2024

SargunNarula commented Nov 20, 2024

Tal-or commented Nov 21, 2024

SargunNarula commented Nov 21, 2024

mrniranjan commented Nov 22, 2024

mrniranjan commented Nov 22, 2024

SargunNarula commented Nov 22, 2024

Tal-or commented Nov 22, 2024

openshift-ci bot commented Nov 22, 2024

shajmakh commented Nov 22, 2024 • edited Loading

openshift-ci-robot commented Sep 4, 2024 •

edited by openshift-ci bot

Loading

shajmakh commented Nov 22, 2024 •

edited

Loading