OCPBUGS-67211: exculude Flaky tests for openshift/csi test suite #30597

Phaow · 2025-12-11T08:21:05Z

We expect the [Disabled:" [Flaky] label tests should be excluded for openshift/csi test suite by default and keep the [Slow] ones so we do not use func withExcludedTestsFilter(baseExpr string).

Unexpected CI failures -> https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_csi-operator/475/pull-ci-openshift-csi-operator-main-e2e-aws-csi/1998064182309687296

# We expect the [Flaky] label tests should be excluded for openshift/csi test suite by default
$ ./openshift-tests run openshift/csi --dry-run --retry-strategy=aggressive --provider '{"type":"aws","region":"us-east-2","zone":"us-east-2b","multizone":true,"multimaster":true}'|grep 'VolumeAttributesClass'|grep 'Flaky'
"External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (block volmode)] volume-modify [FeatureGate:VolumeAttributesClass] [Feature:VolumeAttributesClass] should recover from invalid target VAC by updating PVC to new valid VAC [Flaky]"
"External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] volume-modify [FeatureGate:VolumeAttributesClass] [Feature:VolumeAttributesClass] should recover from invalid target VAC by updating PVC to new valid VAC [Flaky]"
"External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (ntfs)] [Feature:Windows] volume-modify [FeatureGate:VolumeAttributesClass] [Feature:VolumeAttributesClass] should recover from invalid target VAC by updating PVC to new valid VAC [Flaky]"

# Before add the exclude filter
$ ./openshift-tests run openshift/csi --dry-run --retry-strategy=aggressive --provider '{"type":"aws","region":"us-east-2","zone":"us-east-2b","multizone":true,"multimaster":true}'|grep 'External Storage'| wc -l
...
INFO[0009] Filter chain completed with 289 tests         component=test-filter final_count=289
     289
     
# After add the exclude filter
$ ./openshift-tests run openshift/csi --dry-run --retry-strategy=aggressive --provider '{"type":"aws","region":"us-east-2","zone":"us-east-2b","multizone":true,"multimaster":true}'|grep 'External Storage'| wc -l
...
INFO[0009] Filter chain completed with 286 tests         component=test-filter final_count=286
     286
$ ./openshift-tests run openshift/csi --dry-run --retry-strategy=aggressive --provider '{"type":"aws","region":"us-east-2","zone":"us-east-2b","multizone":true,"multimaster":true}'|grep 'External Storage'| wc -l
...
INFO[0009] Filter chain completed with 286 tests         component=test-filter final_count=286
     286
$ ./openshift-tests run openshift/csi --dry-run --retry-strategy=aggressive --provider '{"type":"aws","region":"us-east-2","zone":"us-east-2b","multizone":true,"multimaster":true}'|grep 'External Storage'| grep 'Flaky' | wc -l
...
INFO[0009] Filter chain completed with 286 tests         component=test-filter final_count=286
       0

Signed-off-by: Penghao <[email protected]>

openshift-ci-robot · 2025-12-11T08:21:07Z

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

openshift-ci-robot · 2025-12-11T08:21:14Z

@Phaow: This pull request references Jira Issue OCPBUGS-67211, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.21.0) matches configured target version for branch (4.21.0)
bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

No GitHub users were found matching the public email listed for the QA contact in Jira ([email protected]), skipping review request.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

We expect the [Disabled:" [Flaky] label tests should be excluded for openshift/csi test suite by default and keep the [Slow] ones so we do not use func withExcludedTestsFilter(baseExpr string).

Unexpected CI failures -> https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_csi-operator/475/pull-ci-openshift-csi-operator-main-e2e-aws-csi/1998064182309687296

# We expect the [Flaky] label tests should be excluded for openshift/csi test suite by default
$ ./openshift-tests run openshift/csi --dry-run --retry-strategy=aggressive --provider '{"type":"aws","region":"us-east-2","zone":"us-east-2b","multizone":true,"multimaster":true}'|grep 'VolumeAttributesClass'|grep 'Flaky'
"External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (block volmode)] volume-modify [FeatureGate:VolumeAttributesClass] [Feature:VolumeAttributesClass] should recover from invalid target VAC by updating PVC to new valid VAC [Flaky]"
"External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] volume-modify [FeatureGate:VolumeAttributesClass] [Feature:VolumeAttributesClass] should recover from invalid target VAC by updating PVC to new valid VAC [Flaky]"
"External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (ntfs)] [Feature:Windows] volume-modify [FeatureGate:VolumeAttributesClass] [Feature:VolumeAttributesClass] should recover from invalid target VAC by updating PVC to new valid VAC [Flaky]"

# Before add the exclude filter
$ ./openshift-tests run openshift/csi --dry-run --retry-strategy=aggressive --provider '{"type":"aws","region":"us-east-2","zone":"us-east-2b","multizone":true,"multimaster":true}'|grep 'External Storage'| wc -l
...
INFO[0009] Filter chain completed with 289 tests         component=test-filter final_count=289
    289
    
# After add the exclude filter
$ ./openshift-tests run openshift/csi --dry-run --retry-strategy=aggressive --provider '{"type":"aws","region":"us-east-2","zone":"us-east-2b","multizone":true,"multimaster":true}'|grep 'External Storage'| wc -l
...
INFO[0009] Filter chain completed with 286 tests         component=test-filter final_count=286
    286
$ ./openshift-tests run openshift/csi --dry-run --retry-strategy=aggressive --provider '{"type":"aws","region":"us-east-2","zone":"us-east-2b","multizone":true,"multimaster":true}'|grep 'External Storage'| wc -l
...
INFO[0009] Filter chain completed with 286 tests         component=test-filter final_count=286
    286
$ ./openshift-tests run openshift/csi --dry-run --retry-strategy=aggressive --provider '{"type":"aws","region":"us-east-2","zone":"us-east-2b","multizone":true,"multimaster":true}'|grep 'External Storage'| grep 'Flaky' | wc -l
...
INFO[0009] Filter chain completed with 286 tests         component=test-filter final_count=286
      0

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Phaow · 2025-12-11T08:23:03Z

Hi @jsafrane @stbenjam , could you please help take a look when you get a chance? Thank you! ^^

Phaow · 2025-12-11T08:23:09Z

/assign @jsafrane @stbenjam

Phaow · 2025-12-11T08:26:47Z

/cc @dfajmon @radeore @duanwei33

Phaow · 2025-12-11T08:38:47Z

/payload-job-with-prs periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-csi openshift/csi-operator#475

openshift-ci · 2025-12-11T08:38:50Z

@Phaow: it appears that you have attempted to use some version of the payload command, but your comment was incorrectly formatted and cannot be acted upon. See the docs for usage info.

openshift-ci-robot · 2025-12-11T08:48:09Z

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

jsafrane · 2025-12-11T09:03:54Z

/lgtm

Phaow · 2025-12-11T13:11:46Z

/retest-required

Phaow · 2025-12-12T00:09:48Z

/retest-required

Phaow · 2025-12-12T00:22:08Z

Hi @bertinatto , could you help review and approve the small fix when you get a chance? Thank you!

bertinatto

/lgtm

openshift-ci · 2025-12-15T11:50:43Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bertinatto, jsafrane, Phaow

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [bertinatto]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Phaow · 2025-12-15T11:56:53Z

/verified by @Phaow

openshift-ci-robot · 2025-12-15T11:57:06Z

@Phaow: This PR has been marked as verified by @Phaow.

Details

In response to this:

/verified by @Phaow

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Phaow · 2025-12-15T11:57:07Z

/label acknowledge-critical-fixes-only

openshift-ci-robot · 2025-12-15T12:44:07Z

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

openshift-ci-robot · 2025-12-15T16:19:53Z

/retest-required

Remaining retests: 0 against base HEAD 18c1394 and 2 for PR HEAD 6c3059e in total

openshift-ci-robot · 2025-12-15T19:19:56Z

/retest-required

Remaining retests: 0 against base HEAD 39f4424 and 1 for PR HEAD 6c3059e in total

Phaow · 2025-12-15T22:30:19Z

/retest-required

openshift-ci · 2025-12-16T01:07:59Z

@Phaow: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci-robot · 2025-12-16T01:10:42Z

@Phaow: Jira Issue Verification Checks: Jira Issue OCPBUGS-67211
✔️ This pull request was pre-merge verified.
✔️ All associated pull requests have merged.
✔️ All associated, merged pull requests were pre-merge verified.

Jira Issue OCPBUGS-67211 has been moved to the MODIFIED state and will move to the VERIFIED state when the change is available in an accepted nightly payload. 🕓

Details

In response to this:

We expect the [Disabled:" [Flaky] label tests should be excluded for openshift/csi test suite by default and keep the [Slow] ones so we do not use func withExcludedTestsFilter(baseExpr string).

Unexpected CI failures -> https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_csi-operator/475/pull-ci-openshift-csi-operator-main-e2e-aws-csi/1998064182309687296

# We expect the [Flaky] label tests should be excluded for openshift/csi test suite by default
$ ./openshift-tests run openshift/csi --dry-run --retry-strategy=aggressive --provider '{"type":"aws","region":"us-east-2","zone":"us-east-2b","multizone":true,"multimaster":true}'|grep 'VolumeAttributesClass'|grep 'Flaky'
"External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (block volmode)] volume-modify [FeatureGate:VolumeAttributesClass] [Feature:VolumeAttributesClass] should recover from invalid target VAC by updating PVC to new valid VAC [Flaky]"
"External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] volume-modify [FeatureGate:VolumeAttributesClass] [Feature:VolumeAttributesClass] should recover from invalid target VAC by updating PVC to new valid VAC [Flaky]"
"External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (ntfs)] [Feature:Windows] volume-modify [FeatureGate:VolumeAttributesClass] [Feature:VolumeAttributesClass] should recover from invalid target VAC by updating PVC to new valid VAC [Flaky]"

# Before add the exclude filter
$ ./openshift-tests run openshift/csi --dry-run --retry-strategy=aggressive --provider '{"type":"aws","region":"us-east-2","zone":"us-east-2b","multizone":true,"multimaster":true}'|grep 'External Storage'| wc -l
...
INFO[0009] Filter chain completed with 289 tests         component=test-filter final_count=289
    289
    
# After add the exclude filter
$ ./openshift-tests run openshift/csi --dry-run --retry-strategy=aggressive --provider '{"type":"aws","region":"us-east-2","zone":"us-east-2b","multizone":true,"multimaster":true}'|grep 'External Storage'| wc -l
...
INFO[0009] Filter chain completed with 286 tests         component=test-filter final_count=286
    286
$ ./openshift-tests run openshift/csi --dry-run --retry-strategy=aggressive --provider '{"type":"aws","region":"us-east-2","zone":"us-east-2b","multizone":true,"multimaster":true}'|grep 'External Storage'| wc -l
...
INFO[0009] Filter chain completed with 286 tests         component=test-filter final_count=286
    286
$ ./openshift-tests run openshift/csi --dry-run --retry-strategy=aggressive --provider '{"type":"aws","region":"us-east-2","zone":"us-east-2b","multizone":true,"multimaster":true}'|grep 'External Storage'| grep 'Flaky' | wc -l
...
INFO[0009] Filter chain completed with 286 tests         component=test-filter final_count=286
      0

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Phaow · 2025-12-16T08:08:12Z

/cherry-pick release-4.21

openshift-cherrypick-robot · 2025-12-16T08:08:59Z

@Phaow: new pull request created: #30613

Details

In response to this:

/cherry-pick release-4.21

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-merge-robot · 2025-12-18T20:55:15Z

Fix included in accepted release 4.22.0-0.nightly-2025-12-17-101159

test: exculude Flaky tests for openshift/csi test suite

6c3059e

Signed-off-by: Penghao <[email protected]>

openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. labels Dec 11, 2025

openshift-ci bot requested review from p0lyn0mial and sjenning December 11, 2025 08:21

openshift-ci bot assigned jsafrane and stbenjam Dec 11, 2025

Phaow mentioned this pull request Dec 11, 2025

STOR-2543: add VAC manifest for tests in AWS EBS openshift/csi-operator#475

Merged

openshift-ci bot requested review from dfajmon, duanwei33 and radeore December 11, 2025 08:26

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Dec 11, 2025

bertinatto reviewed Dec 15, 2025

View reviewed changes

openshift-ci bot assigned bertinatto Dec 15, 2025

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 15, 2025

openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Dec 15, 2025

openshift-ci bot added the acknowledge-critical-fixes-only Indicates if the issuer of the label is OK with the policy. label Dec 15, 2025

openshift-merge-bot bot merged commit 0d25c90 into openshift:main Dec 16, 2025
20 checks passed

openshift-cherrypick-robot mentioned this pull request Dec 16, 2025

[release-4.21] OCPBUGS-69389: exculude Flaky tests for openshift/csi test suite #30613

Merged

OCPBUGS-67211: exculude Flaky tests for openshift/csi test suite #30597

OCPBUGS-67211: exculude Flaky tests for openshift/csi test suite #30597

Conversation

Phaow commented Dec 11, 2025

Uh oh!

openshift-ci-robot commented Dec 11, 2025

Uh oh!

openshift-ci-robot commented Dec 11, 2025

Uh oh!

Phaow commented Dec 11, 2025

Uh oh!

Phaow commented Dec 11, 2025

Uh oh!

Phaow commented Dec 11, 2025

Uh oh!

Phaow commented Dec 11, 2025

Uh oh!

openshift-ci bot commented Dec 11, 2025

Uh oh!

openshift-ci-robot commented Dec 11, 2025

Uh oh!

jsafrane commented Dec 11, 2025

Uh oh!

Phaow commented Dec 11, 2025

Uh oh!

Phaow commented Dec 12, 2025

Uh oh!

Phaow commented Dec 12, 2025

Uh oh!

bertinatto left a comment

Choose a reason for hiding this comment

Uh oh!

openshift-ci bot commented Dec 15, 2025

Uh oh!

Phaow commented Dec 15, 2025

Uh oh!

openshift-ci-robot commented Dec 15, 2025

Uh oh!

Phaow commented Dec 15, 2025

Uh oh!

openshift-ci-robot commented Dec 15, 2025

Uh oh!

openshift-ci-robot commented Dec 15, 2025

Uh oh!

openshift-ci-robot commented Dec 15, 2025

Uh oh!

Phaow commented Dec 15, 2025

Uh oh!

openshift-ci bot commented Dec 16, 2025

Uh oh!

Uh oh!

openshift-ci-robot commented Dec 16, 2025

Uh oh!

Phaow commented Dec 16, 2025

Uh oh!

openshift-cherrypick-robot commented Dec 16, 2025

Uh oh!

openshift-merge-robot commented Dec 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants