Add operator-specific PR-gate regression suites

[IshwarKanse]

Goal

Run a small, high-signal regression set on cluster-logging-operator and loki pull requests so operator regressions are caught before merge, without running the full ~158-test e2e suite on every PR.

Each operator gets its own suite:

• openshift-logging-e2e-tests/cluster-logging-operator
• openshift-logging-e2e-tests/loki-operator

The suites select tests using a dedicated [PRGate] marker plus [CLO] or [LokiOperator]. PR-gate membership is intentionally separate from [Level0], which controls membership in OpenShift's global conformance suites.

What changed

• Tag 19 existing tests as PR-gate candidates.
• Tag tests by operator ownership: 11 CLO, 10 Loki Operator, with 2 shared tests included in both suites.
• Register one non-disruptive suite per operator. The openshift-tests runner honors each test's existing [Serial] marker, so separate parallel and serial suites are unnecessary.
• Document the suites and annotations in README.md.
• No test logic or fixtures changed.

Selection principles

A blocking presubmit test should provide an actionable signal for code changed in that repository. Selection therefore favors tests that:

• exercise a distinct operator reconciliation, validation, RBAC, generated-resource, or representative delivery path;
• run without external cloud/service credentials;
• are not disruptive;
• avoid redundant coverage already provided by another selected test or by operator unit tests; and
• have a reasonable path to becoming stable and bounded enough for blocking CI.

The Critical/High/Medium case classification is considered, but is not the sole gating criterion. Product criticality does not guarantee that an e2e test is fast or reliable enough to block every PR.

cluster-logging-operator suite — 11 tests

Case ID	Coverage
65408	CLF service-account role validation and status transitions
65685	CLO deployment, PrometheusRule, collector and Grafana dashboard resources
84892/84897	NetworkPolicy for LFME and CLF LokiStack output
85402	NetworkPolicy for CLF OTLP output
68961/85651	CLF OTLP delivery to an in-cluster OTEL collector
74398	Collector defaults, scheduling configuration and live reconciliation
71749	Drop-filter generation and delivery
71049	Syslog receiver input validation and delivery
76727	ViaQ data-model/stream information
78380	Application, infrastructure and audit collection into LokiStack (shared)
75841	Audit-policy filtering into LokiStack (shared)

loki-operator suite — 10 tests

Case ID	Coverage
48608	Reconcile ConfigMaps and Deployments after manual deletion
48679/48616	Per-tenant limits/overrides and rollout after configuration change
52779/55393	LokiRuler configuration and Rules API
67565/55388	Fine-grained non-admin access and role bindings
67643	LokiStack adminGroups access
75334	Per-tenant and global label keys
76990	Default LokiStack OTLP configuration
85643/85644	Enable and disable LokiStack NetworkPolicies
78380	CLF-to-LokiStack collection for all log sources (shared)
75841	Audit-policy filtering through the LokiStack data path (shared)

Tests intentionally left to periodic coverage

• Real-cloud outputs and managed authentication requiring AWS/Azure/GCP credentials.
• Splunk, Kafka, Elasticsearch and other external-service matrices already backed by per-output unit tests.
• Upgrade, performance, DAST and disruptive tests.
• Resource-intensive sizing test 48607, which requires approximately 150 GiB/64 CPUs and deploys three LokiStack sizes.
• Timing/load-sensitive tests such as 76114 and 67421.
• Specialized or overlapping cases: 64147, 65407, 81397, 81398, 55415, 70683 and 85645.
• Cases 74397 and 85642, whose source comments record only 64.8% and 67.6% pass rates respectively; they are not suitable blocking signals without stabilization.

Runtime and rollout

This reduction is expected to bring the CLO suite to roughly 45–90 minutes and the Loki Operator suite to roughly 1–2 hours, but these are planning estimates rather than measured CI durations. The Loki suite is mostly serial because its tests reconcile cluster-level LokiStack resources.

The corresponding openshift/release presubmits should initially run with optional: true. Their actual duration and pass rate should be collected in CI/Sippy, and only tests that meet the team's reliability and runtime threshold should be promoted to blocking status. The complete suite remains covered by periodic jobs.

Verification

• make build
• Rebuilt extension discovers exactly 19 [PRGate] tests.
• 11 tests match [PRGate][CLO].
• 10 tests match [PRGate][LokiOperator].
• 2 shared tests match both suites.
• No [PRGate] test is [Disruptive].

The actual presubmit configuration in openshift/release is intentionally a follow-up change after this test selection is reviewed.

[kabirbhartiRH]

IMO, I think we can remove the below cases from [Level0] to simply not let the gating test take a lot of time.
65408|65407|64147|48607|48608|81398|81397|78380|55415|76114|67421|70683|85643|85645
Besides wondering whether just running a 'Critical' case against every supported log store should be enough for PR gating while the rest of the cases can be covered under the periodic CI jobs?
Open to suggestions :)

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign ishwarkanse for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[IshwarKanse]

Thanks — agreed that 31 tests is too large for a blocking presubmit, and I have reduced the candidate set substantially.

I don't think one Critical case per supported log store should be the sole selection rule. The Critical label represents product priority, while a blocking gate also needs high stability, bounded runtime, actionable failures, and coverage of core operator reconciliation and validation paths. The complete output-store configuration matrix is already better covered by CLO's per-output unit tests and the periodic e2e jobs.

I removed 64147, 65407, 48607, 81398, 81397, 55415, 76114, 67421, 70683, and 85645 from the gate. I retained:

• 65408 as the representative CLF RBAC/role-validation path;
• 48608 for core loki-operator reconcile-after-deletion behavior;
• 85643/85644 for NetworkPolicy desired-state reconciliation; and
• 78380 as the narrower CLF-to-LokiStack integration path.

I also removed 74397 and 85642, which were not in the suggested list, because their source comments record only 64.8% and 67.6% pass rates. They are not suitable blocking signals until stabilized.

The PR now uses a dedicated [PRGate] marker rather than overloading [Level0], since [Level0] also enrolls tests in OpenShift's global conformance suites. The result is 19 candidates: 11 CLO tests, 10 Loki Operator tests, and 2 shared tests that run in both suites.

When the presubmit jobs are added in openshift/release, I recommend starting them as optional and using actual CI/Sippy pass-rate and duration data before promoting the proven-stable subset to blocking.

[openshift-ci]

@IshwarKanse: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/build	83beb5f	link	true	/test build

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[IshwarKanse]

The failing ci/prow/build check is not caused by this PR's test tags or suite changes. The job fails during make build, before the extension binary is built and before test discovery/execution starts.

The CI builder image (rhel-9-golang-1.26-openshift-4.22) exports GOFLAGS=-mod=vendor, but this repository does not contain a committed vendor/ tree. That causes Go to exit with inconsistent vendoring. This PR does not modify go.mod, go.sum, vendor/, the Makefile, or the CI configuration, and running GOFLAGS=-mod=vendor go build locally reproduces the same error.

The CI job currently runs:

commands: HOME=/tmp make build

It should clear the builder's inherited vendor mode (or explicitly select module mode), for example:

commands: GOFLAGS= HOME=/tmp make build

Alternatively, the repository Makefile can clear GOFLAGS on its final go build command, consistent with test/e2e/bindata.mk, which already uses GOFLAGS= when installing go-bindata.

Failing run: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift-eng_openshift-logging-e2e-tests/1/pull-ci-openshift-eng-openshift-logging-e2e-tests-main-build/2092850128766373888