From 377d702c549d6706e553f725165b39ea00f19869 Mon Sep 17 00:00:00 2001 From: Joseph Date: Tue, 11 Aug 2026 11:27:59 -0700 Subject: [PATCH 01/13] docs: add design proposal for AllNamespaces install mode Adds a phased plan to migrate OADP from OwnNamespace-only to supporting AllNamespaces install mode via a separate OLM channel. Covers namespace decoupling, empty watch namespace handling, two-bundle build infrastructure, e2e test updates, CI integration, and customer migration documentation. Co-Authored-By: Claude Opus 4.6 (1M context) --- .../allnamespaces-install-mode_design.md | 338 ++++++++++++++++++ 1 file changed, 338 insertions(+) create mode 100644 docs/design/allnamespaces-install-mode_design.md diff --git a/docs/design/allnamespaces-install-mode_design.md b/docs/design/allnamespaces-install-mode_design.md new file mode 100644 index 00000000000..ca704af7be6 --- /dev/null +++ b/docs/design/allnamespaces-install-mode_design.md @@ -0,0 +1,338 @@ +# AllNamespaces Install Mode for OADP Operator + +## Abstract + +OADP operator currently supports only `OwnNamespace` install mode via OLM. +This proposal describes a phased plan to add `AllNamespaces` install mode support, delivered as a separate OLM channel to allow customers to migrate at their own pace. + +## Background + +The OADP operator is installed via OLM with a strict `OwnNamespace` install mode. +The CSV declares only `OwnNamespace: true` and all other modes are `supported: false`. +At runtime, the `WATCH_NAMESPACE` environment variable serves double duty: it identifies both the namespace where the operator lives and the namespace scope for the controller-runtime cache. + +In `OwnNamespace` mode, `WATCH_NAMESPACE` is sourced from the `olm.targetNamespaces` CSV annotation (set by the OperatorGroup's `targetNamespaces`), which always resolves to the operator's own namespace. +This conflation works today but breaks in `AllNamespaces` mode, where `olm.targetNamespaces` is empty (meaning "watch everything") but the operator still needs to know its own namespace for PSA labeling, STS credential setup, CLI/VMDP downloads, and sub-controller configuration. + +RBAC is already cluster-scoped (ClusterRoles and ClusterRoleBindings), so no fundamental RBAC changes are required. +No webhooks are currently enabled. +A `ClusterWideClient` (uncached) already exists for cross-namespace DPA validation. + +## Goals + +- Enable `AllNamespaces` install mode as a new OLM channel alongside the existing `OwnNamespace` channel. +- Provide a documented migration path for customers moving from `OwnNamespace` to `AllNamespaces`. +- Maintain full backward compatibility for existing `OwnNamespace` installations. + +## Non Goals + +- Multi-tenant Velero (one DPA per namespace) is out of scope for the initial implementation. Global DPA singleton enforcement will be maintained. +- Removing `OwnNamespace` support. Both modes will coexist until a future deprecation decision. +- `SingleNamespace` or `MultiNamespace` install mode support. + +## High-Level Design + +The work is split into six phases, each independently mergeable: + +1. **Decouple `OPERATOR_NAMESPACE` from `WATCH_NAMESPACE`** (refactoring, no behavioral change). +2. **Handle empty `WATCH_NAMESPACE`** in the controller-runtime manager (enables AllNamespaces code path). +3. **Build infrastructure for two bundle variants** (kustomize overlays, Makefile targets, catalog with two channels). +4. **E2E test infrastructure** for both install modes. +5. **CI/Prow integration** with AllNamespaces-specific jobs. +6. **Migration documentation** for customers. + +## Detailed Design + +### Current State + +| Area | Current Behavior | Key File(s) | +|---|---|---| +| CSV installModes | Only `OwnNamespace: true` | `config/manifests/bases/oadp-operator.clusterserviceversion.yaml` (lines 467-475) | +| CSV WATCH_NAMESPACE source | `olm.targetNamespaces` annotation | `bundle/manifests/oadp-operator.clusterserviceversion.yaml` (lines 1449-1452) | +| Manager WATCH_NAMESPACE source | `metadata.namespace` (downward API) | `config/manager/manager.yaml` (lines 63-66) | +| Cache scoping | `DefaultNamespaces` map with single entry | `cmd/main.go` (lines 204-208) | +| PSA labeling | Patches `watchNamespace`, errors if empty | `cmd/main.go` (lines 362-389) | +| CLI/VMDP downloads | Skipped if `watchNamespace` is empty | `cmd/main.go` (lines 305-306) | +| STS flow | Reads `WATCH_NAMESPACE` as install namespace | `pkg/credentials/stsflow/stsflow.go` (line 115) | +| Sub-controllers | All receive `WATCH_NAMESPACE` = own namespace | `nonadmin_controller.go` (line 176), `kubevirt_datamover_controller.go` (line 152), `vmfilerestore_controller.go` (line 184) | +| E2E OperatorGroup | Always `targetNamespaces: [namespace]` | `Makefile` (line 639), `upgrade_suite_test.go` (lines 31-50) | +| Cross-NS validation | Uses uncached `ClusterWideClient` | `cmd/main.go` (lines 260-270), `validator.go` (line 144) | +| RBAC | Already cluster-scoped (ClusterRoles) | `config/rbac/role.yaml`, CSV `clusterPermissions` | +| OLM channels | Single channel: `dev` (release branches use e.g. `oadp-1.5`) | `Makefile` (lines 23, 33), `bundle/metadata/annotations.yaml` | + +### Phase 1: Decouple OPERATOR_NAMESPACE from WATCH_NAMESPACE + +Introduce `OPERATOR_NAMESPACE` as a distinct concept from `WATCH_NAMESPACE`. +Today `WATCH_NAMESPACE` is used for both "where the operator lives" and "what to watch." +In `AllNamespaces` mode these diverge: `WATCH_NAMESPACE` is empty (watch all) but the operator still needs to know its home namespace. + +#### Changes + +| File | Change | Lines | +|---|---|---| +| `config/manager/manager.yaml` | Add `OPERATOR_NAMESPACE` env var via downward API (`metadata.namespace`) | near 63 | +| `cmd/main.go` | Add `getOperatorNamespace()` helper, modeled on `getWatchNamespace()` | near 348 | +| `cmd/main.go` | `addPodSecurityPrivilegedLabels()` uses `operatorNamespace` instead of `watchNamespace` | 140 | +| `cmd/main.go` | `CLIDownloadSetup` / `VMDPDownloadSetup` `Namespace` and `OperatorNamespace` use `operatorNamespace` | 312-329 | +| `pkg/credentials/stsflow/stsflow.go` | Read `OPERATOR_NAMESPACE` instead of `WATCH_NAMESPACE` | 115 | +| `internal/controller/nonadmin_controller.go` | Propagate `OPERATOR_NAMESPACE` (resolves existing TODO at line 176) | 176 | +| `internal/controller/kubevirt_datamover_controller.go` | Propagate `OPERATOR_NAMESPACE` | 152 | +| `internal/controller/vmfilerestore_controller.go` | Propagate `OPERATOR_NAMESPACE` | 184 | + +#### Validation + +- `make test` passes. +- Existing e2e tests pass unchanged (both vars resolve to the same value under OwnNamespace). + +#### Risk: Low + +Pure refactoring. No behavioral change. + +### Phase 2: Handle Empty WATCH_NAMESPACE in the Manager + +Make the controller-runtime manager work correctly when `WATCH_NAMESPACE` is empty, which signals AllNamespaces mode. + +#### Changes + +| File | Change | Lines | +|---|---|---| +| `cmd/main.go` | Conditional cache config: skip `DefaultNamespaces` when `watchNamespace` is empty (cache watches all namespaces) | 204-208 | +| `cmd/main.go` | `getWatchNamespace()`: empty or unset is now valid; log info instead of error | 127-131, 348-360 | +| `cmd/main.go` | Remove the `watchNamespace == ""` skip for CLI/VMDP setup (these use `operatorNamespace` from Phase 1) | 305-306 | +| `cmd/main.go` | `addPodSecurityPrivilegedLabels`: already fixed in Phase 1 to use `operatorNamespace`; verify empty-string guard is removed | 362-368 | + +#### DPA Singleton Enforcement + +`validator.go` (line 144) uses `ClusterWideClient` to list all DPAs cluster-wide and enforce singleton constraints (NonAdminController, VolumeSnapshotMover). +In AllNamespaces mode, global DPA singleton enforcement is maintained (one DPA across the entire cluster). +Multi-tenant (one DPA per namespace) is a future enhancement. + +#### Validation + +- Unit tests for the empty `WATCH_NAMESPACE` code path. +- Manual testing with `WATCH_NAMESPACE=""` and `OPERATOR_NAMESPACE=openshift-adp`. + +#### Risk: Medium + +Behavioral change for the empty-namespace path, but that path is not reachable until Phase 3 ships an AllNamespaces CSV. + +### Phase 3: Build Infrastructure for Two Bundle Variants + +Produce two distinct OLM bundles from one codebase, shipped as separate channels in the same OLM package. + +#### Two-CSV Approach via Channels + +A single CSV cannot have both `OwnNamespace` and `AllNamespaces` enabled simultaneously. +Instead, the catalog will contain two channels per release, each with its own CSV: + +``` +Catalog: oadp-operator-catalog +└── Package: oadp-operator + ├── Channel: dev ← OwnNamespace CSV + │ └── oadp-operator.v99.0.0 + └── Channel: dev-allnamespaces ← AllNamespaces CSV + └── oadp-operator.v99.0.0-allns +``` + +Release branches follow the same pattern: `stable-1.6` (OwnNamespace) and `stable-1.6-allnamespaces` (AllNamespaces). + +#### Changes + +| Area | Change | +|---|---| +| **CSV base template** | Keep `config/manifests/bases/oadp-operator.clusterserviceversion.yaml` as the shared base | +| **Kustomize overlays** | Create `config/manifests/overlays/ownnamespace/` and `config/manifests/overlays/allnamespaces/` with patches for `installModes` and deployment env vars | +| **AllNamespaces overlay** | Patches: (1) `installModes` set to only `AllNamespaces: true`, (2) add `OPERATOR_NAMESPACE` env var to deploymentSpec, (3) `WATCH_NAMESPACE` source remains `olm.targetNamespaces` (will be empty with global OperatorGroup) | +| **OwnNamespace overlay** | Patches: keeps current behavior (only `OwnNamespace: true`). Identity transform initially. | +| **Makefile** | Add `INSTALL_MODE ?= OwnNamespace`. New targets: `bundle-allnamespaces`, `bundle-build-allnamespaces`, `catalog-build-allnamespaces` | +| **Catalog build** | Parameterize `Dockerfile.catalog` and `catalog-build` target to produce a catalog with two channels: existing channel (OwnNamespace bundle) and new `-allnamespaces` channel (AllNamespaces bundle) | +| **CSV naming** | AllNamespaces CSV uses a distinct version suffix: `oadp-operator.v99.0.0-allns` vs `oadp-operator.v99.0.0` | + +#### Validation + +- `make bundle` and `make bundle-allnamespaces` both produce valid bundles. +- `opm validate` passes on both bundles. +- Catalog with two channels builds and serves correctly. + +#### Risk: Medium + +Build plumbing only, no runtime impact. + +### Phase 4: E2E Test Infrastructure + +Tests need to validate both install modes. + +#### Changes + +| Area | Change | +|---|---| +| `Makefile` `deploy-olm` (line 634-642) | Parameterize OperatorGroup creation: when `INSTALL_MODE=AllNamespaces`, create OperatorGroup with empty `spec` (no `targetNamespaces`). Default keeps current behavior. | +| `Makefile` | New target: `deploy-olm-allnamespaces` that sets `INSTALL_MODE=AllNamespaces` and `DEFAULT_CHANNEL=dev-allnamespaces` | +| `tests/e2e/upgrade_suite_test.go` (lines 31-50) | Parameterize OperatorGroup creation to support both modes based on a test flag | +| E2E test scenarios | Add: install AllNamespaces, create DPA in operator namespace, verify Velero deploys. Verify singleton enforcement. Verify sub-controller namespace config. | + +#### Validation + +Full e2e suite passes with both `make deploy-olm` (OwnNamespace) and `make deploy-olm-allnamespaces`. + +#### Risk: Medium + +Test infrastructure changes are additive. + +### Phase 5: CI/Prow Integration + +AllNamespaces mode needs CI coverage. + +#### Changes + +| Area | Change | +|---|---| +| `openshift/release` config | Add new presubmit job(s) running e2e tests with `INSTALL_MODE=AllNamespaces` | +| Job naming | e.g., `e2e-aws-allnamespaces` alongside existing `e2e-aws` | +| Periodic jobs | Add AllNamespaces variants for nightly runs | + +#### Validation + +CI jobs pass on a test PR. + +#### Risk: Low + +Additive CI config in a separate repo. + +### Phase 6: Migration Documentation + +Customers migrating from OwnNamespace to AllNamespaces need clear, tested manual steps. +OLM does not automate the OperatorGroup swap. + +#### Prerequisites + +- Cluster admin access. +- No active backups or restores in progress. +- Current OADP version supports AllNamespaces channel (minimum version TBD). + +#### Migration Steps + +**Step 1: Verify current state** + +```bash +oc get subscription oadp-operator -n openshift-adp -o yaml +oc get operatorgroup -n openshift-adp -o yaml +oc get dpa -n openshift-adp +``` + +**Step 2: Switch subscription channel** + +```bash +oc patch subscription oadp-operator -n openshift-adp \ + --type merge -p '{"spec":{"channel":"stable-1.x-allnamespaces"}}' +``` + +OLM installs the new CSV. +The operator pod restarts, but the OperatorGroup still scopes it to OwnNamespace. +No behavioral change yet. + +**Step 3: Delete the existing namespaced OperatorGroup** + +```bash +oc delete operatorgroup oadp-operator-group -n openshift-adp +``` + +The operator pod stops (OLM removes the deployment when no valid OperatorGroup exists). + +**Step 4: Create a global OperatorGroup** + +```bash +cat <-allnamespaces` (e.g., `dev-allnamespaces`, `stable-1.6-allnamespaces`). +Open to shorter alternatives if the convention is too verbose. + +3. **Velero operational scope**: Even when the operator watches all namespaces, Velero's deployment lives in one namespace and backs up resources across namespaces (this is existing behavior). +No change expected, but worth validating in e2e tests. + +4. **Minimum version for migration**: Which OADP release will be the first to ship the AllNamespaces channel? +This determines the migration documentation's version requirements. From cc94cdae96ac8fb2667fc21032578e9b3e8a05b0 Mon Sep 17 00:00:00 2001 From: Joseph Date: Tue, 11 Aug 2026 11:32:05 -0700 Subject: [PATCH 02/13] docs: present channels and packages as open delivery options Phase 3 now presents both approaches with trade-offs and decision criteria rather than pre-deciding on channels. Phase 4 and Phase 6 document how e2e tests and migration steps differ per option. Co-Authored-By: Claude Opus 4.6 (1M context) --- .../allnamespaces-install-mode_design.md | 224 +++++++++++++++--- 1 file changed, 188 insertions(+), 36 deletions(-) diff --git a/docs/design/allnamespaces-install-mode_design.md b/docs/design/allnamespaces-install-mode_design.md index ca704af7be6..0076e01048c 100644 --- a/docs/design/allnamespaces-install-mode_design.md +++ b/docs/design/allnamespaces-install-mode_design.md @@ -3,7 +3,8 @@ ## Abstract OADP operator currently supports only `OwnNamespace` install mode via OLM. -This proposal describes a phased plan to add `AllNamespaces` install mode support, delivered as a separate OLM channel to allow customers to migrate at their own pace. +This proposal describes a phased plan to add `AllNamespaces` install mode support, delivered via a second CSV to allow customers to migrate at their own pace. +Two delivery options are presented for the second CSV: a separate channel within the existing OLM package, or a separate OLM package with its own catalog entry. ## Background @@ -20,7 +21,7 @@ A `ClusterWideClient` (uncached) already exists for cross-namespace DPA validati ## Goals -- Enable `AllNamespaces` install mode as a new OLM channel alongside the existing `OwnNamespace` channel. +- Enable `AllNamespaces` install mode via a second CSV, delivered alongside the existing `OwnNamespace` CSV. - Provide a documented migration path for customers moving from `OwnNamespace` to `AllNamespaces`. - Maintain full backward compatibility for existing `OwnNamespace` installations. @@ -36,7 +37,7 @@ The work is split into six phases, each independently mergeable: 1. **Decouple `OPERATOR_NAMESPACE` from `WATCH_NAMESPACE`** (refactoring, no behavioral change). 2. **Handle empty `WATCH_NAMESPACE`** in the controller-runtime manager (enables AllNamespaces code path). -3. **Build infrastructure for two bundle variants** (kustomize overlays, Makefile targets, catalog with two channels). +3. **Build infrastructure for two bundle variants** (kustomize overlays, Makefile targets, two CSVs). Delivery mechanism (channels vs packages) is decided in this phase. 4. **E2E test infrastructure** for both install modes. 5. **CI/Prow integration** with AllNamespaces-specific jobs. 6. **Migration documentation** for customers. @@ -118,12 +119,26 @@ Behavioral change for the empty-namespace path, but that path is not reachable u ### Phase 3: Build Infrastructure for Two Bundle Variants -Produce two distinct OLM bundles from one codebase, shipped as separate channels in the same OLM package. +Produce two distinct OLM bundles from one codebase. +A single CSV cannot have both `OwnNamespace` and `AllNamespaces` enabled simultaneously, so two CSVs are required. +The delivery mechanism for the second CSV is decided in this phase. -#### Two-CSV Approach via Channels +#### Common Build Changes (both options) -A single CSV cannot have both `OwnNamespace` and `AllNamespaces` enabled simultaneously. -Instead, the catalog will contain two channels per release, each with its own CSV: +Regardless of delivery option, the following build changes are needed to produce two bundle variants from one codebase: + +| Area | Change | +|---|---| +| **CSV base template** | Keep `config/manifests/bases/oadp-operator.clusterserviceversion.yaml` as the shared base | +| **Kustomize overlays** | Create `config/manifests/overlays/ownnamespace/` and `config/manifests/overlays/allnamespaces/` with patches for `installModes` and deployment env vars | +| **AllNamespaces overlay** | Patches: (1) `installModes` set to only `AllNamespaces: true`, (2) add `OPERATOR_NAMESPACE` env var to deploymentSpec, (3) `WATCH_NAMESPACE` source remains `olm.targetNamespaces` (will be empty with global OperatorGroup) | +| **OwnNamespace overlay** | Patches: keeps current behavior (only `OwnNamespace: true`). Identity transform initially. | +| **Makefile** | Add `INSTALL_MODE ?= OwnNamespace`. New targets: `bundle-allnamespaces`, `bundle-build-allnamespaces` | + +#### Option A: Two Channels in the Same Package + +The AllNamespaces CSV is shipped as a separate channel within the existing `oadp-operator` package. +Both channels live in the same catalog and share the same package identity. ``` Catalog: oadp-operator-catalog @@ -136,23 +151,69 @@ Catalog: oadp-operator-catalog Release branches follow the same pattern: `stable-1.6` (OwnNamespace) and `stable-1.6-allnamespaces` (AllNamespaces). -#### Changes +**Additional changes for Option A:** | Area | Change | |---|---| -| **CSV base template** | Keep `config/manifests/bases/oadp-operator.clusterserviceversion.yaml` as the shared base | -| **Kustomize overlays** | Create `config/manifests/overlays/ownnamespace/` and `config/manifests/overlays/allnamespaces/` with patches for `installModes` and deployment env vars | -| **AllNamespaces overlay** | Patches: (1) `installModes` set to only `AllNamespaces: true`, (2) add `OPERATOR_NAMESPACE` env var to deploymentSpec, (3) `WATCH_NAMESPACE` source remains `olm.targetNamespaces` (will be empty with global OperatorGroup) | -| **OwnNamespace overlay** | Patches: keeps current behavior (only `OwnNamespace: true`). Identity transform initially. | -| **Makefile** | Add `INSTALL_MODE ?= OwnNamespace`. New targets: `bundle-allnamespaces`, `bundle-build-allnamespaces`, `catalog-build-allnamespaces` | -| **Catalog build** | Parameterize `Dockerfile.catalog` and `catalog-build` target to produce a catalog with two channels: existing channel (OwnNamespace bundle) and new `-allnamespaces` channel (AllNamespaces bundle) | +| **Catalog build** | Parameterize `Dockerfile.catalog` and `catalog-build` target to produce a single catalog with two channels: existing channel (OwnNamespace bundle) and new `-allnamespaces` channel (AllNamespaces bundle) | | **CSV naming** | AllNamespaces CSV uses a distinct version suffix: `oadp-operator.v99.0.0-allns` vs `oadp-operator.v99.0.0` | +| **Channel naming** | Convention: `-allnamespaces` (e.g., `dev-allnamespaces`, `stable-1.6-allnamespaces`) | + +**Trade-offs:** + +| Pro | Con | +|---|---| +| Single package in OperatorHub; cleaner customer experience | Every release branch must produce two bundles and two channel entries | +| Migration via Subscription channel change (no uninstall/reinstall of the package) | Channel names are overloaded (channels typically mean release stability, not install topology) | +| Single catalog image to build and publish | Customer must still manually swap the OperatorGroup after changing channel | + +#### Option B: Two Separate OLM Packages + +The AllNamespaces CSV is shipped as a separate OLM package with its own catalog entry. +Each package has its own identity and upgrade graph. + +``` +Catalog: oadp-operator-catalog +├── Package: oadp-operator ← OwnNamespace CSV +│ └── Channel: dev +│ └── oadp-operator.v99.0.0 +└── Package: oadp-operator-allnamespaces ← AllNamespaces CSV + └── Channel: dev + └── oadp-operator-allnamespaces.v99.0.0 +``` + +**Additional changes for Option B:** + +| Area | Change | +|---|---| +| **Catalog build** | Produce a single catalog containing two packages, each with its own channel(s). Or produce two separate catalogs (one per package). | +| **CSV naming** | AllNamespaces CSV uses a distinct package name: `oadp-operator-allnamespaces` | +| **Bundle metadata** | AllNamespaces bundle has its own `annotations.yaml` with `operators.operatorframework.io.bundle.package.v1: oadp-operator-allnamespaces` | +| **Makefile** | Additional target: `catalog-build-allnamespaces` if producing separate catalogs | + +**Trade-offs:** + +| Pro | Con | +|---|---| +| Clean separation; no channel naming overload | Two tiles in OperatorHub; customers must know which to pick | +| Each package has its own independent upgrade graph | Migration requires uninstalling the old package and installing the new one | +| Channel names stay semantic (stable, dev, etc.) | Doubles catalog/release artifacts per version | +| Simpler catalog structure per package | Customer loses the Subscription during migration (must re-create) | + +#### Decision Criteria + +The choice between Option A and Option B should consider: + +1. **Downstream release pipeline**: Does Konflux/ART handle two channels in one package easily, or is a second package simpler to wire up? +2. **Migration UX priority**: Is avoiding uninstall/reinstall (Option A) worth the channel naming complexity? +3. **Long-term intent**: If OwnNamespace will eventually be deprecated, Option A lets you sunset a channel; Option B requires deprecating an entire package. +4. **OperatorHub presentation**: One tile with a channel picker (Option A) vs two tiles (Option B). #### Validation - `make bundle` and `make bundle-allnamespaces` both produce valid bundles. - `opm validate` passes on both bundles. -- Catalog with two channels builds and serves correctly. +- Catalog builds and serves correctly with the chosen delivery structure. #### Risk: Medium @@ -161,15 +222,17 @@ Build plumbing only, no runtime impact. ### Phase 4: E2E Test Infrastructure Tests need to validate both install modes. +The test infrastructure changes depend on the delivery option chosen in Phase 3. #### Changes | Area | Change | |---|---| | `Makefile` `deploy-olm` (line 634-642) | Parameterize OperatorGroup creation: when `INSTALL_MODE=AllNamespaces`, create OperatorGroup with empty `spec` (no `targetNamespaces`). Default keeps current behavior. | -| `Makefile` | New target: `deploy-olm-allnamespaces` that sets `INSTALL_MODE=AllNamespaces` and `DEFAULT_CHANNEL=dev-allnamespaces` | +| `Makefile` | New target: `deploy-olm-allnamespaces`. For Option A (channels): sets `INSTALL_MODE=AllNamespaces` and `DEFAULT_CHANNEL=dev-allnamespaces`. For Option B (packages): sets `INSTALL_MODE=AllNamespaces` and overrides `CATALOG_SOURCE_NAME` and subscription package name. | | `tests/e2e/upgrade_suite_test.go` (lines 31-50) | Parameterize OperatorGroup creation to support both modes based on a test flag | | E2E test scenarios | Add: install AllNamespaces, create DPA in operator namespace, verify Velero deploys. Verify singleton enforcement. Verify sub-controller namespace config. | +| Migration test (Option A only) | Test switching channel on an existing Subscription and swapping the OperatorGroup to verify the documented migration path works end-to-end. | #### Validation @@ -203,14 +266,15 @@ Additive CI config in a separate repo. Customers migrating from OwnNamespace to AllNamespaces need clear, tested manual steps. OLM does not automate the OperatorGroup swap. +The migration path differs depending on the delivery option chosen in Phase 3. -#### Prerequisites +#### Prerequisites (both options) - Cluster admin access. - No active backups or restores in progress. -- Current OADP version supports AllNamespaces channel (minimum version TBD). +- Current OADP version supports the AllNamespaces CSV (minimum version TBD). -#### Migration Steps +#### Migration Path for Option A (Channels) **Step 1: Verify current state** @@ -272,31 +336,114 @@ oc get dpa -n openshift-adp -o jsonpath='{.items[0].status.conditions}' oc get deployment -n openshift-adp -l app.kubernetes.io/name=velero ``` -#### Rollback Steps +**Rollback (Option A):** 1. Delete the global OperatorGroup. 2. Create the namespaced OperatorGroup with `targetNamespaces: [openshift-adp]`. 3. Switch the subscription channel back to the OwnNamespace channel. -#### Expected Downtime +#### Migration Path for Option B (Packages) + +**Step 1: Verify current state** + +```bash +oc get subscription oadp-operator -n openshift-adp -o yaml +oc get operatorgroup -n openshift-adp -o yaml +oc get dpa -n openshift-adp +``` + +**Step 2: Delete the existing subscription (keeps the DPA and Velero resources)** + +```bash +oc delete subscription oadp-operator -n openshift-adp +``` + +**Step 3: Delete the OwnNamespace CSV** + +```bash +CSV_NAME=$(oc get csv -n openshift-adp -o name | grep oadp-operator) +oc delete $CSV_NAME -n openshift-adp +``` -Brief operator unavailability between steps 3 and 4 (seconds to approximately one minute). +The operator pod is removed. DPA and Velero resources remain. + +**Step 4: Delete the existing namespaced OperatorGroup** + +```bash +oc delete operatorgroup oadp-operator-group -n openshift-adp +``` + +**Step 5: Create a global OperatorGroup** + +```bash +cat <-allnamespaces` (e.g., `dev-allnamespaces`, `stable-1.6-allnamespaces`). +3. **Channel naming convention (Option A only)**: If channels are chosen, proposed convention is `-allnamespaces` (e.g., `dev-allnamespaces`, `stable-1.6-allnamespaces`). Open to shorter alternatives if the convention is too verbose. -3. **Velero operational scope**: Even when the operator watches all namespaces, Velero's deployment lives in one namespace and backs up resources across namespaces (this is existing behavior). +4. **Velero operational scope**: Even when the operator watches all namespaces, Velero's deployment lives in one namespace and backs up resources across namespaces (this is existing behavior). No change expected, but worth validating in e2e tests. -4. **Minimum version for migration**: Which OADP release will be the first to ship the AllNamespaces channel? +5. **Minimum version for migration**: Which OADP release will be the first to ship the AllNamespaces CSV? This determines the migration documentation's version requirements. From 6a3c028255426805873885d5390ca45ef24f9435 Mon Sep 17 00:00:00 2001 From: Joseph Date: Tue, 11 Aug 2026 11:44:25 -0700 Subject: [PATCH 03/13] =?UTF-8?q?docs:=20simplify=20plan=20=E2=80=94=20All?= =?UTF-8?q?Namespaces=20as=20install=20mode=20only?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit WATCH_NAMESPACE sourced from metadata.namespace (not olm.targetNamespaces) means the operator keeps watching only its own namespace regardless of install mode. No Go code changes needed. Moves OPERATOR_NAMESPACE decoupling and cluster-wide watching to Future Enhancements. Reduces plan from 6 phases to 5. Phase 1 is now just the CSV change. Co-Authored-By: Claude Opus 4.6 (1M context) --- .../allnamespaces-install-mode_design.md | 180 +++++++++--------- 1 file changed, 92 insertions(+), 88 deletions(-) diff --git a/docs/design/allnamespaces-install-mode_design.md b/docs/design/allnamespaces-install-mode_design.md index 0076e01048c..89e3409fbd8 100644 --- a/docs/design/allnamespaces-install-mode_design.md +++ b/docs/design/allnamespaces-install-mode_design.md @@ -4,16 +4,21 @@ OADP operator currently supports only `OwnNamespace` install mode via OLM. This proposal describes a phased plan to add `AllNamespaces` install mode support, delivered via a second CSV to allow customers to migrate at their own pace. +The operator's runtime behavior remains identical: it watches only the namespace it is deployed in, regardless of install mode. Two delivery options are presented for the second CSV: a separate channel within the existing OLM package, or a separate OLM package with its own catalog entry. ## Background The OADP operator is installed via OLM with a strict `OwnNamespace` install mode. The CSV declares only `OwnNamespace: true` and all other modes are `supported: false`. -At runtime, the `WATCH_NAMESPACE` environment variable serves double duty: it identifies both the namespace where the operator lives and the namespace scope for the controller-runtime cache. -In `OwnNamespace` mode, `WATCH_NAMESPACE` is sourced from the `olm.targetNamespaces` CSV annotation (set by the OperatorGroup's `targetNamespaces`), which always resolves to the operator's own namespace. -This conflation works today but breaks in `AllNamespaces` mode, where `olm.targetNamespaces` is empty (meaning "watch everything") but the operator still needs to know its own namespace for PSA labeling, STS credential setup, CLI/VMDP downloads, and sub-controller configuration. +At runtime, the `WATCH_NAMESPACE` environment variable controls which namespace the controller-runtime cache monitors. +In the OwnNamespace CSV, `WATCH_NAMESPACE` is sourced from the `olm.targetNamespaces` annotation, which OLM sets to the operator's own namespace. +In the non-OLM deployment (`config/manager/manager.yaml`), `WATCH_NAMESPACE` is sourced from `metadata.namespace` (the pod's own namespace via downward API). + +The key insight is that `AllNamespaces` is an OLM install mode — it controls what OperatorGroup the CSV is compatible with, not what the operator actually watches at runtime. +The AllNamespaces CSV can source `WATCH_NAMESPACE` from `metadata.namespace` (just like the non-OLM deployment does today) instead of from `olm.targetNamespaces`. +This means the operator keeps watching only its own namespace by default, even when installed via a global OperatorGroup. RBAC is already cluster-scoped (ClusterRoles and ClusterRoleBindings), so no fundamental RBAC changes are required. No webhooks are currently enabled. @@ -22,25 +27,26 @@ A `ClusterWideClient` (uncached) already exists for cross-namespace DPA validati ## Goals - Enable `AllNamespaces` install mode via a second CSV, delivered alongside the existing `OwnNamespace` CSV. +- Keep runtime behavior identical to today: the operator watches only the namespace it is deployed in. - Provide a documented migration path for customers moving from `OwnNamespace` to `AllNamespaces`. - Maintain full backward compatibility for existing `OwnNamespace` installations. ## Non Goals -- Multi-tenant Velero (one DPA per namespace) is out of scope for the initial implementation. Global DPA singleton enforcement will be maintained. +- Actually watching all namespaces. The operator continues to watch only its own namespace. Cluster-wide watching can be enabled in the future by setting `WATCH_NAMESPACE` to empty, but that is a separate enhancement requiring additional work (see Future Enhancements). +- Multi-tenant Velero (one DPA per namespace). Global DPA singleton enforcement will be maintained. - Removing `OwnNamespace` support. Both modes will coexist until a future deprecation decision. - `SingleNamespace` or `MultiNamespace` install mode support. ## High-Level Design -The work is split into six phases, each independently mergeable: +The work is split into five phases, each independently mergeable: -1. **Decouple `OPERATOR_NAMESPACE` from `WATCH_NAMESPACE`** (refactoring, no behavioral change). -2. **Handle empty `WATCH_NAMESPACE`** in the controller-runtime manager (enables AllNamespaces code path). -3. **Build infrastructure for two bundle variants** (kustomize overlays, Makefile targets, two CSVs). Delivery mechanism (channels vs packages) is decided in this phase. -4. **E2E test infrastructure** for both install modes. -5. **CI/Prow integration** with AllNamespaces-specific jobs. -6. **Migration documentation** for customers. +1. **Enable AllNamespaces install mode in the CSV** (the core change: flip installModes, source `WATCH_NAMESPACE` from `metadata.namespace`). +2. **Build infrastructure for two bundle variants** (kustomize overlays, Makefile targets, two CSVs). Delivery mechanism (channels vs packages) is decided in this phase. +3. **E2E test infrastructure** for both install modes. +4. **CI/Prow integration** with AllNamespaces-specific jobs. +5. **Migration documentation** for customers. ## Detailed Design @@ -61,63 +67,42 @@ The work is split into six phases, each independently mergeable: | RBAC | Already cluster-scoped (ClusterRoles) | `config/rbac/role.yaml`, CSV `clusterPermissions` | | OLM channels | Single channel: `dev` (release branches use e.g. `oadp-1.5`) | `Makefile` (lines 23, 33), `bundle/metadata/annotations.yaml` | -### Phase 1: Decouple OPERATOR_NAMESPACE from WATCH_NAMESPACE +### Phase 1: Enable AllNamespaces Install Mode in the CSV -Introduce `OPERATOR_NAMESPACE` as a distinct concept from `WATCH_NAMESPACE`. -Today `WATCH_NAMESPACE` is used for both "where the operator lives" and "what to watch." -In `AllNamespaces` mode these diverge: `WATCH_NAMESPACE` is empty (watch all) but the operator still needs to know its home namespace. +The core change. +The AllNamespaces CSV differs from the OwnNamespace CSV in exactly two ways: -#### Changes +1. `installModes` — only `AllNamespaces: true` (instead of only `OwnNamespace: true`). +2. `WATCH_NAMESPACE` source — `metadata.namespace` via downward API (instead of `olm.targetNamespaces` annotation). -| File | Change | Lines | -|---|---|---| -| `config/manager/manager.yaml` | Add `OPERATOR_NAMESPACE` env var via downward API (`metadata.namespace`) | near 63 | -| `cmd/main.go` | Add `getOperatorNamespace()` helper, modeled on `getWatchNamespace()` | near 348 | -| `cmd/main.go` | `addPodSecurityPrivilegedLabels()` uses `operatorNamespace` instead of `watchNamespace` | 140 | -| `cmd/main.go` | `CLIDownloadSetup` / `VMDPDownloadSetup` `Namespace` and `OperatorNamespace` use `operatorNamespace` | 312-329 | -| `pkg/credentials/stsflow/stsflow.go` | Read `OPERATOR_NAMESPACE` instead of `WATCH_NAMESPACE` | 115 | -| `internal/controller/nonadmin_controller.go` | Propagate `OPERATOR_NAMESPACE` (resolves existing TODO at line 176) | 176 | -| `internal/controller/kubevirt_datamover_controller.go` | Propagate `OPERATOR_NAMESPACE` | 152 | -| `internal/controller/vmfilerestore_controller.go` | Propagate `OPERATOR_NAMESPACE` | 184 | - -#### Validation - -- `make test` passes. -- Existing e2e tests pass unchanged (both vars resolve to the same value under OwnNamespace). - -#### Risk: Low - -Pure refactoring. No behavioral change. - -### Phase 2: Handle Empty WATCH_NAMESPACE in the Manager - -Make the controller-runtime manager work correctly when `WATCH_NAMESPACE` is empty, which signals AllNamespaces mode. +With `WATCH_NAMESPACE` sourced from `metadata.namespace`, the operator always resolves to the pod's own namespace regardless of what `olm.targetNamespaces` contains (which would be empty under a global OperatorGroup). +No Go code changes are needed. The operator binary is identical for both CSVs. #### Changes -| File | Change | Lines | -|---|---|---| -| `cmd/main.go` | Conditional cache config: skip `DefaultNamespaces` when `watchNamespace` is empty (cache watches all namespaces) | 204-208 | -| `cmd/main.go` | `getWatchNamespace()`: empty or unset is now valid; log info instead of error | 127-131, 348-360 | -| `cmd/main.go` | Remove the `watchNamespace == ""` skip for CLI/VMDP setup (these use `operatorNamespace` from Phase 1) | 305-306 | -| `cmd/main.go` | `addPodSecurityPrivilegedLabels`: already fixed in Phase 1 to use `operatorNamespace`; verify empty-string guard is removed | 362-368 | +| File | Change | +|---|---| +| `config/manifests/bases/oadp-operator.clusterserviceversion.yaml` (lines 467-475) | Create an AllNamespaces variant with only `AllNamespaces: true` in `installModes` | +| CSV deploymentSpec `WATCH_NAMESPACE` env var | Change source from `metadata.annotations['olm.targetNamespaces']` to `metadata.namespace` (downward API) in the AllNamespaces variant | +| `make bundle` | Regenerate bundle to verify the OwnNamespace CSV is unchanged | -#### DPA Singleton Enforcement +#### What does NOT change -`validator.go` (line 144) uses `ClusterWideClient` to list all DPAs cluster-wide and enforce singleton constraints (NonAdminController, VolumeSnapshotMover). -In AllNamespaces mode, global DPA singleton enforcement is maintained (one DPA across the entire cluster). -Multi-tenant (one DPA per namespace) is a future enhancement. +- No Go code changes. `cmd/main.go`, controllers, and all runtime behavior are untouched. +- `WATCH_NAMESPACE` always resolves to a non-empty namespace name (the pod's own namespace). +- Cache scoping, PSA labeling, STS flow, CLI/VMDP downloads, sub-controller propagation all continue to work exactly as today. +- RBAC is already cluster-scoped and does not need modification. #### Validation -- Unit tests for the empty `WATCH_NAMESPACE` code path. -- Manual testing with `WATCH_NAMESPACE=""` and `OPERATOR_NAMESPACE=openshift-adp`. +- `make test` passes (no code changes). +- Manual OLM install with AllNamespaces OperatorGroup: operator starts, `WATCH_NAMESPACE` = pod namespace, DPA reconciles normally. -#### Risk: Medium +#### Risk: Low -Behavioral change for the empty-namespace path, but that path is not reachable until Phase 3 ships an AllNamespaces CSV. +No runtime behavioral change. The only change is in the CSV metadata and env var source. -### Phase 3: Build Infrastructure for Two Bundle Variants +### Phase 2: Build Infrastructure for Two Bundle Variants Produce two distinct OLM bundles from one codebase. A single CSV cannot have both `OwnNamespace` and `AllNamespaces` enabled simultaneously, so two CSVs are required. @@ -130,9 +115,9 @@ Regardless of delivery option, the following build changes are needed to produce | Area | Change | |---|---| | **CSV base template** | Keep `config/manifests/bases/oadp-operator.clusterserviceversion.yaml` as the shared base | -| **Kustomize overlays** | Create `config/manifests/overlays/ownnamespace/` and `config/manifests/overlays/allnamespaces/` with patches for `installModes` and deployment env vars | -| **AllNamespaces overlay** | Patches: (1) `installModes` set to only `AllNamespaces: true`, (2) add `OPERATOR_NAMESPACE` env var to deploymentSpec, (3) `WATCH_NAMESPACE` source remains `olm.targetNamespaces` (will be empty with global OperatorGroup) | -| **OwnNamespace overlay** | Patches: keeps current behavior (only `OwnNamespace: true`). Identity transform initially. | +| **Kustomize overlays** | Create `config/manifests/overlays/ownnamespace/` and `config/manifests/overlays/allnamespaces/` with patches for `installModes` and `WATCH_NAMESPACE` env var source | +| **AllNamespaces overlay** | Patches: (1) `installModes` set to only `AllNamespaces: true`, (2) `WATCH_NAMESPACE` source changed from `olm.targetNamespaces` to `metadata.namespace` | +| **OwnNamespace overlay** | Patches: keeps current behavior (only `OwnNamespace: true`, `WATCH_NAMESPACE` from `olm.targetNamespaces`). Identity transform initially. | | **Makefile** | Add `INSTALL_MODE ?= OwnNamespace`. New targets: `bundle-allnamespaces`, `bundle-build-allnamespaces` | #### Option A: Two Channels in the Same Package @@ -219,10 +204,10 @@ The choice between Option A and Option B should consider: Build plumbing only, no runtime impact. -### Phase 4: E2E Test Infrastructure +### Phase 3: E2E Test Infrastructure Tests need to validate both install modes. -The test infrastructure changes depend on the delivery option chosen in Phase 3. +The test infrastructure changes depend on the delivery option chosen in Phase 2. #### Changes @@ -231,7 +216,7 @@ The test infrastructure changes depend on the delivery option chosen in Phase 3. | `Makefile` `deploy-olm` (line 634-642) | Parameterize OperatorGroup creation: when `INSTALL_MODE=AllNamespaces`, create OperatorGroup with empty `spec` (no `targetNamespaces`). Default keeps current behavior. | | `Makefile` | New target: `deploy-olm-allnamespaces`. For Option A (channels): sets `INSTALL_MODE=AllNamespaces` and `DEFAULT_CHANNEL=dev-allnamespaces`. For Option B (packages): sets `INSTALL_MODE=AllNamespaces` and overrides `CATALOG_SOURCE_NAME` and subscription package name. | | `tests/e2e/upgrade_suite_test.go` (lines 31-50) | Parameterize OperatorGroup creation to support both modes based on a test flag | -| E2E test scenarios | Add: install AllNamespaces, create DPA in operator namespace, verify Velero deploys. Verify singleton enforcement. Verify sub-controller namespace config. | +| E2E test scenarios | Add: install AllNamespaces, create DPA in operator namespace, verify Velero deploys. Verify `WATCH_NAMESPACE` resolves to operator namespace. Verify singleton enforcement. Verify sub-controller namespace config. | | Migration test (Option A only) | Test switching channel on an existing Subscription and swapping the OperatorGroup to verify the documented migration path works end-to-end. | #### Validation @@ -242,7 +227,7 @@ Full e2e suite passes with both `make deploy-olm` (OwnNamespace) and `make deplo Test infrastructure changes are additive. -### Phase 5: CI/Prow Integration +### Phase 4: CI/Prow Integration AllNamespaces mode needs CI coverage. @@ -262,11 +247,15 @@ CI jobs pass on a test PR. Additive CI config in a separate repo. -### Phase 6: Migration Documentation +### Phase 5: Migration Documentation Customers migrating from OwnNamespace to AllNamespaces need clear, tested manual steps. OLM does not automate the OperatorGroup swap. -The migration path differs depending on the delivery option chosen in Phase 3. +The migration path differs depending on the delivery option chosen in Phase 2. + +Note: after migration, the operator's runtime behavior is identical. +`WATCH_NAMESPACE` is sourced from `metadata.namespace` in the AllNamespaces CSV, so it always resolves to the operator's own namespace. +The operator does not start watching all namespaces. #### Prerequisites (both options) @@ -316,9 +305,9 @@ spec: {} EOF ``` -OLM re-deploys the operator with `olm.targetNamespaces` empty. -`WATCH_NAMESPACE` becomes empty. -AllNamespaces mode is now active. +OLM re-deploys the operator. +`WATCH_NAMESPACE` is sourced from `metadata.namespace`, so the operator continues to watch only the `openshift-adp` namespace. +Behavior is identical to before the migration. **Step 5: Verify the migration** @@ -404,8 +393,9 @@ spec: EOF ``` -OLM installs the AllNamespaces CSV with `olm.targetNamespaces` empty. -The operator starts in AllNamespaces mode and reconciles the existing DPA. +OLM installs the AllNamespaces CSV. +`WATCH_NAMESPACE` is sourced from `metadata.namespace`, so the operator watches only the `openshift-adp` namespace. +The operator reconciles the existing DPA. Behavior is identical to before the migration. **Step 7: Verify the migration** @@ -450,41 +440,55 @@ Two separate CSVs make the install mode explicit and independently releasable. The `velero` ServiceAccount's ClusterRole grants near-cluster-admin permissions (`apiGroups: ['*'], resources: ['*']`). In `OwnNamespace` mode this is contained by the OperatorGroup scope. In `AllNamespaces` mode the RBAC is identical (already cluster-scoped), but the perception of blast radius changes. -A security review of the velero SA permissions should be conducted as part of Phase 3. +Since the operator's runtime behavior is unchanged (still watches only its own namespace), the actual security posture does not change. +A security review of the velero SA permissions is still recommended as a general hygiene item. ## Compatibility - Existing `OwnNamespace` installations are unaffected. The OwnNamespace CSV continues to exist and receive updates regardless of delivery option. -- Migration from `OwnNamespace` to `AllNamespaces` is a manual process documented in Phase 6. The exact steps depend on the delivery option chosen in Phase 3. -- The `OPERATOR_NAMESPACE` env var (Phase 1) is additive and backward-compatible. -- The `WATCH_NAMESPACE` empty-string handling (Phase 2) does not affect existing deployments where the var is always set to a non-empty value. +- Migration from `OwnNamespace` to `AllNamespaces` is a manual process documented in Phase 5. The exact steps depend on the delivery option chosen in Phase 2. +- No Go code changes are required. The operator binary is identical for both CSVs. +- Runtime behavior is identical in both modes: `WATCH_NAMESPACE` always resolves to the operator's own namespace. ## Implementation | Phase | Scope | Risk | Depends on | Parallelizable | |---|---|---|---|---| -| 1. Decouple OPERATOR_NAMESPACE | Refactoring | Low | None | No (foundation) | -| 2. Handle empty WATCH_NAMESPACE | Runtime behavior | Medium | Phase 1 | No | -| 3. Two bundle variants | Build infrastructure | Medium | Phase 2 | No | -| 4. E2E test infrastructure | Test infrastructure | Medium | Phase 3 | No | -| 5. CI/Prow integration | CI config | Low | Phase 4 | Yes (with Phase 6) | -| 6. Migration documentation | Docs | Low | Phase 3 | Yes (with Phase 5) | +| 1. Enable AllNamespaces in CSV | CSV metadata | Low | None | No (foundation) | +| 2. Two bundle variants | Build infrastructure | Medium | Phase 1 | No | +| 3. E2E test infrastructure | Test infrastructure | Medium | Phase 2 | No | +| 4. CI/Prow integration | CI config | Low | Phase 3 | Yes (with Phase 5) | +| 5. Migration documentation | Docs | Low | Phase 2 | Yes (with Phase 4) | ## Open Issues 1. **Delivery option**: Two channels in the same package (Option A) or two separate OLM packages (Option B)? -This decision must be made in Phase 3 and affects Phases 4, 5, and 6. +This decision must be made in Phase 2 and affects Phases 3, 4, and 5. Key factors: downstream release pipeline compatibility (Konflux/ART), migration UX priority, and long-term deprecation strategy for OwnNamespace. -See the Decision Criteria section in Phase 3 for details. +See the Decision Criteria section in Phase 2 for details. -2. **DPA singleton scope**: In AllNamespaces mode, should we allow one DPA per namespace (multi-tenant Velero) or enforce a single DPA globally? -Initial recommendation is global singleton to minimize blast radius; multi-tenant support can follow as a separate enhancement. - -3. **Channel naming convention (Option A only)**: If channels are chosen, proposed convention is `-allnamespaces` (e.g., `dev-allnamespaces`, `stable-1.6-allnamespaces`). +2. **Channel naming convention (Option A only)**: If channels are chosen, proposed convention is `-allnamespaces` (e.g., `dev-allnamespaces`, `stable-1.6-allnamespaces`). Open to shorter alternatives if the convention is too verbose. -4. **Velero operational scope**: Even when the operator watches all namespaces, Velero's deployment lives in one namespace and backs up resources across namespaces (this is existing behavior). -No change expected, but worth validating in e2e tests. - -5. **Minimum version for migration**: Which OADP release will be the first to ship the AllNamespaces CSV? +3. **Minimum version for migration**: Which OADP release will be the first to ship the AllNamespaces CSV? This determines the migration documentation's version requirements. + +## Future Enhancements + +These are out of scope for this proposal but are natural follow-on work: + +### Decouple OPERATOR_NAMESPACE from WATCH_NAMESPACE + +If a future requirement is for the operator to actually watch all namespaces (or a different namespace), `WATCH_NAMESPACE` would need to be set to empty or to a different value than the operator's own namespace. +In that case, a separate `OPERATOR_NAMESPACE` env var (sourced from `metadata.namespace`) would be needed so the operator knows its own namespace for PSA labeling, STS credential setup, CLI/VMDP downloads, and sub-controller configuration. + +Key files that would need changes: +- `cmd/main.go`: `addPodSecurityPrivilegedLabels()`, `CLIDownloadSetup`, `VMDPDownloadSetup` would use `OPERATOR_NAMESPACE` instead of `WATCH_NAMESPACE`. +- `pkg/credentials/stsflow/stsflow.go`: Read `OPERATOR_NAMESPACE` for install namespace. +- `internal/controller/nonadmin_controller.go`, `kubevirt_datamover_controller.go`, `vmfilerestore_controller.go`: Propagate `OPERATOR_NAMESPACE` to sub-controllers. + +### Handle Empty WATCH_NAMESPACE for Cluster-Wide Watching + +If `WATCH_NAMESPACE` is set to empty (to watch all namespaces), the controller-runtime manager's cache config must be updated to omit `DefaultNamespaces` instead of inserting an empty-string key. +The `getWatchNamespace()` function in `cmd/main.go` would also need to treat empty/unset as valid rather than logging an error. +DPA singleton enforcement scope would need a design decision: one DPA globally or one per namespace (multi-tenant Velero). From b7d5af099bb2d8a25345d6e7294bb75d11b1e366 Mon Sep 17 00:00:00 2001 From: Joseph Date: Tue, 11 Aug 2026 12:03:52 -0700 Subject: [PATCH 04/13] docs: merge CSV change into build infra phase MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 1 (CSV change) cannot be merged alone — it would modify the existing OwnNamespace CSV. The installModes flip and WATCH_NAMESPACE source change must live inside the AllNamespaces kustomize overlay, which is part of the build infrastructure work. Merges old Phases 1+2 into a single Phase 1. Renumbers to 4 phases. Co-Authored-By: Claude Opus 4.6 (1M context) --- .../allnamespaces-install-mode_design.md | 76 +++++++------------ 1 file changed, 28 insertions(+), 48 deletions(-) diff --git a/docs/design/allnamespaces-install-mode_design.md b/docs/design/allnamespaces-install-mode_design.md index 89e3409fbd8..a75216c0cc7 100644 --- a/docs/design/allnamespaces-install-mode_design.md +++ b/docs/design/allnamespaces-install-mode_design.md @@ -40,13 +40,12 @@ A `ClusterWideClient` (uncached) already exists for cross-namespace DPA validati ## High-Level Design -The work is split into five phases, each independently mergeable: +The work is split into four phases, each independently mergeable: -1. **Enable AllNamespaces install mode in the CSV** (the core change: flip installModes, source `WATCH_NAMESPACE` from `metadata.namespace`). -2. **Build infrastructure for two bundle variants** (kustomize overlays, Makefile targets, two CSVs). Delivery mechanism (channels vs packages) is decided in this phase. -3. **E2E test infrastructure** for both install modes. -4. **CI/Prow integration** with AllNamespaces-specific jobs. -5. **Migration documentation** for customers. +1. **Build infrastructure for two bundle variants and enable AllNamespaces** (kustomize overlays that produce two CSVs — one with `OwnNamespace`, one with `AllNamespaces` and `WATCH_NAMESPACE` sourced from `metadata.namespace`). Delivery mechanism (channels vs packages) is decided in this phase. The existing OwnNamespace CSV is not modified. +2. **E2E test infrastructure** for both install modes. +3. **CI/Prow integration** with AllNamespaces-specific jobs. +4. **Migration documentation** for customers. ## Detailed Design @@ -67,9 +66,13 @@ The work is split into five phases, each independently mergeable: | RBAC | Already cluster-scoped (ClusterRoles) | `config/rbac/role.yaml`, CSV `clusterPermissions` | | OLM channels | Single channel: `dev` (release branches use e.g. `oadp-1.5`) | `Makefile` (lines 23, 33), `bundle/metadata/annotations.yaml` | -### Phase 1: Enable AllNamespaces Install Mode in the CSV +### Phase 1: Build Infrastructure for Two Bundle Variants and Enable AllNamespaces + +Produce two distinct OLM bundles from one codebase. +A single CSV cannot have both `OwnNamespace` and `AllNamespaces` enabled simultaneously, so two CSVs are required. +The CSV changes (installModes flip, `WATCH_NAMESPACE` source change) are applied only to the AllNamespaces variant via a kustomize overlay — the existing OwnNamespace CSV is never modified. +The delivery mechanism for the second CSV is also decided in this phase. -The core change. The AllNamespaces CSV differs from the OwnNamespace CSV in exactly two ways: 1. `installModes` — only `AllNamespaces: true` (instead of only `OwnNamespace: true`). @@ -78,43 +81,21 @@ The AllNamespaces CSV differs from the OwnNamespace CSV in exactly two ways: With `WATCH_NAMESPACE` sourced from `metadata.namespace`, the operator always resolves to the pod's own namespace regardless of what `olm.targetNamespaces` contains (which would be empty under a global OperatorGroup). No Go code changes are needed. The operator binary is identical for both CSVs. -#### Changes - -| File | Change | -|---|---| -| `config/manifests/bases/oadp-operator.clusterserviceversion.yaml` (lines 467-475) | Create an AllNamespaces variant with only `AllNamespaces: true` in `installModes` | -| CSV deploymentSpec `WATCH_NAMESPACE` env var | Change source from `metadata.annotations['olm.targetNamespaces']` to `metadata.namespace` (downward API) in the AllNamespaces variant | -| `make bundle` | Regenerate bundle to verify the OwnNamespace CSV is unchanged | - #### What does NOT change - No Go code changes. `cmd/main.go`, controllers, and all runtime behavior are untouched. +- The existing OwnNamespace CSV, bundle, and channel are not modified. - `WATCH_NAMESPACE` always resolves to a non-empty namespace name (the pod's own namespace). - Cache scoping, PSA labeling, STS flow, CLI/VMDP downloads, sub-controller propagation all continue to work exactly as today. - RBAC is already cluster-scoped and does not need modification. -#### Validation - -- `make test` passes (no code changes). -- Manual OLM install with AllNamespaces OperatorGroup: operator starts, `WATCH_NAMESPACE` = pod namespace, DPA reconciles normally. - -#### Risk: Low - -No runtime behavioral change. The only change is in the CSV metadata and env var source. - -### Phase 2: Build Infrastructure for Two Bundle Variants - -Produce two distinct OLM bundles from one codebase. -A single CSV cannot have both `OwnNamespace` and `AllNamespaces` enabled simultaneously, so two CSVs are required. -The delivery mechanism for the second CSV is decided in this phase. - -#### Common Build Changes (both options) +#### Common Build Changes (both delivery options) Regardless of delivery option, the following build changes are needed to produce two bundle variants from one codebase: | Area | Change | |---|---| -| **CSV base template** | Keep `config/manifests/bases/oadp-operator.clusterserviceversion.yaml` as the shared base | +| **CSV base template** | Keep `config/manifests/bases/oadp-operator.clusterserviceversion.yaml` as the shared base, unchanged | | **Kustomize overlays** | Create `config/manifests/overlays/ownnamespace/` and `config/manifests/overlays/allnamespaces/` with patches for `installModes` and `WATCH_NAMESPACE` env var source | | **AllNamespaces overlay** | Patches: (1) `installModes` set to only `AllNamespaces: true`, (2) `WATCH_NAMESPACE` source changed from `olm.targetNamespaces` to `metadata.namespace` | | **OwnNamespace overlay** | Patches: keeps current behavior (only `OwnNamespace: true`, `WATCH_NAMESPACE` from `olm.targetNamespaces`). Identity transform initially. | @@ -204,17 +185,17 @@ The choice between Option A and Option B should consider: Build plumbing only, no runtime impact. -### Phase 3: E2E Test Infrastructure +### Phase 2: E2E Test Infrastructure Tests need to validate both install modes. -The test infrastructure changes depend on the delivery option chosen in Phase 2. +The test infrastructure changes depend on the delivery option chosen in Phase 1. #### Changes | Area | Change | |---|---| -| `Makefile` `deploy-olm` (line 634-642) | Parameterize OperatorGroup creation: when `INSTALL_MODE=AllNamespaces`, create OperatorGroup with empty `spec` (no `targetNamespaces`). Default keeps current behavior. | -| `Makefile` | New target: `deploy-olm-allnamespaces`. For Option A (channels): sets `INSTALL_MODE=AllNamespaces` and `DEFAULT_CHANNEL=dev-allnamespaces`. For Option B (packages): sets `INSTALL_MODE=AllNamespaces` and overrides `CATALOG_SOURCE_NAME` and subscription package name. | +| `Makefile` `deploy-olm` (line 634-642) | Parameterize OperatorGroup creation: when `INSTALL_MODE=AllNamespaces`, create OperatorGroup with empty `spec` (no `targetNamespaces`). Default (`OwnNamespace`) keeps current behavior. | +| `Makefile` | New target: `deploy-olm-allnamespaces`. For Option A (channels): sets `INSTALL_MODE=AllNamespaces` and `DEFAULT_CHANNEL=dev-allnamespaces`. For Option B (packages): sets `INSTALL_MODE=AllNamespaces` and overrides subscription package name. | | `tests/e2e/upgrade_suite_test.go` (lines 31-50) | Parameterize OperatorGroup creation to support both modes based on a test flag | | E2E test scenarios | Add: install AllNamespaces, create DPA in operator namespace, verify Velero deploys. Verify `WATCH_NAMESPACE` resolves to operator namespace. Verify singleton enforcement. Verify sub-controller namespace config. | | Migration test (Option A only) | Test switching channel on an existing Subscription and swapping the OperatorGroup to verify the documented migration path works end-to-end. | @@ -227,7 +208,7 @@ Full e2e suite passes with both `make deploy-olm` (OwnNamespace) and `make deplo Test infrastructure changes are additive. -### Phase 4: CI/Prow Integration +### Phase 3: CI/Prow Integration AllNamespaces mode needs CI coverage. @@ -247,11 +228,11 @@ CI jobs pass on a test PR. Additive CI config in a separate repo. -### Phase 5: Migration Documentation +### Phase 4: Migration Documentation Customers migrating from OwnNamespace to AllNamespaces need clear, tested manual steps. OLM does not automate the OperatorGroup swap. -The migration path differs depending on the delivery option chosen in Phase 2. +The migration path differs depending on the delivery option chosen in Phase 1. Note: after migration, the operator's runtime behavior is identical. `WATCH_NAMESPACE` is sourced from `metadata.namespace` in the AllNamespaces CSV, so it always resolves to the operator's own namespace. @@ -446,7 +427,7 @@ A security review of the velero SA permissions is still recommended as a general ## Compatibility - Existing `OwnNamespace` installations are unaffected. The OwnNamespace CSV continues to exist and receive updates regardless of delivery option. -- Migration from `OwnNamespace` to `AllNamespaces` is a manual process documented in Phase 5. The exact steps depend on the delivery option chosen in Phase 2. +- Migration from `OwnNamespace` to `AllNamespaces` is a manual process documented in Phase 4. The exact steps depend on the delivery option chosen in Phase 1. - No Go code changes are required. The operator binary is identical for both CSVs. - Runtime behavior is identical in both modes: `WATCH_NAMESPACE` always resolves to the operator's own namespace. @@ -454,18 +435,17 @@ A security review of the velero SA permissions is still recommended as a general | Phase | Scope | Risk | Depends on | Parallelizable | |---|---|---|---|---| -| 1. Enable AllNamespaces in CSV | CSV metadata | Low | None | No (foundation) | -| 2. Two bundle variants | Build infrastructure | Medium | Phase 1 | No | -| 3. E2E test infrastructure | Test infrastructure | Medium | Phase 2 | No | -| 4. CI/Prow integration | CI config | Low | Phase 3 | Yes (with Phase 5) | -| 5. Migration documentation | Docs | Low | Phase 2 | Yes (with Phase 4) | +| 1. Bundle variants + AllNamespaces CSV | Build infrastructure + CSV metadata | Medium | None | No (foundation) | +| 2. E2E test infrastructure | Test infrastructure | Medium | Phase 1 | No | +| 3. CI/Prow integration | CI config | Low | Phase 2 | Yes (with Phase 4) | +| 4. Migration documentation | Docs | Low | Phase 1 | Yes (with Phase 3) | ## Open Issues 1. **Delivery option**: Two channels in the same package (Option A) or two separate OLM packages (Option B)? -This decision must be made in Phase 2 and affects Phases 3, 4, and 5. +This decision must be made in Phase 1 and affects Phases 2, 3, and 4. Key factors: downstream release pipeline compatibility (Konflux/ART), migration UX priority, and long-term deprecation strategy for OwnNamespace. -See the Decision Criteria section in Phase 2 for details. +See the Decision Criteria section in Phase 1 for details. 2. **Channel naming convention (Option A only)**: If channels are chosen, proposed convention is `-allnamespaces` (e.g., `dev-allnamespaces`, `stable-1.6-allnamespaces`). Open to shorter alternatives if the convention is too verbose. From ea522e83996917a67c34b85d28c08949da79fd1a Mon Sep 17 00:00:00 2001 From: Joseph Date: Tue, 11 Aug 2026 12:27:55 -0700 Subject: [PATCH 05/13] docs: commit to channel approach, add upgrade path rationale Removes the two-packages option. Documents why both channels must coexist for at least one release: without it, existing OwnNamespace customers hit an OLM deadlock where they can't upgrade (new CSV doesn't support their OperatorGroup) and can't change the OperatorGroup (current CSV doesn't support AllNamespaces either). Co-Authored-By: Claude Opus 4.6 (1M context) --- .../allnamespaces-install-mode_design.md | 253 ++++-------------- 1 file changed, 59 insertions(+), 194 deletions(-) diff --git a/docs/design/allnamespaces-install-mode_design.md b/docs/design/allnamespaces-install-mode_design.md index a75216c0cc7..17e0fd06014 100644 --- a/docs/design/allnamespaces-install-mode_design.md +++ b/docs/design/allnamespaces-install-mode_design.md @@ -3,9 +3,9 @@ ## Abstract OADP operator currently supports only `OwnNamespace` install mode via OLM. -This proposal describes a phased plan to add `AllNamespaces` install mode support, delivered via a second CSV to allow customers to migrate at their own pace. +This proposal describes a phased plan to add `AllNamespaces` install mode support, delivered as a separate channel within the existing OLM package. The operator's runtime behavior remains identical: it watches only the namespace it is deployed in, regardless of install mode. -Two delivery options are presented for the second CSV: a separate channel within the existing OLM package, or a separate OLM package with its own catalog entry. +Both channels must coexist for at least one release cycle to provide a safe upgrade path for existing customers. ## Background @@ -24,9 +24,22 @@ RBAC is already cluster-scoped (ClusterRoles and ClusterRoleBindings), so no fun No webhooks are currently enabled. A `ClusterWideClient` (uncached) already exists for cross-namespace DPA validation. +### Why Two Channels Are Required + +A single CSV cannot have both `OwnNamespace` and `AllNamespaces` enabled simultaneously. +More critically, OLM validates that a CSV supports the OperatorGroup's install mode before allowing an install or upgrade. + +If a future release shipped only an AllNamespaces CSV, existing customers with a namespaced OperatorGroup (`targetNamespaces: [openshift-adp]`) would be **blocked from upgrading** — OLM would reject the new CSV because it doesn't support `OwnNamespace`. +The customer cannot change the OperatorGroup first, because their current CSV also doesn't support `AllNamespaces`, creating a **deadlock**. + +Both channels must coexist for at least one release cycle so customers can: +1. Upgrade to the version that offers both channels (staying on OwnNamespace). +2. Migrate to AllNamespaces at their own pace using the documented migration steps. +3. A future release can then deprecate the OwnNamespace channel after the migration window closes. + ## Goals -- Enable `AllNamespaces` install mode via a second CSV, delivered alongside the existing `OwnNamespace` CSV. +- Enable `AllNamespaces` install mode via a second channel in the existing OLM package. - Keep runtime behavior identical to today: the operator watches only the namespace it is deployed in. - Provide a documented migration path for customers moving from `OwnNamespace` to `AllNamespaces`. - Maintain full backward compatibility for existing `OwnNamespace` installations. @@ -35,14 +48,14 @@ A `ClusterWideClient` (uncached) already exists for cross-namespace DPA validati - Actually watching all namespaces. The operator continues to watch only its own namespace. Cluster-wide watching can be enabled in the future by setting `WATCH_NAMESPACE` to empty, but that is a separate enhancement requiring additional work (see Future Enhancements). - Multi-tenant Velero (one DPA per namespace). Global DPA singleton enforcement will be maintained. -- Removing `OwnNamespace` support. Both modes will coexist until a future deprecation decision. +- Removing `OwnNamespace` support. Both channels will coexist for at least one release cycle. Deprecation of OwnNamespace is a separate future decision. - `SingleNamespace` or `MultiNamespace` install mode support. ## High-Level Design The work is split into four phases, each independently mergeable: -1. **Build infrastructure for two bundle variants and enable AllNamespaces** (kustomize overlays that produce two CSVs — one with `OwnNamespace`, one with `AllNamespaces` and `WATCH_NAMESPACE` sourced from `metadata.namespace`). Delivery mechanism (channels vs packages) is decided in this phase. The existing OwnNamespace CSV is not modified. +1. **Build infrastructure for two bundle variants and enable AllNamespaces** (kustomize overlays that produce two CSVs in separate channels within the same OLM package). The existing OwnNamespace CSV is not modified. 2. **E2E test infrastructure** for both install modes. 3. **CI/Prow integration** with AllNamespaces-specific jobs. 4. **Migration documentation** for customers. @@ -68,10 +81,8 @@ The work is split into four phases, each independently mergeable: ### Phase 1: Build Infrastructure for Two Bundle Variants and Enable AllNamespaces -Produce two distinct OLM bundles from one codebase. -A single CSV cannot have both `OwnNamespace` and `AllNamespaces` enabled simultaneously, so two CSVs are required. +Produce two distinct OLM bundles from one codebase, shipped as separate channels in the same `oadp-operator` package. The CSV changes (installModes flip, `WATCH_NAMESPACE` source change) are applied only to the AllNamespaces variant via a kustomize overlay — the existing OwnNamespace CSV is never modified. -The delivery mechanism for the second CSV is also decided in this phase. The AllNamespaces CSV differs from the OwnNamespace CSV in exactly two ways: @@ -81,6 +92,19 @@ The AllNamespaces CSV differs from the OwnNamespace CSV in exactly two ways: With `WATCH_NAMESPACE` sourced from `metadata.namespace`, the operator always resolves to the pod's own namespace regardless of what `olm.targetNamespaces` contains (which would be empty under a global OperatorGroup). No Go code changes are needed. The operator binary is identical for both CSVs. +#### Catalog Structure + +``` +Catalog: oadp-operator-catalog +└── Package: oadp-operator + ├── Channel: dev ← OwnNamespace CSV + │ └── oadp-operator.v99.0.0 + └── Channel: dev-allnamespaces ← AllNamespaces CSV + └── oadp-operator.v99.0.0-allns +``` + +Release branches follow the same pattern: `stable-1.7` (OwnNamespace) and `stable-1.7-allnamespaces` (AllNamespaces). + #### What does NOT change - No Go code changes. `cmd/main.go`, controllers, and all runtime behavior are untouched. @@ -89,9 +113,7 @@ No Go code changes are needed. The operator binary is identical for both CSVs. - Cache scoping, PSA labeling, STS flow, CLI/VMDP downloads, sub-controller propagation all continue to work exactly as today. - RBAC is already cluster-scoped and does not need modification. -#### Common Build Changes (both delivery options) - -Regardless of delivery option, the following build changes are needed to produce two bundle variants from one codebase: +#### Changes | Area | Change | |---|---| @@ -100,86 +122,15 @@ Regardless of delivery option, the following build changes are needed to produce | **AllNamespaces overlay** | Patches: (1) `installModes` set to only `AllNamespaces: true`, (2) `WATCH_NAMESPACE` source changed from `olm.targetNamespaces` to `metadata.namespace` | | **OwnNamespace overlay** | Patches: keeps current behavior (only `OwnNamespace: true`, `WATCH_NAMESPACE` from `olm.targetNamespaces`). Identity transform initially. | | **Makefile** | Add `INSTALL_MODE ?= OwnNamespace`. New targets: `bundle-allnamespaces`, `bundle-build-allnamespaces` | - -#### Option A: Two Channels in the Same Package - -The AllNamespaces CSV is shipped as a separate channel within the existing `oadp-operator` package. -Both channels live in the same catalog and share the same package identity. - -``` -Catalog: oadp-operator-catalog -└── Package: oadp-operator - ├── Channel: dev ← OwnNamespace CSV - │ └── oadp-operator.v99.0.0 - └── Channel: dev-allnamespaces ← AllNamespaces CSV - └── oadp-operator.v99.0.0-allns -``` - -Release branches follow the same pattern: `stable-1.6` (OwnNamespace) and `stable-1.6-allnamespaces` (AllNamespaces). - -**Additional changes for Option A:** - -| Area | Change | -|---|---| | **Catalog build** | Parameterize `Dockerfile.catalog` and `catalog-build` target to produce a single catalog with two channels: existing channel (OwnNamespace bundle) and new `-allnamespaces` channel (AllNamespaces bundle) | | **CSV naming** | AllNamespaces CSV uses a distinct version suffix: `oadp-operator.v99.0.0-allns` vs `oadp-operator.v99.0.0` | -| **Channel naming** | Convention: `-allnamespaces` (e.g., `dev-allnamespaces`, `stable-1.6-allnamespaces`) | - -**Trade-offs:** - -| Pro | Con | -|---|---| -| Single package in OperatorHub; cleaner customer experience | Every release branch must produce two bundles and two channel entries | -| Migration via Subscription channel change (no uninstall/reinstall of the package) | Channel names are overloaded (channels typically mean release stability, not install topology) | -| Single catalog image to build and publish | Customer must still manually swap the OperatorGroup after changing channel | - -#### Option B: Two Separate OLM Packages - -The AllNamespaces CSV is shipped as a separate OLM package with its own catalog entry. -Each package has its own identity and upgrade graph. - -``` -Catalog: oadp-operator-catalog -├── Package: oadp-operator ← OwnNamespace CSV -│ └── Channel: dev -│ └── oadp-operator.v99.0.0 -└── Package: oadp-operator-allnamespaces ← AllNamespaces CSV - └── Channel: dev - └── oadp-operator-allnamespaces.v99.0.0 -``` - -**Additional changes for Option B:** - -| Area | Change | -|---|---| -| **Catalog build** | Produce a single catalog containing two packages, each with its own channel(s). Or produce two separate catalogs (one per package). | -| **CSV naming** | AllNamespaces CSV uses a distinct package name: `oadp-operator-allnamespaces` | -| **Bundle metadata** | AllNamespaces bundle has its own `annotations.yaml` with `operators.operatorframework.io.bundle.package.v1: oadp-operator-allnamespaces` | -| **Makefile** | Additional target: `catalog-build-allnamespaces` if producing separate catalogs | - -**Trade-offs:** - -| Pro | Con | -|---|---| -| Clean separation; no channel naming overload | Two tiles in OperatorHub; customers must know which to pick | -| Each package has its own independent upgrade graph | Migration requires uninstalling the old package and installing the new one | -| Channel names stay semantic (stable, dev, etc.) | Doubles catalog/release artifacts per version | -| Simpler catalog structure per package | Customer loses the Subscription during migration (must re-create) | - -#### Decision Criteria - -The choice between Option A and Option B should consider: - -1. **Downstream release pipeline**: Does Konflux/ART handle two channels in one package easily, or is a second package simpler to wire up? -2. **Migration UX priority**: Is avoiding uninstall/reinstall (Option A) worth the channel naming complexity? -3. **Long-term intent**: If OwnNamespace will eventually be deprecated, Option A lets you sunset a channel; Option B requires deprecating an entire package. -4. **OperatorHub presentation**: One tile with a channel picker (Option A) vs two tiles (Option B). +| **Channel naming** | Convention: `-allnamespaces` (e.g., `dev-allnamespaces`, `stable-1.7-allnamespaces`) | #### Validation - `make bundle` and `make bundle-allnamespaces` both produce valid bundles. - `opm validate` passes on both bundles. -- Catalog builds and serves correctly with the chosen delivery structure. +- Catalog with two channels builds and serves correctly. #### Risk: Medium @@ -188,17 +139,16 @@ Build plumbing only, no runtime impact. ### Phase 2: E2E Test Infrastructure Tests need to validate both install modes. -The test infrastructure changes depend on the delivery option chosen in Phase 1. #### Changes | Area | Change | |---|---| | `Makefile` `deploy-olm` (line 634-642) | Parameterize OperatorGroup creation: when `INSTALL_MODE=AllNamespaces`, create OperatorGroup with empty `spec` (no `targetNamespaces`). Default (`OwnNamespace`) keeps current behavior. | -| `Makefile` | New target: `deploy-olm-allnamespaces`. For Option A (channels): sets `INSTALL_MODE=AllNamespaces` and `DEFAULT_CHANNEL=dev-allnamespaces`. For Option B (packages): sets `INSTALL_MODE=AllNamespaces` and overrides subscription package name. | +| `Makefile` | New target: `deploy-olm-allnamespaces` that sets `INSTALL_MODE=AllNamespaces` and `DEFAULT_CHANNEL=dev-allnamespaces` | | `tests/e2e/upgrade_suite_test.go` (lines 31-50) | Parameterize OperatorGroup creation to support both modes based on a test flag | | E2E test scenarios | Add: install AllNamespaces, create DPA in operator namespace, verify Velero deploys. Verify `WATCH_NAMESPACE` resolves to operator namespace. Verify singleton enforcement. Verify sub-controller namespace config. | -| Migration test (Option A only) | Test switching channel on an existing Subscription and swapping the OperatorGroup to verify the documented migration path works end-to-end. | +| Migration test | Test switching channel on an existing Subscription and swapping the OperatorGroup to verify the documented migration path works end-to-end. | #### Validation @@ -232,19 +182,18 @@ Additive CI config in a separate repo. Customers migrating from OwnNamespace to AllNamespaces need clear, tested manual steps. OLM does not automate the OperatorGroup swap. -The migration path differs depending on the delivery option chosen in Phase 1. Note: after migration, the operator's runtime behavior is identical. `WATCH_NAMESPACE` is sourced from `metadata.namespace` in the AllNamespaces CSV, so it always resolves to the operator's own namespace. The operator does not start watching all namespaces. -#### Prerequisites (both options) +#### Prerequisites - Cluster admin access. - No active backups or restores in progress. -- Current OADP version supports the AllNamespaces CSV (minimum version TBD). +- Current OADP version supports the AllNamespaces channel (minimum version TBD). -#### Migration Path for Option A (Channels) +#### Migration Steps **Step 1: Verify current state** @@ -306,107 +255,17 @@ oc get dpa -n openshift-adp -o jsonpath='{.items[0].status.conditions}' oc get deployment -n openshift-adp -l app.kubernetes.io/name=velero ``` -**Rollback (Option A):** +#### Rollback 1. Delete the global OperatorGroup. 2. Create the namespaced OperatorGroup with `targetNamespaces: [openshift-adp]`. 3. Switch the subscription channel back to the OwnNamespace channel. -#### Migration Path for Option B (Packages) - -**Step 1: Verify current state** - -```bash -oc get subscription oadp-operator -n openshift-adp -o yaml -oc get operatorgroup -n openshift-adp -o yaml -oc get dpa -n openshift-adp -``` - -**Step 2: Delete the existing subscription (keeps the DPA and Velero resources)** - -```bash -oc delete subscription oadp-operator -n openshift-adp -``` - -**Step 3: Delete the OwnNamespace CSV** - -```bash -CSV_NAME=$(oc get csv -n openshift-adp -o name | grep oadp-operator) -oc delete $CSV_NAME -n openshift-adp -``` - -The operator pod is removed. DPA and Velero resources remain. - -**Step 4: Delete the existing namespaced OperatorGroup** - -```bash -oc delete operatorgroup oadp-operator-group -n openshift-adp -``` - -**Step 5: Create a global OperatorGroup** - -```bash -cat <-allnamespaces` (e.g., `dev-allnamespaces`, `stable-1.6-allnamespaces`). +1. **Channel naming convention**: Proposed convention is `-allnamespaces` (e.g., `dev-allnamespaces`, `stable-1.7-allnamespaces`). Open to shorter alternatives if the convention is too verbose. -3. **Minimum version for migration**: Which OADP release will be the first to ship the AllNamespaces CSV? -This determines the migration documentation's version requirements. +2. **Minimum version for migration**: Which OADP release will be the first to ship the AllNamespaces channel? +This determines the migration documentation's version requirements and the minimum coexistence window before OwnNamespace can be deprecated. + +3. **OwnNamespace deprecation timeline**: How many release cycles should both channels coexist before the OwnNamespace channel is removed? +At minimum one cycle is required for a safe upgrade path. ## Future Enhancements From 1ba6a7a7459f9a2ab981293ad5933d01b4e33bbd Mon Sep 17 00:00:00 2001 From: Joseph Date: Tue, 11 Aug 2026 13:10:10 -0700 Subject: [PATCH 06/13] docs: add deploy+CI phase before test scenarios Phase 2 now adds deploy-olm-allnamespaces target (using operator-sdk run bundle --install-mode AllNamespaces) and a Prow job that runs the existing e2e suite against it. This gives immediate signal before writing any AllNamespaces-specific test code. Phase 3 becomes AllNamespaces-specific test scenarios (migration, upgrade parameterization). Adds Phase 5 for OwnNamespace deprecation planning. Updates current state table to reflect operator-sdk run bundle as the deploy mechanism. Co-Authored-By: Claude Opus 4.6 (1M context) --- .../allnamespaces-install-mode_design.md | 84 +++++++++++++------ 1 file changed, 57 insertions(+), 27 deletions(-) diff --git a/docs/design/allnamespaces-install-mode_design.md b/docs/design/allnamespaces-install-mode_design.md index 17e0fd06014..cfad8ed448d 100644 --- a/docs/design/allnamespaces-install-mode_design.md +++ b/docs/design/allnamespaces-install-mode_design.md @@ -3,8 +3,11 @@ ## Abstract OADP operator currently supports only `OwnNamespace` install mode via OLM. + This proposal describes a phased plan to add `AllNamespaces` install mode support, delivered as a separate channel within the existing OLM package. + The operator's runtime behavior remains identical: it watches only the namespace it is deployed in, regardless of install mode. + Both channels must coexist for at least one release cycle to provide a safe upgrade path for existing customers. ## Background @@ -13,20 +16,23 @@ The OADP operator is installed via OLM with a strict `OwnNamespace` install mode The CSV declares only `OwnNamespace: true` and all other modes are `supported: false`. At runtime, the `WATCH_NAMESPACE` environment variable controls which namespace the controller-runtime cache monitors. + In the OwnNamespace CSV, `WATCH_NAMESPACE` is sourced from the `olm.targetNamespaces` annotation, which OLM sets to the operator's own namespace. + In the non-OLM deployment (`config/manager/manager.yaml`), `WATCH_NAMESPACE` is sourced from `metadata.namespace` (the pod's own namespace via downward API). The key insight is that `AllNamespaces` is an OLM install mode — it controls what OperatorGroup the CSV is compatible with, not what the operator actually watches at runtime. The AllNamespaces CSV can source `WATCH_NAMESPACE` from `metadata.namespace` (just like the non-OLM deployment does today) instead of from `olm.targetNamespaces`. + This means the operator keeps watching only its own namespace by default, even when installed via a global OperatorGroup. RBAC is already cluster-scoped (ClusterRoles and ClusterRoleBindings), so no fundamental RBAC changes are required. -No webhooks are currently enabled. +No webhooks are currently enabled, but this opens the door to having conversion webhooks, allowing us to have new CRD versions without breaking changes. A `ClusterWideClient` (uncached) already exists for cross-namespace DPA validation. ### Why Two Channels Are Required -A single CSV cannot have both `OwnNamespace` and `AllNamespaces` enabled simultaneously. +A single CSV **cannot** have both `OwnNamespace` and `AllNamespaces` enabled simultaneously. More critically, OLM validates that a CSV supports the OperatorGroup's install mode before allowing an install or upgrade. If a future release shipped only an AllNamespaces CSV, existing customers with a namespaced OperatorGroup (`targetNamespaces: [openshift-adp]`) would be **blocked from upgrading** — OLM would reject the new CSV because it doesn't support `OwnNamespace`. @@ -53,12 +59,13 @@ Both channels must coexist for at least one release cycle so customers can: ## High-Level Design -The work is split into four phases, each independently mergeable: +The work is split into five phases, each independently mergeable: 1. **Build infrastructure for two bundle variants and enable AllNamespaces** (kustomize overlays that produce two CSVs in separate channels within the same OLM package). The existing OwnNamespace CSV is not modified. -2. **E2E test infrastructure** for both install modes. -3. **CI/Prow integration** with AllNamespaces-specific jobs. +2. **Deploy and CI for AllNamespaces** — add `deploy-olm-allnamespaces` Makefile target using `operator-sdk run bundle --install-mode AllNamespaces` and a Prow CI job running the existing e2e suite against it. This gives immediate signal that the AllNamespaces bundle works end-to-end. +3. **AllNamespaces-specific e2e test scenarios** — migration test (channel switch + OperatorGroup swap), upgrade test parameterization, and additional validation. 4. **Migration documentation** for customers. +5. **OwnNamespace deprecation plan** — timeline and communication for eventually sunsetting the OwnNamespace channel. ## Detailed Design @@ -74,7 +81,8 @@ The work is split into four phases, each independently mergeable: | CLI/VMDP downloads | Skipped if `watchNamespace` is empty | `cmd/main.go` (lines 305-306) | | STS flow | Reads `WATCH_NAMESPACE` as install namespace | `pkg/credentials/stsflow/stsflow.go` (line 115) | | Sub-controllers | All receive `WATCH_NAMESPACE` = own namespace | `nonadmin_controller.go` (line 176), `kubevirt_datamover_controller.go` (line 152), `vmfilerestore_controller.go` (line 184) | -| E2E OperatorGroup | Always `targetNamespaces: [namespace]` | `Makefile` (line 639), `upgrade_suite_test.go` (lines 31-50) | +| E2E deploy | `operator-sdk run bundle` (implicitly OwnNamespace) | `Makefile` (line 459) | +| E2E OperatorGroup | Always `targetNamespaces: [namespace]` | `upgrade_suite_test.go` (lines 31-50) | | Cross-NS validation | Uses uncached `ClusterWideClient` | `cmd/main.go` (lines 260-270), `validator.go` (line 144) | | RBAC | Already cluster-scoped (ClusterRoles) | `config/rbac/role.yaml`, CSV `clusterPermissions` | | OLM channels | Single channel: `dev` (release branches use e.g. `oadp-1.5`) | `Makefile` (lines 23, 33), `bundle/metadata/annotations.yaml` | @@ -122,7 +130,7 @@ Release branches follow the same pattern: `stable-1.7` (OwnNamespace) and `stabl | **AllNamespaces overlay** | Patches: (1) `installModes` set to only `AllNamespaces: true`, (2) `WATCH_NAMESPACE` source changed from `olm.targetNamespaces` to `metadata.namespace` | | **OwnNamespace overlay** | Patches: keeps current behavior (only `OwnNamespace: true`, `WATCH_NAMESPACE` from `olm.targetNamespaces`). Identity transform initially. | | **Makefile** | Add `INSTALL_MODE ?= OwnNamespace`. New targets: `bundle-allnamespaces`, `bundle-build-allnamespaces` | -| **Catalog build** | Parameterize `Dockerfile.catalog` and `catalog-build` target to produce a single catalog with two channels: existing channel (OwnNamespace bundle) and new `-allnamespaces` channel (AllNamespaces bundle) | +| **Catalog build** | Extend `catalog-build` to produce a single catalog with two channels: existing channel (OwnNamespace bundle) and new `-allnamespaces` channel (AllNamespaces bundle). Adds a second `opm render` + `olm.channel` entry to the FBC output. | | **CSV naming** | AllNamespaces CSV uses a distinct version suffix: `oadp-operator.v99.0.0-allns` vs `oadp-operator.v99.0.0` | | **Channel naming** | Convention: `-allnamespaces` (e.g., `dev-allnamespaces`, `stable-1.7-allnamespaces`) | @@ -136,47 +144,57 @@ Release branches follow the same pattern: `stable-1.7` (OwnNamespace) and `stabl Build plumbing only, no runtime impact. -### Phase 2: E2E Test Infrastructure +### Phase 2: Deploy and CI for AllNamespaces + +Add the ability to deploy and test with AllNamespaces mode, then wire up CI to run the existing e2e suite against it. +This gives immediate signal that the AllNamespaces bundle works end-to-end before writing any new test code. -Tests need to validate both install modes. +`operator-sdk run bundle` already supports `--install-mode` as a flag. +Today the `deploy-olm` target does not pass this flag, so it defaults to OwnNamespace. #### Changes | Area | Change | |---|---| -| `Makefile` `deploy-olm` (line 634-642) | Parameterize OperatorGroup creation: when `INSTALL_MODE=AllNamespaces`, create OperatorGroup with empty `spec` (no `targetNamespaces`). Default (`OwnNamespace`) keeps current behavior. | -| `Makefile` | New target: `deploy-olm-allnamespaces` that sets `INSTALL_MODE=AllNamespaces` and `DEFAULT_CHANNEL=dev-allnamespaces` | -| `tests/e2e/upgrade_suite_test.go` (lines 31-50) | Parameterize OperatorGroup creation to support both modes based on a test flag | -| E2E test scenarios | Add: install AllNamespaces, create DPA in operator namespace, verify Velero deploys. Verify `WATCH_NAMESPACE` resolves to operator namespace. Verify singleton enforcement. Verify sub-controller namespace config. | -| Migration test | Test switching channel on an existing Subscription and swapping the OperatorGroup to verify the documented migration path works end-to-end. | +| **Makefile** | New target `deploy-olm-allnamespaces` that builds the AllNamespaces bundle and runs `operator-sdk run bundle --install-mode AllNamespaces --security-context-config restricted $(THIS_BUNDLE_IMAGE_ALLNS) --namespace $(OADP_TEST_NAMESPACE)` | +| **`openshift/release` config** | Add new presubmit job (e.g., `e2e-aws-allnamespaces`) that runs `make deploy-olm-allnamespaces` then `make test-e2e`. Runs the existing e2e suite unchanged. | +| **Periodic jobs** | Add AllNamespaces variant for nightly runs | + +#### What this validates + +- OLM accepts the AllNamespaces CSV with a global OperatorGroup. +- `WATCH_NAMESPACE` resolves to the pod's namespace (not empty). +- The full e2e suite passes with identical behavior: DPA creation, Velero deployment, backup/restore operations, sub-controllers, credential management. +- Any failures at this stage reveal real incompatibilities rather than test infrastructure gaps. #### Validation -Full e2e suite passes with both `make deploy-olm` (OwnNamespace) and `make deploy-olm-allnamespaces`. +Full existing e2e suite passes with `make deploy-olm-allnamespaces`. -#### Risk: Medium +#### Risk: Low -Test infrastructure changes are additive. +One new Makefile target and one new CI job. No code changes. The existing e2e suite is the test. -### Phase 3: CI/Prow Integration +### Phase 3: AllNamespaces-Specific E2E Test Scenarios -AllNamespaces mode needs CI coverage. +With CI running the existing suite against AllNamespaces (Phase 2), this phase adds test scenarios specific to the AllNamespaces install mode. #### Changes | Area | Change | |---|---| -| `openshift/release` config | Add new presubmit job(s) running e2e tests with `INSTALL_MODE=AllNamespaces` | -| Job naming | e.g., `e2e-aws-allnamespaces` alongside existing `e2e-aws` | -| Periodic jobs | Add AllNamespaces variants for nightly runs | +| `tests/e2e/upgrade_suite_test.go` (lines 31-50) | Parameterize OperatorGroup creation to support both modes based on a test flag or env var | +| Migration test | Test the documented migration path end-to-end: install OwnNamespace, switch channel, swap OperatorGroup, verify operator continues functioning | +| Upgrade test | Verify upgrading from prior version on OwnNamespace channel still works (no regression in existing upgrade path) | +| Singleton enforcement | Verify global DPA singleton is enforced in AllNamespaces mode via the existing `ClusterWideClient` validator | #### Validation -CI jobs pass on a test PR. +All new test scenarios pass. Existing OwnNamespace e2e suite continues to pass (no regressions). -#### Risk: Low +#### Risk: Medium -Additive CI config in a separate repo. +Test infrastructure changes are additive. ### Phase 4: Migration Documentation @@ -267,6 +285,17 @@ Brief operator unavailability during the OperatorGroup swap (seconds to approxim No impact on existing backups at rest. In-flight backups or restores should be completed before migration. +### Phase 5: OwnNamespace Deprecation Plan + +Once both channels have coexisted for at least one release cycle and customers have had a migration window: + +- Announce deprecation of the OwnNamespace channel with a timeline. +- Add a deprecation warning to the OwnNamespace CSV (via `olm.deprecated` annotation or operator log message). +- After the deprecation window, stop publishing new versions to the OwnNamespace channel. +- The final OwnNamespace CSV remains installable but receives no further updates. + +The timeline and communication plan are TBD and depend on customer adoption metrics. + ## Alternatives Considered ### Single CSV with both install modes enabled @@ -303,9 +332,10 @@ A security review of the velero SA permissions is still recommended as a general | Phase | Scope | Risk | Depends on | Parallelizable | |---|---|---|---|---| | 1. Bundle variants + AllNamespaces channel | Build infrastructure + CSV metadata | Medium | None | No (foundation) | -| 2. E2E test infrastructure | Test infrastructure | Medium | Phase 1 | No | -| 3. CI/Prow integration | CI config | Low | Phase 2 | Yes (with Phase 4) | +| 2. Deploy + CI for AllNamespaces | Makefile target + Prow job | Low | Phase 1 | No | +| 3. AllNamespaces-specific e2e tests | Test scenarios | Medium | Phase 2 | Yes (with Phase 4) | | 4. Migration documentation | Docs | Low | Phase 1 | Yes (with Phase 3) | +| 5. OwnNamespace deprecation plan | Communication + timeline | Low | Phase 4 | No | ## Open Issues From 71fde88d401e4bb243149e98cc2fa5dcf290a41c Mon Sep 17 00:00:00 2001 From: Joseph Date: Wed, 12 Aug 2026 13:09:10 -0700 Subject: [PATCH 07/13] docs: address CodeRabbit review feedback - Add cross-channel update graph requirement to Phase 1: AllNamespaces CSV must include olm.skipRange covering OwnNamespace versions so OLM can resolve the channel switch - Fix security section: ClusterRoleBindings are cluster-scoped regardless of OperatorGroup mode, OwnNamespace does not contain velero SA perms - Add language identifier to catalog tree code fence (markdownlint MD040) Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/design/allnamespaces-install-mode_design.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/docs/design/allnamespaces-install-mode_design.md b/docs/design/allnamespaces-install-mode_design.md index cfad8ed448d..f2b73e6915f 100644 --- a/docs/design/allnamespaces-install-mode_design.md +++ b/docs/design/allnamespaces-install-mode_design.md @@ -102,7 +102,7 @@ No Go code changes are needed. The operator binary is identical for both CSVs. #### Catalog Structure -``` +```text Catalog: oadp-operator-catalog └── Package: oadp-operator ├── Channel: dev ← OwnNamespace CSV @@ -133,6 +133,7 @@ Release branches follow the same pattern: `stable-1.7` (OwnNamespace) and `stabl | **Catalog build** | Extend `catalog-build` to produce a single catalog with two channels: existing channel (OwnNamespace bundle) and new `-allnamespaces` channel (AllNamespaces bundle). Adds a second `opm render` + `olm.channel` entry to the FBC output. | | **CSV naming** | AllNamespaces CSV uses a distinct version suffix: `oadp-operator.v99.0.0-allns` vs `oadp-operator.v99.0.0` | | **Channel naming** | Convention: `-allnamespaces` (e.g., `dev-allnamespaces`, `stable-1.7-allnamespaces`) | +| **Cross-channel update graph** | The AllNamespaces channel's CSV must include an `olm.skipRange` that covers the OwnNamespace CSV version (e.g., `>=0.0.0 <99.0.0`), so OLM can resolve a valid update path when a customer switches channels. Without this, the channel switch fails silently — OLM cannot find an upgrade edge from the installed CSV to the new channel's head. | #### Validation @@ -313,10 +314,10 @@ Both approaches still require manual OperatorGroup changes, so the channel appro ## Security Considerations The `velero` ServiceAccount's ClusterRole grants near-cluster-admin permissions (`apiGroups: ['*'], resources: ['*']`). -In `OwnNamespace` mode this is contained by the OperatorGroup scope. -In `AllNamespaces` mode the RBAC is identical (already cluster-scoped), but the perception of blast radius changes. -Since the operator's runtime behavior is unchanged (still watches only its own namespace), the actual security posture does not change. -A security review of the velero SA permissions is still recommended as a general hygiene item. +These permissions are declared as `clusterPermissions` in the CSV and bound via `ClusterRoleBinding`, which is cluster-scoped regardless of the OperatorGroup's install mode. +The `OwnNamespace` OperatorGroup does not restrict or contain these permissions — the RBAC posture is identical in both `OwnNamespace` and `AllNamespaces` modes. +Since the operator's runtime behavior is also unchanged (still watches only its own namespace), the actual security posture does not change. +A security review of the velero SA permissions is recommended as a general hygiene item, independent of this install mode change. ## Compatibility From 9fa0a11b5222e7f530f38ad8a1d4d75db6009091 Mon Sep 17 00:00:00 2001 From: Joseph Date: Thu, 13 Aug 2026 13:45:36 -0700 Subject: [PATCH 08/13] docs: rewrite plan for single-CSV approach Cluster testing confirmed that both OwnNamespace and AllNamespaces install modes can be enabled in the same CSV. The two-channel approach is no longer needed. Three CSV metadata changes replace the entire multi-channel plan: 1. Flip AllNamespaces to supported: true 2. Source WATCH_NAMESPACE from metadata.namespace (not olm.targetNamespaces) 3. Add permissions entries for non-admin-controller and velero SAs Reduces plan from 5 phases to 3. Links to HackMD test log from OpenShift 4.22 cluster validation. Co-Authored-By: Claude Opus 4.6 (1M context) --- .../allnamespaces-install-mode_design.md | 319 +++++++++--------- 1 file changed, 153 insertions(+), 166 deletions(-) diff --git a/docs/design/allnamespaces-install-mode_design.md b/docs/design/allnamespaces-install-mode_design.md index f2b73e6915f..82cda8c3f7b 100644 --- a/docs/design/allnamespaces-install-mode_design.md +++ b/docs/design/allnamespaces-install-mode_design.md @@ -4,11 +4,10 @@ OADP operator currently supports only `OwnNamespace` install mode via OLM. -This proposal describes a phased plan to add `AllNamespaces` install mode support, delivered as a separate channel within the existing OLM package. +This proposal enables `AllNamespaces` install mode in the same CSV by enabling both install modes simultaneously and changing the `WATCH_NAMESPACE` source from `olm.targetNamespaces` to `metadata.namespace`. The operator's runtime behavior remains identical: it watches only the namespace it is deployed in, regardless of install mode. - -Both channels must coexist for at least one release cycle to provide a safe upgrade path for existing customers. +No Go code changes are required. ## Background @@ -21,51 +20,51 @@ In the OwnNamespace CSV, `WATCH_NAMESPACE` is sourced from the `olm.targetNamesp In the non-OLM deployment (`config/manager/manager.yaml`), `WATCH_NAMESPACE` is sourced from `metadata.namespace` (the pod's own namespace via downward API). -The key insight is that `AllNamespaces` is an OLM install mode — it controls what OperatorGroup the CSV is compatible with, not what the operator actually watches at runtime. -The AllNamespaces CSV can source `WATCH_NAMESPACE` from `metadata.namespace` (just like the non-OLM deployment does today) instead of from `olm.targetNamespaces`. +Two things blocked AllNamespaces support: -This means the operator keeps watching only its own namespace by default, even when installed via a global OperatorGroup. +1. **`WATCH_NAMESPACE` sourced from `olm.targetNamespaces`** — in AllNamespaces mode, `olm.targetNamespaces` is empty, which would break PSA labeling, STS credential flow, CLI/VMDP setup, and the controller-runtime cache configuration. +2. **Missing namespace-scoped `permissions` entries** — OLM requires every ServiceAccount declared in `clusterPermissions` to also have a `permissions` (namespace-scoped Role) entry when running in AllNamespaces mode. Without this, OLM refuses to create the ServiceAccounts and the CSV stays `Pending` with `"no owned roles found"`. -RBAC is already cluster-scoped (ClusterRoles and ClusterRoleBindings), so no fundamental RBAC changes are required. -No webhooks are currently enabled, but this opens the door to having conversion webhooks, allowing us to have new CRD versions without breaking changes. -A `ClusterWideClient` (uncached) already exists for cross-namespace DPA validation. +Both blockers are CSV metadata issues, not Go code issues. -### Why Two Channels Are Required +### Validated on Cluster -A single CSV **cannot** have both `OwnNamespace` and `AllNamespaces` enabled simultaneously. -More critically, OLM validates that a CSV supports the OperatorGroup's install mode before allowing an install or upgrade. +The single-CSV approach was tested on OpenShift 4.22.0-ec.3 (2026-08-13). +See [HackMD test log](https://hackmd.io/ZAjwOe39SjWv2yCWIlGdzg) for the full chronological record. -If a future release shipped only an AllNamespaces CSV, existing customers with a namespaced OperatorGroup (`targetNamespaces: [openshift-adp]`) would be **blocked from upgrading** — OLM would reject the new CSV because it doesn't support `OwnNamespace`. -The customer cannot change the OperatorGroup first, because their current CSV also doesn't support `AllNamespaces`, creating a **deadlock**. +Key results: +- CSV reached `Succeeded` with a global OperatorGroup in `openshift-adp` +- `WATCH_NAMESPACE` resolved to `openshift-adp` inside the pod +- All controllers started cleanly (DPA, CloudStorage, DataProtectionTest, CLI/VMDP downloads) +- The `operatorframework.io/suggested-namespace: openshift-adp` annotation (already in the CSV) causes OperatorHub to default to `openshift-adp` even in AllNamespaces mode (OpenShift 4.14+) -Both channels must coexist for at least one release cycle so customers can: -1. Upgrade to the version that offers both channels (staying on OwnNamespace). -2. Migrate to AllNamespaces at their own pace using the documented migration steps. -3. A future release can then deprecate the OwnNamespace channel after the migration window closes. +RBAC is already cluster-scoped (ClusterRoles and ClusterRoleBindings), so no fundamental RBAC changes are required. +No webhooks are currently enabled, but this opens the door to having conversion webhooks, allowing us to have new CRD versions without breaking changes. +A `ClusterWideClient` (uncached) already exists for cross-namespace DPA validation. ## Goals -- Enable `AllNamespaces` install mode via a second channel in the existing OLM package. +- Enable `AllNamespaces` install mode alongside `OwnNamespace` in the same CSV. - Keep runtime behavior identical to today: the operator watches only the namespace it is deployed in. -- Provide a documented migration path for customers moving from `OwnNamespace` to `AllNamespaces`. -- Maintain full backward compatibility for existing `OwnNamespace` installations. +- Maintain full backward compatibility for existing `OwnNamespace` installations (no OperatorGroup change required for existing customers). ## Non Goals - Actually watching all namespaces. The operator continues to watch only its own namespace. Cluster-wide watching can be enabled in the future by setting `WATCH_NAMESPACE` to empty, but that is a separate enhancement requiring additional work (see Future Enhancements). - Multi-tenant Velero (one DPA per namespace). Global DPA singleton enforcement will be maintained. -- Removing `OwnNamespace` support. Both channels will coexist for at least one release cycle. Deprecation of OwnNamespace is a separate future decision. - `SingleNamespace` or `MultiNamespace` install mode support. ## High-Level Design -The work is split into five phases, each independently mergeable: +The change is a CSV metadata patch — no Go code changes, no new channels, no kustomize overlays, no catalog restructuring. + +Three changes to the CSV: + +1. Enable `AllNamespaces: true` in `installModes` (alongside the existing `OwnNamespace: true`). +2. Change `WATCH_NAMESPACE` source from `metadata.annotations['olm.targetNamespaces']` to `metadata.namespace`. +3. Add `permissions` (namespace-scoped Roles) entries for `non-admin-controller` and `velero` ServiceAccounts. -1. **Build infrastructure for two bundle variants and enable AllNamespaces** (kustomize overlays that produce two CSVs in separate channels within the same OLM package). The existing OwnNamespace CSV is not modified. -2. **Deploy and CI for AllNamespaces** — add `deploy-olm-allnamespaces` Makefile target using `operator-sdk run bundle --install-mode AllNamespaces` and a Prow CI job running the existing e2e suite against it. This gives immediate signal that the AllNamespaces bundle works end-to-end. -3. **AllNamespaces-specific e2e test scenarios** — migration test (channel switch + OperatorGroup swap), upgrade test parameterization, and additional validation. -4. **Migration documentation** for customers. -5. **OwnNamespace deprecation plan** — timeline and communication for eventually sunsetting the OwnNamespace channel. +Existing OwnNamespace installations are unaffected — under a namespaced OperatorGroup, `metadata.namespace` and `olm.targetNamespaces` resolve to the same value. ## Detailed Design @@ -73,175 +72,188 @@ The work is split into five phases, each independently mergeable: | Area | Current Behavior | Key File(s) | |---|---|---| -| CSV installModes | Only `OwnNamespace: true` | `config/manifests/bases/oadp-operator.clusterserviceversion.yaml` (lines 467-475) | -| CSV WATCH_NAMESPACE source | `olm.targetNamespaces` annotation | `bundle/manifests/oadp-operator.clusterserviceversion.yaml` (lines 1449-1452) | -| Manager WATCH_NAMESPACE source | `metadata.namespace` (downward API) | `config/manager/manager.yaml` (lines 63-66) | +| CSV installModes | Only `OwnNamespace: true` | `config/manifests/bases/oadp-operator.clusterserviceversion.yaml` (lines 463-471) | +| CSV WATCH_NAMESPACE source | `olm.targetNamespaces` annotation | `bundle/manifests/oadp-operator.clusterserviceversion.yaml` (line 1105) | +| Manager WATCH_NAMESPACE source | `metadata.namespace` (downward API) | `config/manager/manager.yaml` (lines 57-60) | +| CSV permissions | Only `openshift-adp-controller-manager` has a `permissions` entry | `bundle/manifests/oadp-operator.clusterserviceversion.yaml` (line 1183) | +| CSV clusterPermissions | Three SAs: `non-admin-controller`, `openshift-adp-controller-manager`, `velero` | `bundle/manifests/oadp-operator.clusterserviceversion.yaml` (line 727) | | Cache scoping | `DefaultNamespaces` map with single entry | `cmd/main.go` (lines 204-208) | | PSA labeling | Patches `watchNamespace`, errors if empty | `cmd/main.go` (lines 362-389) | | CLI/VMDP downloads | Skipped if `watchNamespace` is empty | `cmd/main.go` (lines 305-306) | | STS flow | Reads `WATCH_NAMESPACE` as install namespace | `pkg/credentials/stsflow/stsflow.go` (line 115) | | Sub-controllers | All receive `WATCH_NAMESPACE` = own namespace | `nonadmin_controller.go` (line 176), `kubevirt_datamover_controller.go` (line 152), `vmfilerestore_controller.go` (line 184) | | E2E deploy | `operator-sdk run bundle` (implicitly OwnNamespace) | `Makefile` (line 459) | -| E2E OperatorGroup | Always `targetNamespaces: [namespace]` | `upgrade_suite_test.go` (lines 31-50) | -| Cross-NS validation | Uses uncached `ClusterWideClient` | `cmd/main.go` (lines 260-270), `validator.go` (line 144) | +| Suggested namespace | `operatorframework.io/suggested-namespace: openshift-adp` | `config/manifests/bases/oadp-operator.clusterserviceversion.yaml` (line 24) | | RBAC | Already cluster-scoped (ClusterRoles) | `config/rbac/role.yaml`, CSV `clusterPermissions` | -| OLM channels | Single channel: `dev` (release branches use e.g. `oadp-1.5`) | `Makefile` (lines 23, 33), `bundle/metadata/annotations.yaml` | -### Phase 1: Build Infrastructure for Two Bundle Variants and Enable AllNamespaces +### Change 1: Enable AllNamespaces Install Mode + +In `config/manifests/bases/oadp-operator.clusterserviceversion.yaml`: + +```yaml + installModes: + - supported: true + type: OwnNamespace + - supported: false + type: SingleNamespace + - supported: false + type: MultiNamespace + - supported: true # ← changed from false + type: AllNamespaces +``` -Produce two distinct OLM bundles from one codebase, shipped as separate channels in the same `oadp-operator` package. -The CSV changes (installModes flip, `WATCH_NAMESPACE` source change) are applied only to the AllNamespaces variant via a kustomize overlay — the existing OwnNamespace CSV is never modified. +This allows the CSV to be installed with either a namespaced OperatorGroup (OwnNamespace) or a global OperatorGroup (AllNamespaces). +Existing customers with a namespaced OperatorGroup are unaffected — their install mode stays OwnNamespace. -The AllNamespaces CSV differs from the OwnNamespace CSV in exactly two ways: +### Change 2: WATCH_NAMESPACE Source -1. `installModes` — only `AllNamespaces: true` (instead of only `OwnNamespace: true`). -2. `WATCH_NAMESPACE` source — `metadata.namespace` via downward API (instead of `olm.targetNamespaces` annotation). +The `WATCH_NAMESPACE` env var in the CSV deployment spec must be changed from `olm.targetNamespaces` to `metadata.namespace`. -With `WATCH_NAMESPACE` sourced from `metadata.namespace`, the operator always resolves to the pod's own namespace regardless of what `olm.targetNamespaces` contains (which would be empty under a global OperatorGroup). -No Go code changes are needed. The operator binary is identical for both CSVs. +Currently, `operator-sdk generate bundle` automatically rewrites `metadata.namespace` to `metadata.annotations['olm.targetNamespaces']` during bundle generation. +This must be patched after generation, or the bundle generation process must be modified to preserve `metadata.namespace`. -#### Catalog Structure +**Before:** +```yaml +- name: WATCH_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.annotations['olm.targetNamespaces'] +``` -```text -Catalog: oadp-operator-catalog -└── Package: oadp-operator - ├── Channel: dev ← OwnNamespace CSV - │ └── oadp-operator.v99.0.0 - └── Channel: dev-allnamespaces ← AllNamespaces CSV - └── oadp-operator.v99.0.0-allns +**After:** +```yaml +- name: WATCH_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace ``` -Release branches follow the same pattern: `stable-1.7` (OwnNamespace) and `stable-1.7-allnamespaces` (AllNamespaces). +Under a namespaced OperatorGroup (OwnNamespace), both sources resolve to the same value — the pod's namespace. +Under a global OperatorGroup (AllNamespaces), `olm.targetNamespaces` would be empty, but `metadata.namespace` correctly resolves to the pod's namespace. + +This change is backward-compatible and has no behavioral impact on existing installations. + +### Change 3: Add Permissions for Missing ServiceAccounts + +In AllNamespaces mode, OLM requires every ServiceAccount declared in `clusterPermissions` to also have a corresponding `permissions` (namespace-scoped Role) entry. +Without this, OLM cannot create the ServiceAccounts and the CSV stays `Pending` with `"no owned roles found"`. + +The CSV currently declares `clusterPermissions` for three SAs but only has a `permissions` entry for `openshift-adp-controller-manager` (leader-election Role). + +Add `permissions` entries for `non-admin-controller` and `velero` SAs with leader-election rules (configmaps, leases, events). +These are the same rules already used by the existing `openshift-adp-controller-manager` permissions entry. -#### What does NOT change +The corresponding RBAC config files also need updating: +- `config/non-admin-controller_rbac/` — add a `leader_election_role.yaml` and binding +- `config/velero/` — add a `leader_election_role.yaml` and binding + +### What does NOT change - No Go code changes. `cmd/main.go`, controllers, and all runtime behavior are untouched. -- The existing OwnNamespace CSV, bundle, and channel are not modified. - `WATCH_NAMESPACE` always resolves to a non-empty namespace name (the pod's own namespace). - Cache scoping, PSA labeling, STS flow, CLI/VMDP downloads, sub-controller propagation all continue to work exactly as today. -- RBAC is already cluster-scoped and does not need modification. - -#### Changes +- RBAC `clusterPermissions` (ClusterRoles and ClusterRoleBindings) are unchanged. +- The operator binary is identical — the same image is used regardless of install mode. +- The `operatorframework.io/suggested-namespace: openshift-adp` annotation remains, ensuring OperatorHub defaults to `openshift-adp` for both install modes. -| Area | Change | -|---|---| -| **CSV base template** | Keep `config/manifests/bases/oadp-operator.clusterserviceversion.yaml` as the shared base, unchanged | -| **Kustomize overlays** | Create `config/manifests/overlays/ownnamespace/` and `config/manifests/overlays/allnamespaces/` with patches for `installModes` and `WATCH_NAMESPACE` env var source | -| **AllNamespaces overlay** | Patches: (1) `installModes` set to only `AllNamespaces: true`, (2) `WATCH_NAMESPACE` source changed from `olm.targetNamespaces` to `metadata.namespace` | -| **OwnNamespace overlay** | Patches: keeps current behavior (only `OwnNamespace: true`, `WATCH_NAMESPACE` from `olm.targetNamespaces`). Identity transform initially. | -| **Makefile** | Add `INSTALL_MODE ?= OwnNamespace`. New targets: `bundle-allnamespaces`, `bundle-build-allnamespaces` | -| **Catalog build** | Extend `catalog-build` to produce a single catalog with two channels: existing channel (OwnNamespace bundle) and new `-allnamespaces` channel (AllNamespaces bundle). Adds a second `opm render` + `olm.channel` entry to the FBC output. | -| **CSV naming** | AllNamespaces CSV uses a distinct version suffix: `oadp-operator.v99.0.0-allns` vs `oadp-operator.v99.0.0` | -| **Channel naming** | Convention: `-allnamespaces` (e.g., `dev-allnamespaces`, `stable-1.7-allnamespaces`) | -| **Cross-channel update graph** | The AllNamespaces channel's CSV must include an `olm.skipRange` that covers the OwnNamespace CSV version (e.g., `>=0.0.0 <99.0.0`), so OLM can resolve a valid update path when a customer switches channels. Without this, the channel switch fails silently — OLM cannot find an upgrade edge from the installed CSV to the new channel's head. | +### OperatorHub User Experience -#### Validation +When both install modes are enabled, the OperatorHub UI (OpenShift 4.14+) presents: -- `make bundle` and `make bundle-allnamespaces` both produce valid bundles. -- `opm validate` passes on both bundles. -- Catalog with two channels builds and serves correctly. +1. **Install mode selection** — radio buttons: "All namespaces on the cluster" and "A specific namespace on the cluster" +2. **Namespace selection** — dropdown defaulting to `openshift-adp` (driven by the `suggested-namespace` annotation) in both modes -#### Risk: Medium +The `suggested-namespace` annotation ensures the operator installs in `openshift-adp` regardless of which install mode the user selects. +This follows the established pattern used by the Loki Operator (`openshift-operators-redhat`) and OpenShift Serverless (`openshift-serverless`). -Build plumbing only, no runtime impact. +## Implementation -### Phase 2: Deploy and CI for AllNamespaces +The work is split into three phases: -Add the ability to deploy and test with AllNamespaces mode, then wire up CI to run the existing e2e suite against it. -This gives immediate signal that the AllNamespaces bundle works end-to-end before writing any new test code. +| Phase | Scope | Risk | Depends on | +|---|---|---|---| +| 1. CSV changes | CSV metadata (installModes, WATCH_NAMESPACE source, permissions) | Low | None | +| 2. Deploy + CI for AllNamespaces | Makefile target + Prow job | Low | Phase 1 | +| 3. Migration documentation | Docs for OperatorGroup swap | Low | Phase 1 | -`operator-sdk run bundle` already supports `--install-mode` as a flag. -Today the `deploy-olm` target does not pass this flag, so it defaults to OwnNamespace. +### Phase 1: CSV Changes -#### Changes +Apply the three changes described above. -| Area | Change | +| File | Change | |---|---| -| **Makefile** | New target `deploy-olm-allnamespaces` that builds the AllNamespaces bundle and runs `operator-sdk run bundle --install-mode AllNamespaces --security-context-config restricted $(THIS_BUNDLE_IMAGE_ALLNS) --namespace $(OADP_TEST_NAMESPACE)` | -| **`openshift/release` config** | Add new presubmit job (e.g., `e2e-aws-allnamespaces`) that runs `make deploy-olm-allnamespaces` then `make test-e2e`. Runs the existing e2e suite unchanged. | -| **Periodic jobs** | Add AllNamespaces variant for nightly runs | - -#### What this validates - -- OLM accepts the AllNamespaces CSV with a global OperatorGroup. -- `WATCH_NAMESPACE` resolves to the pod's namespace (not empty). -- The full e2e suite passes with identical behavior: DPA creation, Velero deployment, backup/restore operations, sub-controllers, credential management. -- Any failures at this stage reveal real incompatibilities rather than test infrastructure gaps. +| `config/manifests/bases/oadp-operator.clusterserviceversion.yaml` | Set `AllNamespaces: supported: true` | +| `bundle/manifests/oadp-operator.clusterserviceversion.yaml` | Change `WATCH_NAMESPACE` source to `metadata.namespace` (post-generation patch or Makefile sed command) | +| `bundle/manifests/oadp-operator.clusterserviceversion.yaml` | Add `permissions` entries for `non-admin-controller` and `velero` SAs | +| `config/non-admin-controller_rbac/` | Add `leader_election_role.yaml` and `leader_election_role_binding.yaml` | +| `config/velero/` | Add `leader_election_role.yaml` and `leader_election_role_binding.yaml` | +| `Makefile` `bundle` target | Add post-generation step to replace `olm.targetNamespaces` with `metadata.namespace` in the generated CSV | #### Validation -Full existing e2e suite passes with `make deploy-olm-allnamespaces`. +- `make bundle` produces a valid bundle with both install modes enabled. +- `opm validate` passes. +- `operator-sdk run bundle --install-mode AllNamespaces` succeeds on a cluster (verified 2026-08-13). +- `operator-sdk run bundle --install-mode OwnNamespace` still works (backward compatibility). +- `WATCH_NAMESPACE` resolves to the pod's namespace in both modes. #### Risk: Low -One new Makefile target and one new CI job. No code changes. The existing e2e suite is the test. +CSV metadata changes only. No runtime behavioral change. Backward-compatible with existing OwnNamespace installations. -### Phase 3: AllNamespaces-Specific E2E Test Scenarios +### Phase 2: Deploy and CI for AllNamespaces -With CI running the existing suite against AllNamespaces (Phase 2), this phase adds test scenarios specific to the AllNamespaces install mode. +Add the ability to deploy and test with AllNamespaces mode, then wire up CI. -#### Changes +`operator-sdk run bundle` supports `--install-mode AllNamespaces` as a flag. | Area | Change | |---|---| +| **Makefile** | New target `deploy-olm-allnamespaces` that runs `operator-sdk run bundle --install-mode AllNamespaces --security-context-config restricted $(THIS_BUNDLE_IMAGE) --namespace $(OADP_TEST_NAMESPACE)` | +| **`openshift/release` config** | Add new presubmit job (e.g., `e2e-aws-allnamespaces`) that runs `make deploy-olm-allnamespaces` then `make test-e2e` | | `tests/e2e/upgrade_suite_test.go` (lines 31-50) | Parameterize OperatorGroup creation to support both modes based on a test flag or env var | -| Migration test | Test the documented migration path end-to-end: install OwnNamespace, switch channel, swap OperatorGroup, verify operator continues functioning | -| Upgrade test | Verify upgrading from prior version on OwnNamespace channel still works (no regression in existing upgrade path) | -| Singleton enforcement | Verify global DPA singleton is enforced in AllNamespaces mode via the existing `ClusterWideClient` validator | -#### Validation +#### What this validates -All new test scenarios pass. Existing OwnNamespace e2e suite continues to pass (no regressions). +- The full e2e suite passes with AllNamespaces mode: DPA creation, Velero deployment, backup/restore operations, sub-controllers, credential management. +- Any failures reveal real incompatibilities. -#### Risk: Medium +#### Risk: Low -Test infrastructure changes are additive. +One new Makefile target and one new CI job. The existing e2e suite is the test. -### Phase 4: Migration Documentation +### Phase 3: Migration Documentation -Customers migrating from OwnNamespace to AllNamespaces need clear, tested manual steps. -OLM does not automate the OperatorGroup swap. +Document how existing OwnNamespace customers can switch to AllNamespaces mode. -Note: after migration, the operator's runtime behavior is identical. -`WATCH_NAMESPACE` is sourced from `metadata.namespace` in the AllNamespaces CSV, so it always resolves to the operator's own namespace. -The operator does not start watching all namespaces. +Since both install modes are supported in the same CSV, migration is a single step: swap the OperatorGroup. +No channel switch or CSV change is needed. #### Prerequisites - Cluster admin access. - No active backups or restores in progress. -- Current OADP version supports the AllNamespaces channel (minimum version TBD). +- OADP version that supports AllNamespaces (the version containing Phase 1 changes). #### Migration Steps **Step 1: Verify current state** ```bash -oc get subscription oadp-operator -n openshift-adp -o yaml oc get operatorgroup -n openshift-adp -o yaml +oc get csv -n openshift-adp oc get dpa -n openshift-adp ``` -**Step 2: Switch subscription channel** - -```bash -oc patch subscription oadp-operator -n openshift-adp \ - --type merge -p '{"spec":{"channel":"stable-1.x-allnamespaces"}}' -``` - -OLM installs the new CSV. -The operator pod restarts, but the OperatorGroup still scopes it to OwnNamespace. -No behavioral change yet. - -**Step 3: Delete the existing namespaced OperatorGroup** +**Step 2: Delete the existing namespaced OperatorGroup** ```bash -oc delete operatorgroup oadp-operator-group -n openshift-adp +oc delete operatorgroup -n openshift-adp ``` The operator pod stops (OLM removes the deployment when no valid OperatorGroup exists). -**Step 4: Create a global OperatorGroup** +**Step 3: Create a global OperatorGroup** ```bash cat <-allnamespaces` (e.g., `dev-allnamespaces`, `stable-1.7-allnamespaces`). -Open to shorter alternatives if the convention is too verbose. - -2. **Minimum version for migration**: Which OADP release will be the first to ship the AllNamespaces channel? -This determines the migration documentation's version requirements and the minimum coexistence window before OwnNamespace can be deprecated. +1. **`operator-sdk generate bundle` override**: `operator-sdk generate bundle` automatically replaces `metadata.namespace` with `metadata.annotations['olm.targetNamespaces']` in the generated CSV. A post-generation patch (e.g., `sed` in the Makefile) or upstream SDK configuration is needed to preserve `metadata.namespace`. -3. **OwnNamespace deprecation timeline**: How many release cycles should both channels coexist before the OwnNamespace channel is removed? -At minimum one cycle is required for a safe upgrade path. +2. **Minimum version**: Which OADP release will include this change? ## Future Enhancements From b89bbb2e12b5a2b3199fe1a38d3bb048c2aa6567 Mon Sep 17 00:00:00 2001 From: Joseph Date: Thu, 13 Aug 2026 14:20:43 -0700 Subject: [PATCH 09/13] docs: incorporate review feedback and dual-mode test results MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Key improvements based on design review and OLM edge case research: - Added OLM behavioral differences section: CSV copies on large clusters, CRD ownership constraints, dual-installation prevention - Added OLMv1 considerations: AllNamespaces is the strategic direction, OLMv1 GA required it, OwnNamespace came later as compat feature - Added fresh AllNamespaces install path via OperatorHub (Console auto-creates namespace + OperatorGroup) - Clarified that velero SA permissions are placeholder rules for OLM, not actual leader election - Added OLM ensureSingletonRBAC behavior: permissions promoted to ClusterRoles in AllNamespaces mode - Noted operator-sdk substitution is hardcoded and cannot be disabled - Added alternative: handle empty WATCH_NAMESPACE in Go (rejected — requires code changes) - Added security note for new namespace-scoped permissions entries - Added minimum OpenShift version note - Updated test log link to full dual-mode validation - Removed Current State table (implementation detail, not design) - Simplified migration to essential steps only Co-Authored-By: Claude Opus 4.6 (1M context) --- .../allnamespaces-install-mode_design.md | 243 +++++++----------- 1 file changed, 87 insertions(+), 156 deletions(-) diff --git a/docs/design/allnamespaces-install-mode_design.md b/docs/design/allnamespaces-install-mode_design.md index 82cda8c3f7b..c86c61830c1 100644 --- a/docs/design/allnamespaces-install-mode_design.md +++ b/docs/design/allnamespaces-install-mode_design.md @@ -15,9 +15,7 @@ The OADP operator is installed via OLM with a strict `OwnNamespace` install mode The CSV declares only `OwnNamespace: true` and all other modes are `supported: false`. At runtime, the `WATCH_NAMESPACE` environment variable controls which namespace the controller-runtime cache monitors. - In the OwnNamespace CSV, `WATCH_NAMESPACE` is sourced from the `olm.targetNamespaces` annotation, which OLM sets to the operator's own namespace. - In the non-OLM deployment (`config/manager/manager.yaml`), `WATCH_NAMESPACE` is sourced from `metadata.namespace` (the pod's own namespace via downward API). Two things blocked AllNamespaces support: @@ -30,11 +28,13 @@ Both blockers are CSV metadata issues, not Go code issues. ### Validated on Cluster The single-CSV approach was tested on OpenShift 4.22.0-ec.3 (2026-08-13). -See [HackMD test log](https://hackmd.io/ZAjwOe39SjWv2yCWIlGdzg) for the full chronological record. +Both OwnNamespace and AllNamespaces install modes were validated with the same bundle. +See [full test log](https://hackmd.io/MVRrs4zTTHiwGcxfRUwhBA) for the chronological record. Key results: -- CSV reached `Succeeded` with a global OperatorGroup in `openshift-adp` -- `WATCH_NAMESPACE` resolved to `openshift-adp` inside the pod +- CSV reached `Succeeded` in both install modes +- `WATCH_NAMESPACE` resolved to `openshift-adp` in both modes +- DPA reconciliation, Velero deployment, and BSL creation worked identically in both modes - All controllers started cleanly (DPA, CloudStorage, DataProtectionTest, CLI/VMDP downloads) - The `operatorframework.io/suggested-namespace: openshift-adp` annotation (already in the CSV) causes OperatorHub to default to `openshift-adp` even in AllNamespaces mode (OpenShift 4.14+) @@ -56,36 +56,17 @@ A `ClusterWideClient` (uncached) already exists for cross-namespace DPA validati ## High-Level Design -The change is a CSV metadata patch — no Go code changes, no new channels, no kustomize overlays, no catalog restructuring. - Three changes to the CSV: 1. Enable `AllNamespaces: true` in `installModes` (alongside the existing `OwnNamespace: true`). 2. Change `WATCH_NAMESPACE` source from `metadata.annotations['olm.targetNamespaces']` to `metadata.namespace`. 3. Add `permissions` (namespace-scoped Roles) entries for `non-admin-controller` and `velero` ServiceAccounts. +No Go code changes, no new channels, no kustomize overlays, no catalog restructuring. Existing OwnNamespace installations are unaffected — under a namespaced OperatorGroup, `metadata.namespace` and `olm.targetNamespaces` resolve to the same value. ## Detailed Design -### Current State - -| Area | Current Behavior | Key File(s) | -|---|---|---| -| CSV installModes | Only `OwnNamespace: true` | `config/manifests/bases/oadp-operator.clusterserviceversion.yaml` (lines 463-471) | -| CSV WATCH_NAMESPACE source | `olm.targetNamespaces` annotation | `bundle/manifests/oadp-operator.clusterserviceversion.yaml` (line 1105) | -| Manager WATCH_NAMESPACE source | `metadata.namespace` (downward API) | `config/manager/manager.yaml` (lines 57-60) | -| CSV permissions | Only `openshift-adp-controller-manager` has a `permissions` entry | `bundle/manifests/oadp-operator.clusterserviceversion.yaml` (line 1183) | -| CSV clusterPermissions | Three SAs: `non-admin-controller`, `openshift-adp-controller-manager`, `velero` | `bundle/manifests/oadp-operator.clusterserviceversion.yaml` (line 727) | -| Cache scoping | `DefaultNamespaces` map with single entry | `cmd/main.go` (lines 204-208) | -| PSA labeling | Patches `watchNamespace`, errors if empty | `cmd/main.go` (lines 362-389) | -| CLI/VMDP downloads | Skipped if `watchNamespace` is empty | `cmd/main.go` (lines 305-306) | -| STS flow | Reads `WATCH_NAMESPACE` as install namespace | `pkg/credentials/stsflow/stsflow.go` (line 115) | -| Sub-controllers | All receive `WATCH_NAMESPACE` = own namespace | `nonadmin_controller.go` (line 176), `kubevirt_datamover_controller.go` (line 152), `vmfilerestore_controller.go` (line 184) | -| E2E deploy | `operator-sdk run bundle` (implicitly OwnNamespace) | `Makefile` (line 459) | -| Suggested namespace | `operatorframework.io/suggested-namespace: openshift-adp` | `config/manifests/bases/oadp-operator.clusterserviceversion.yaml` (line 24) | -| RBAC | Already cluster-scoped (ClusterRoles) | `config/rbac/role.yaml`, CSV `clusterPermissions` | - ### Change 1: Enable AllNamespaces Install Mode In `config/manifests/bases/oadp-operator.clusterserviceversion.yaml`: @@ -105,13 +86,13 @@ In `config/manifests/bases/oadp-operator.clusterserviceversion.yaml`: This allows the CSV to be installed with either a namespaced OperatorGroup (OwnNamespace) or a global OperatorGroup (AllNamespaces). Existing customers with a namespaced OperatorGroup are unaffected — their install mode stays OwnNamespace. +Adding installMode support is a safe superset change in OLM. +The reverse (removing a previously-supported installMode) would block upgrades, but adding support never does. + ### Change 2: WATCH_NAMESPACE Source The `WATCH_NAMESPACE` env var in the CSV deployment spec must be changed from `olm.targetNamespaces` to `metadata.namespace`. -Currently, `operator-sdk generate bundle` automatically rewrites `metadata.namespace` to `metadata.annotations['olm.targetNamespaces']` during bundle generation. -This must be patched after generation, or the bundle generation process must be modified to preserve `metadata.namespace`. - **Before:** ```yaml - name: WATCH_NAMESPACE @@ -133,6 +114,13 @@ Under a global OperatorGroup (AllNamespaces), `olm.targetNamespaces` would be em This change is backward-compatible and has no behavioral impact on existing installations. +Note: `operator-sdk generate bundle` automatically rewrites `metadata.namespace` to `metadata.annotations['olm.targetNamespaces']` during bundle generation. +This substitution is hardcoded in the SDK (`setNamespacedFields` in `clusterserviceversion_updaters.go`) and cannot be disabled. +A post-generation patch step is required to restore `metadata.namespace`. + +Note: OLM still sets the `olm.targetNamespaces` annotation on the pod template regardless of whether the operator reads it. +This annotation continues to function for OLM's internal RBAC management — it is simply unused by the operator's env var. + ### Change 3: Add Permissions for Missing ServiceAccounts In AllNamespaces mode, OLM requires every ServiceAccount declared in `clusterPermissions` to also have a corresponding `permissions` (namespace-scoped Role) entry. @@ -142,10 +130,11 @@ The CSV currently declares `clusterPermissions` for three SAs but only has a `pe Add `permissions` entries for `non-admin-controller` and `velero` SAs with leader-election rules (configmaps, leases, events). These are the same rules already used by the existing `openshift-adp-controller-manager` permissions entry. +The `velero` SA does not actually perform leader election — these are placeholder rules required solely to satisfy OLM's SA creation requirement. -The corresponding RBAC config files also need updating: -- `config/non-admin-controller_rbac/` — add a `leader_election_role.yaml` and binding -- `config/velero/` — add a `leader_election_role.yaml` and binding +In AllNamespaces mode, OLM promotes these namespace-scoped `permissions` to ClusterRoles/ClusterRoleBindings via `ensureSingletonRBAC`. +In OwnNamespace mode, they remain namespace-scoped Roles/RoleBindings. +This promotion is handled entirely by OLM and is transparent to the operator. ### What does NOT change @@ -156,147 +145,79 @@ The corresponding RBAC config files also need updating: - The operator binary is identical — the same image is used regardless of install mode. - The `operatorframework.io/suggested-namespace: openshift-adp` annotation remains, ensuring OperatorHub defaults to `openshift-adp` for both install modes. -### OperatorHub User Experience - -When both install modes are enabled, the OperatorHub UI (OpenShift 4.14+) presents: +### OLM Behavioral Differences in AllNamespaces Mode -1. **Install mode selection** — radio buttons: "All namespaces on the cluster" and "A specific namespace on the cluster" -2. **Namespace selection** — dropdown defaulting to `openshift-adp` (driven by the `suggested-namespace` annotation) in both modes +In AllNamespaces mode, OLM behaves differently in several ways that do not affect operator functionality but should be understood: -The `suggested-namespace` annotation ensures the operator installs in `openshift-adp` regardless of which install mode the user selects. -This follows the established pattern used by the Loki Operator (`openshift-operators-redhat`) and OpenShift Serverless (`openshift-serverless`). +- **CSV copies**: OLM creates a copy of the CSV resource in every namespace on the cluster. + On large clusters this has performance implications (etcd storage, API server load). + Mitigation: `OLMConfig` provides `spec.features.disableCopiedCSVs: true` to disable copies for AllNamespaces operators. + Copied CSVs are informational only and do not affect operator behavior. -## Implementation +- **CRD ownership**: In AllNamespaces mode, the operator globally owns its CRDs. + No other operator can declare ownership of the same CRDs (e.g., Velero CRDs). + This is not an issue for OADP since it is the sole owner of both `oadp.openshift.io` and `velero.io` CRDs on the cluster. + However, customers running a standalone upstream Velero alongside OADP would hit an `InterOperatorGroupOwnerConflict`. -The work is split into three phases: +- **Dual installation prevention**: OLM prevents installing the same operator in two different namespaces with overlapping OperatorGroups. + A customer cannot have OADP in both `openshift-adp` (global) and `openshift-operators` (global) simultaneously. -| Phase | Scope | Risk | Depends on | -|---|---|---|---| -| 1. CSV changes | CSV metadata (installModes, WATCH_NAMESPACE source, permissions) | Low | None | -| 2. Deploy + CI for AllNamespaces | Makefile target + Prow job | Low | Phase 1 | -| 3. Migration documentation | Docs for OperatorGroup swap | Low | Phase 1 | - -### Phase 1: CSV Changes - -Apply the three changes described above. - -| File | Change | -|---|---| -| `config/manifests/bases/oadp-operator.clusterserviceversion.yaml` | Set `AllNamespaces: supported: true` | -| `bundle/manifests/oadp-operator.clusterserviceversion.yaml` | Change `WATCH_NAMESPACE` source to `metadata.namespace` (post-generation patch or Makefile sed command) | -| `bundle/manifests/oadp-operator.clusterserviceversion.yaml` | Add `permissions` entries for `non-admin-controller` and `velero` SAs | -| `config/non-admin-controller_rbac/` | Add `leader_election_role.yaml` and `leader_election_role_binding.yaml` | -| `config/velero/` | Add `leader_election_role.yaml` and `leader_election_role_binding.yaml` | -| `Makefile` `bundle` target | Add post-generation step to replace `olm.targetNamespaces` with `metadata.namespace` in the generated CSV | +### OperatorHub User Experience -#### Validation +When both install modes are enabled, the OperatorHub UI (OpenShift 4.14+) presents: -- `make bundle` produces a valid bundle with both install modes enabled. -- `opm validate` passes. -- `operator-sdk run bundle --install-mode AllNamespaces` succeeds on a cluster (verified 2026-08-13). -- `operator-sdk run bundle --install-mode OwnNamespace` still works (backward compatibility). -- `WATCH_NAMESPACE` resolves to the pod's namespace in both modes. +1. **Install mode selection** — radio buttons: "All namespaces on the cluster" and "A specific namespace on the cluster" +2. **Namespace selection** — dropdown defaulting to `openshift-adp` (driven by the `suggested-namespace` annotation) in both modes -#### Risk: Low +For fresh AllNamespaces installations via OperatorHub, the Console automatically creates: +- The `openshift-adp` namespace (if it doesn't exist) +- A global OperatorGroup in that namespace +- A Subscription pointing to the operator -CSV metadata changes only. No runtime behavioral change. Backward-compatible with existing OwnNamespace installations. +This follows the established pattern used by the Loki Operator (`openshift-operators-redhat`) and OpenShift Serverless (`openshift-serverless`). -### Phase 2: Deploy and CI for AllNamespaces +For CLI installations, users must manually create the namespace, OperatorGroup, and Subscription. -Add the ability to deploy and test with AllNamespaces mode, then wire up CI. +### OLMv1 Considerations -`operator-sdk run bundle` supports `--install-mode AllNamespaces` as a flag. +AllNamespaces is the strategically correct direction for OLMv1. +At OLMv1 GA (OCP 4.18), only AllNamespaces operators were installable. +OwnNamespace support was added later as a backward-compatibility feature (Tech Preview in OCP 4.19, GA in OCP 4.22). -| Area | Change | -|---|---| -| **Makefile** | New target `deploy-olm-allnamespaces` that runs `operator-sdk run bundle --install-mode AllNamespaces --security-context-config restricted $(THIS_BUNDLE_IMAGE) --namespace $(OADP_TEST_NAMESPACE)` | -| **`openshift/release` config** | Add new presubmit job (e.g., `e2e-aws-allnamespaces`) that runs `make deploy-olm-allnamespaces` then `make test-e2e` | -| `tests/e2e/upgrade_suite_test.go` (lines 31-50) | Parameterize OperatorGroup creation to support both modes based on a test flag or env var | +OLMv1 does not use `installModes` or `OperatorGroups`. +Namespace scoping is handled via `ClusterExtension.spec.config.inline.watchNamespace`. +No operator code changes are needed for OLMv1 compatibility — only the installation mechanism changes. -#### What this validates +OLM Classic coexists with OLMv1 throughout the OpenShift 4 lifecycle. -- The full e2e suite passes with AllNamespaces mode: DPA creation, Velero deployment, backup/restore operations, sub-controllers, credential management. -- Any failures reveal real incompatibilities. +## Implementation -#### Risk: Low +| Phase | Scope | Risk | Depends on | +|---|---|---|---| +| 1. CSV changes | CSV metadata (installModes, WATCH_NAMESPACE source, permissions) | Low | None | +| 2. Deploy + CI for AllNamespaces | Makefile target + Prow job | Low | Phase 1 | +| 3. Migration documentation | Docs for OperatorGroup swap | Low | Phase 1 | -One new Makefile target and one new CI job. The existing e2e suite is the test. +### Upgrade Path -### Phase 3: Migration Documentation +Upgrading from a prior OADP version (OwnNamespace-only) to the version with this change is seamless. +Adding `AllNamespaces: true` to the installModes is a safe superset change — OLM does not reject the upgrade. +The customer's existing namespaced OperatorGroup continues to work. +No customer action is required unless they want to switch to AllNamespaces mode. -Document how existing OwnNamespace customers can switch to AllNamespaces mode. +### Migration (OwnNamespace to AllNamespaces) Since both install modes are supported in the same CSV, migration is a single step: swap the OperatorGroup. No channel switch or CSV change is needed. -#### Prerequisites - -- Cluster admin access. -- No active backups or restores in progress. -- OADP version that supports AllNamespaces (the version containing Phase 1 changes). - -#### Migration Steps - -**Step 1: Verify current state** - -```bash -oc get operatorgroup -n openshift-adp -o yaml -oc get csv -n openshift-adp -oc get dpa -n openshift-adp -``` - -**Step 2: Delete the existing namespaced OperatorGroup** - -```bash -oc delete operatorgroup -n openshift-adp -``` - -The operator pod stops (OLM removes the deployment when no valid OperatorGroup exists). - -**Step 3: Create a global OperatorGroup** - -```bash -cat < Date: Thu, 13 Aug 2026 16:21:15 -0700 Subject: [PATCH 10/13] docs: add Phase 3 for AllNamespaces install e2e tests MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds dedicated test phase between CI setup and migration docs: - AllNamespaces fresh install validation - OwnNamespace → AllNamespaces migration test - AllNamespaces → OwnNamespace rollback test - Upgrade from OwnNamespace-only to dual-mode CSV - Upgrade test parameterization for global OperatorGroup Co-Authored-By: Claude Opus 4.6 (1M context) --- .../design/allnamespaces-install-mode_design.md | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/docs/design/allnamespaces-install-mode_design.md b/docs/design/allnamespaces-install-mode_design.md index c86c61830c1..84208aa4a3a 100644 --- a/docs/design/allnamespaces-install-mode_design.md +++ b/docs/design/allnamespaces-install-mode_design.md @@ -195,8 +195,21 @@ OLM Classic coexists with OLMv1 throughout the OpenShift 4 lifecycle. | Phase | Scope | Risk | Depends on | |---|---|---|---| | 1. CSV changes | CSV metadata (installModes, WATCH_NAMESPACE source, permissions) | Low | None | -| 2. Deploy + CI for AllNamespaces | Makefile target + Prow job | Low | Phase 1 | -| 3. Migration documentation | Docs for OperatorGroup swap | Low | Phase 1 | +| 2. Deploy + CI for AllNamespaces | Makefile target + Prow job running existing e2e suite | Low | Phase 1 | +| 3. AllNamespaces install e2e tests | Test scenarios for AllNamespaces-specific behavior | Medium | Phase 2 | +| 4. Migration documentation | Docs for OperatorGroup swap | Low | Phase 2 | + +### Phase 3: AllNamespaces Install E2E Tests + +With Phase 2 providing baseline signal from the existing e2e suite, this phase adds test scenarios that specifically validate AllNamespaces install behavior. + +Test scenarios: + +- **AllNamespaces fresh install**: install with `--install-mode AllNamespaces`, verify CSV Succeeded, WATCH_NAMESPACE = pod namespace, DPA reconciles, Velero deploys. +- **OwnNamespace to AllNamespaces migration**: install OwnNamespace, swap OperatorGroup to global, verify operator re-deploys and DPA continues functioning without re-creation. +- **AllNamespaces to OwnNamespace rollback**: reverse the migration, verify operator recovers. +- **Upgrade with AllNamespaces**: upgrade from a prior OADP version (OwnNamespace-only CSV) to the new version (dual-mode CSV), verify the upgrade succeeds and the existing namespaced OperatorGroup continues to work. +- **Upgrade test parameterization**: the existing upgrade test (`upgrade_suite_test.go`) hardcodes a namespaced OperatorGroup. Parameterize to also test with a global OperatorGroup. ### Upgrade Path From 0ecd96db564169a95995bc6954a999b9e4738906 Mon Sep 17 00:00:00 2001 From: Joseph Date: Fri, 21 Aug 2026 11:34:34 -0700 Subject: [PATCH 11/13] docs: tighten design doc structure and clarity - Replace defensive "What does NOT change" list with confident design - Add WATCH_NAMESPACE resolution comparison table in High-Level Design - Add concise change summary table (what + why) - Move validation results from Background to dedicated section - Move OLMv1 alignment to Background (it's motivation, not design) - Give all four phases consistent detail sections - Remove repetitive backward-compatibility statements - Consolidate Future Enhancements into single "Cluster-Wide Watching" - Overall: shorter, clearer, better flow Co-Authored-By: Claude Opus 4.6 (1M context) --- .../allnamespaces-install-mode_design.md | 314 +++++++----------- 1 file changed, 127 insertions(+), 187 deletions(-) diff --git a/docs/design/allnamespaces-install-mode_design.md b/docs/design/allnamespaces-install-mode_design.md index 84208aa4a3a..b0828bdb6a1 100644 --- a/docs/design/allnamespaces-install-mode_design.md +++ b/docs/design/allnamespaces-install-mode_design.md @@ -2,300 +2,240 @@ ## Abstract -OADP operator currently supports only `OwnNamespace` install mode via OLM. - -This proposal enables `AllNamespaces` install mode in the same CSV by enabling both install modes simultaneously and changing the `WATCH_NAMESPACE` source from `olm.targetNamespaces` to `metadata.namespace`. - -The operator's runtime behavior remains identical: it watches only the namespace it is deployed in, regardless of install mode. +Enable `AllNamespaces` install mode alongside the existing `OwnNamespace` mode in a single CSV. +The operator's runtime behavior is unchanged — it watches only the namespace it is deployed in. No Go code changes are required. ## Background -The OADP operator is installed via OLM with a strict `OwnNamespace` install mode. -The CSV declares only `OwnNamespace: true` and all other modes are `supported: false`. - +OADP is installed via OLM with only `OwnNamespace` install mode supported. At runtime, the `WATCH_NAMESPACE` environment variable controls which namespace the controller-runtime cache monitors. -In the OwnNamespace CSV, `WATCH_NAMESPACE` is sourced from the `olm.targetNamespaces` annotation, which OLM sets to the operator's own namespace. -In the non-OLM deployment (`config/manager/manager.yaml`), `WATCH_NAMESPACE` is sourced from `metadata.namespace` (the pod's own namespace via downward API). -Two things blocked AllNamespaces support: +The OLM-deployed CSV sources `WATCH_NAMESPACE` from the `olm.targetNamespaces` annotation, which OLM sets based on the OperatorGroup. +In OwnNamespace mode this resolves to the operator's namespace. +In AllNamespaces mode this would be empty — breaking PSA labeling, STS credential flow, CLI/VMDP setup, and the cache configuration. -1. **`WATCH_NAMESPACE` sourced from `olm.targetNamespaces`** — in AllNamespaces mode, `olm.targetNamespaces` is empty, which would break PSA labeling, STS credential flow, CLI/VMDP setup, and the controller-runtime cache configuration. -2. **Missing namespace-scoped `permissions` entries** — OLM requires every ServiceAccount declared in `clusterPermissions` to also have a `permissions` (namespace-scoped Role) entry when running in AllNamespaces mode. Without this, OLM refuses to create the ServiceAccounts and the CSV stays `Pending` with `"no owned roles found"`. +The fix is to source `WATCH_NAMESPACE` from `metadata.namespace` (the pod's own namespace via Kubernetes downward API) instead. +This resolves to the same value under OwnNamespace (backward-compatible) and correctly resolves to the pod's namespace under AllNamespaces. -Both blockers are CSV metadata issues, not Go code issues. +Additionally, OLM requires every ServiceAccount declared in `clusterPermissions` to have a corresponding `permissions` (namespace-scoped Role) entry when running in AllNamespaces mode. +Two of the three OADP ServiceAccounts currently lack this entry. -### Validated on Cluster +Both issues are CSV metadata problems, not Go code problems. -The single-CSV approach was tested on OpenShift 4.22.0-ec.3 (2026-08-13). -Both OwnNamespace and AllNamespaces install modes were validated with the same bundle. -See [full test log](https://hackmd.io/MVRrs4zTTHiwGcxfRUwhBA) for the chronological record. +### OLMv1 Alignment -Key results: -- CSV reached `Succeeded` in both install modes -- `WATCH_NAMESPACE` resolved to `openshift-adp` in both modes -- DPA reconciliation, Velero deployment, and BSL creation worked identically in both modes -- All controllers started cleanly (DPA, CloudStorage, DataProtectionTest, CLI/VMDP downloads) -- The `operatorframework.io/suggested-namespace: openshift-adp` annotation (already in the CSV) causes OperatorHub to default to `openshift-adp` even in AllNamespaces mode (OpenShift 4.14+) - -RBAC is already cluster-scoped (ClusterRoles and ClusterRoleBindings), so no fundamental RBAC changes are required. -No webhooks are currently enabled, but this opens the door to having conversion webhooks, allowing us to have new CRD versions without breaking changes. -A `ClusterWideClient` (uncached) already exists for cross-namespace DPA validation. +AllNamespaces is the strategically correct direction. +At OLMv1 GA (OCP 4.18), only AllNamespaces operators were installable. +OwnNamespace support was added later as backward-compatibility (Tech Preview in OCP 4.19, GA in OCP 4.22). +Enabling AllNamespaces now positions OADP for OLMv1 readiness. ## Goals - Enable `AllNamespaces` install mode alongside `OwnNamespace` in the same CSV. -- Keep runtime behavior identical to today: the operator watches only the namespace it is deployed in. -- Maintain full backward compatibility for existing `OwnNamespace` installations (no OperatorGroup change required for existing customers). +- Keep runtime behavior identical: the operator watches only the namespace it is deployed in. +- Maintain full backward compatibility for existing `OwnNamespace` installations. ## Non Goals -- Actually watching all namespaces. The operator continues to watch only its own namespace. Cluster-wide watching can be enabled in the future by setting `WATCH_NAMESPACE` to empty, but that is a separate enhancement requiring additional work (see Future Enhancements). -- Multi-tenant Velero (one DPA per namespace). Global DPA singleton enforcement will be maintained. +- Actually watching all namespaces or supporting multi-tenant Velero (one DPA per namespace). - `SingleNamespace` or `MultiNamespace` install mode support. ## High-Level Design -Three changes to the CSV: +Three CSV metadata changes, no Go code changes: + +| # | Change | Why | +|---|---|---| +| 1 | Enable `AllNamespaces: true` in `installModes` | Allow installation with a global OperatorGroup | +| 2 | Source `WATCH_NAMESPACE` from `metadata.namespace` | Avoid empty value under AllNamespaces; backward-compatible under OwnNamespace | +| 3 | Add `permissions` entries for `non-admin-controller` and `velero` SAs | OLM requires these to create the ServiceAccounts in AllNamespaces mode | -1. Enable `AllNamespaces: true` in `installModes` (alongside the existing `OwnNamespace: true`). -2. Change `WATCH_NAMESPACE` source from `metadata.annotations['olm.targetNamespaces']` to `metadata.namespace`. -3. Add `permissions` (namespace-scoped Roles) entries for `non-admin-controller` and `velero` ServiceAccounts. +The operator binary, RBAC `clusterPermissions`, and all runtime behavior remain identical. -No Go code changes, no new channels, no kustomize overlays, no catalog restructuring. -Existing OwnNamespace installations are unaffected — under a namespaced OperatorGroup, `metadata.namespace` and `olm.targetNamespaces` resolve to the same value. +**How `WATCH_NAMESPACE` resolves in each mode:** + +| OperatorGroup | `olm.targetNamespaces` | `metadata.namespace` | Operator watches | +|---|---|---|---| +| Namespaced (OwnNamespace) | `openshift-adp` | `openshift-adp` | `openshift-adp` | +| Global (AllNamespaces) | `""` (empty) | `openshift-adp` | `openshift-adp` | ## Detailed Design ### Change 1: Enable AllNamespaces Install Mode -In `config/manifests/bases/oadp-operator.clusterserviceversion.yaml`: - ```yaml - installModes: - - supported: true - type: OwnNamespace - - supported: false - type: SingleNamespace - - supported: false - type: MultiNamespace - - supported: true # ← changed from false - type: AllNamespaces +installModes: +- supported: true + type: OwnNamespace +- supported: false + type: SingleNamespace +- supported: false + type: MultiNamespace +- supported: true # ← changed from false + type: AllNamespaces ``` -This allows the CSV to be installed with either a namespaced OperatorGroup (OwnNamespace) or a global OperatorGroup (AllNamespaces). -Existing customers with a namespaced OperatorGroup are unaffected — their install mode stays OwnNamespace. - -Adding installMode support is a safe superset change in OLM. -The reverse (removing a previously-supported installMode) would block upgrades, but adding support never does. +Adding installMode support is a safe superset change in OLM — it never blocks upgrades. +Existing customers with a namespaced OperatorGroup are unaffected; their install mode stays OwnNamespace. ### Change 2: WATCH_NAMESPACE Source -The `WATCH_NAMESPACE` env var in the CSV deployment spec must be changed from `olm.targetNamespaces` to `metadata.namespace`. - -**Before:** ```yaml +# Before - name: WATCH_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.annotations['olm.targetNamespaces'] -``` -**After:** -```yaml +# After - name: WATCH_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace ``` -Under a namespaced OperatorGroup (OwnNamespace), both sources resolve to the same value — the pod's namespace. -Under a global OperatorGroup (AllNamespaces), `olm.targetNamespaces` would be empty, but `metadata.namespace` correctly resolves to the pod's namespace. - -This change is backward-compatible and has no behavioral impact on existing installations. - -Note: `operator-sdk generate bundle` automatically rewrites `metadata.namespace` to `metadata.annotations['olm.targetNamespaces']` during bundle generation. -This substitution is hardcoded in the SDK (`setNamespacedFields` in `clusterserviceversion_updaters.go`) and cannot be disabled. +`operator-sdk generate bundle` automatically rewrites `metadata.namespace` to `olm.targetNamespaces` during bundle generation. +This substitution is hardcoded in the SDK and cannot be disabled. A post-generation patch step is required to restore `metadata.namespace`. -Note: OLM still sets the `olm.targetNamespaces` annotation on the pod template regardless of whether the operator reads it. -This annotation continues to function for OLM's internal RBAC management — it is simply unused by the operator's env var. +OLM still sets the `olm.targetNamespaces` annotation on the pod template regardless — it is simply unused by the operator's env var. ### Change 3: Add Permissions for Missing ServiceAccounts -In AllNamespaces mode, OLM requires every ServiceAccount declared in `clusterPermissions` to also have a corresponding `permissions` (namespace-scoped Role) entry. -Without this, OLM cannot create the ServiceAccounts and the CSV stays `Pending` with `"no owned roles found"`. - -The CSV currently declares `clusterPermissions` for three SAs but only has a `permissions` entry for `openshift-adp-controller-manager` (leader-election Role). +The CSV declares `clusterPermissions` for three SAs but only `openshift-adp-controller-manager` has a `permissions` entry. +Add `permissions` entries for `non-admin-controller` and `velero` with leader-election rules (configmaps, leases, events). -Add `permissions` entries for `non-admin-controller` and `velero` SAs with leader-election rules (configmaps, leases, events). -These are the same rules already used by the existing `openshift-adp-controller-manager` permissions entry. The `velero` SA does not actually perform leader election — these are placeholder rules required solely to satisfy OLM's SA creation requirement. - -In AllNamespaces mode, OLM promotes these namespace-scoped `permissions` to ClusterRoles/ClusterRoleBindings via `ensureSingletonRBAC`. +In AllNamespaces mode, OLM promotes these to ClusterRoles/ClusterRoleBindings via `ensureSingletonRBAC`. In OwnNamespace mode, they remain namespace-scoped Roles/RoleBindings. -This promotion is handled entirely by OLM and is transparent to the operator. -### What does NOT change +### OLM Behavior in AllNamespaces Mode -- No Go code changes. `cmd/main.go`, controllers, and all runtime behavior are untouched. -- `WATCH_NAMESPACE` always resolves to a non-empty namespace name (the pod's own namespace). -- Cache scoping, PSA labeling, STS flow, CLI/VMDP downloads, sub-controller propagation all continue to work exactly as today. -- RBAC `clusterPermissions` (ClusterRoles and ClusterRoleBindings) are unchanged. -- The operator binary is identical — the same image is used regardless of install mode. -- The `operatorframework.io/suggested-namespace: openshift-adp` annotation remains, ensuring OperatorHub defaults to `openshift-adp` for both install modes. +These OLM behaviors do not affect operator functionality but should be understood: -### OLM Behavioral Differences in AllNamespaces Mode - -In AllNamespaces mode, OLM behaves differently in several ways that do not affect operator functionality but should be understood: - -- **CSV copies**: OLM creates a copy of the CSV resource in every namespace on the cluster. - On large clusters this has performance implications (etcd storage, API server load). - Mitigation: `OLMConfig` provides `spec.features.disableCopiedCSVs: true` to disable copies for AllNamespaces operators. - Copied CSVs are informational only and do not affect operator behavior. - -- **CRD ownership**: In AllNamespaces mode, the operator globally owns its CRDs. - No other operator can declare ownership of the same CRDs (e.g., Velero CRDs). - This is not an issue for OADP since it is the sole owner of both `oadp.openshift.io` and `velero.io` CRDs on the cluster. - However, customers running a standalone upstream Velero alongside OADP would hit an `InterOperatorGroupOwnerConflict`. - -- **Dual installation prevention**: OLM prevents installing the same operator in two different namespaces with overlapping OperatorGroups. - A customer cannot have OADP in both `openshift-adp` (global) and `openshift-operators` (global) simultaneously. +- **CSV copies**: OLM copies the CSV to every namespace. On large clusters, `OLMConfig.spec.features.disableCopiedCSVs: true` disables this. +- **CRD ownership**: The operator globally owns its CRDs. Customers running standalone upstream Velero alongside OADP would hit `InterOperatorGroupOwnerConflict`. +- **Dual installation prevention**: OLM prevents installing the operator in two namespaces with overlapping OperatorGroups. ### OperatorHub User Experience -When both install modes are enabled, the OperatorHub UI (OpenShift 4.14+) presents: - -1. **Install mode selection** — radio buttons: "All namespaces on the cluster" and "A specific namespace on the cluster" -2. **Namespace selection** — dropdown defaulting to `openshift-adp` (driven by the `suggested-namespace` annotation) in both modes +When both install modes are enabled, OperatorHub (OpenShift 4.14+) presents install mode radio buttons and a namespace dropdown. +The existing `suggested-namespace: openshift-adp` annotation defaults the namespace to `openshift-adp` in both modes. +For fresh AllNamespaces installs, the Console automatically creates the namespace, a global OperatorGroup, and a Subscription. -For fresh AllNamespaces installations via OperatorHub, the Console automatically creates: -- The `openshift-adp` namespace (if it doesn't exist) -- A global OperatorGroup in that namespace -- A Subscription pointing to the operator - -This follows the established pattern used by the Loki Operator (`openshift-operators-redhat`) and OpenShift Serverless (`openshift-serverless`). - -For CLI installations, users must manually create the namespace, OperatorGroup, and Subscription. - -### OLMv1 Considerations - -AllNamespaces is the strategically correct direction for OLMv1. -At OLMv1 GA (OCP 4.18), only AllNamespaces operators were installable. -OwnNamespace support was added later as a backward-compatibility feature (Tech Preview in OCP 4.19, GA in OCP 4.22). - -OLMv1 does not use `installModes` or `OperatorGroups`. -Namespace scoping is handled via `ClusterExtension.spec.config.inline.watchNamespace`. -No operator code changes are needed for OLMv1 compatibility — only the installation mechanism changes. - -OLM Classic coexists with OLMv1 throughout the OpenShift 4 lifecycle. +This follows the pattern used by the Loki Operator (`openshift-operators-redhat`) and OpenShift Serverless (`openshift-serverless`). ## Implementation | Phase | Scope | Risk | Depends on | |---|---|---|---| -| 1. CSV changes | CSV metadata (installModes, WATCH_NAMESPACE source, permissions) | Low | None | -| 2. Deploy + CI for AllNamespaces | Makefile target + Prow job running existing e2e suite | Low | Phase 1 | -| 3. AllNamespaces install e2e tests | Test scenarios for AllNamespaces-specific behavior | Medium | Phase 2 | -| 4. Migration documentation | Docs for OperatorGroup swap | Low | Phase 2 | +| 1. CSV changes | installModes, WATCH_NAMESPACE source, permissions | Low | None | +| 2. Deploy + CI | Makefile target + Prow job with existing e2e suite | Low | Phase 1 | +| 3. AllNamespaces e2e tests | AllNamespaces-specific test scenarios | Medium | Phase 2 | +| 4. Migration documentation | OperatorGroup swap procedure | Low | Phase 3 | -### Phase 3: AllNamespaces Install E2E Tests +### Phase 1: CSV Changes -With Phase 2 providing baseline signal from the existing e2e suite, this phase adds test scenarios that specifically validate AllNamespaces install behavior. +Apply the three changes described in Detailed Design. +The `bundle` Makefile target needs a post-generation step to replace `olm.targetNamespaces` with `metadata.namespace`. +This must survive the `bundle-isupdated` CI check. -Test scenarios: +### Phase 2: Deploy + CI for AllNamespaces -- **AllNamespaces fresh install**: install with `--install-mode AllNamespaces`, verify CSV Succeeded, WATCH_NAMESPACE = pod namespace, DPA reconciles, Velero deploys. -- **OwnNamespace to AllNamespaces migration**: install OwnNamespace, swap OperatorGroup to global, verify operator re-deploys and DPA continues functioning without re-creation. -- **AllNamespaces to OwnNamespace rollback**: reverse the migration, verify operator recovers. -- **Upgrade with AllNamespaces**: upgrade from a prior OADP version (OwnNamespace-only CSV) to the new version (dual-mode CSV), verify the upgrade succeeds and the existing namespaced OperatorGroup continues to work. -- **Upgrade test parameterization**: the existing upgrade test (`upgrade_suite_test.go`) hardcodes a namespaced OperatorGroup. Parameterize to also test with a global OperatorGroup. +Add a `deploy-olm-allnamespaces` Makefile target using `operator-sdk run bundle --install-mode AllNamespaces`. +Add a Prow presubmit job that deploys with this target and runs the existing e2e suite unchanged. +This gives immediate signal before writing any new test code. -### Upgrade Path +### Phase 3: AllNamespaces Install E2E Tests -Upgrading from a prior OADP version (OwnNamespace-only) to the version with this change is seamless. -Adding `AllNamespaces: true` to the installModes is a safe superset change — OLM does not reject the upgrade. -The customer's existing namespaced OperatorGroup continues to work. -No customer action is required unless they want to switch to AllNamespaces mode. +Test scenarios specific to AllNamespaces behavior: + +- **AllNamespaces fresh install**: verify CSV Succeeded, `WATCH_NAMESPACE` = pod namespace, DPA reconciles, Velero deploys. +- **OwnNamespace to AllNamespaces migration**: install OwnNamespace, swap OperatorGroup to global, verify operator re-deploys and existing DPA continues functioning. +- **AllNamespaces to OwnNamespace rollback**: reverse the migration, verify operator recovers. +- **Upgrade to dual-mode CSV**: upgrade from a prior version (OwnNamespace-only) to the new version, verify the existing namespaced OperatorGroup continues to work. +- **Upgrade test parameterization**: the existing upgrade test hardcodes a namespaced OperatorGroup — parameterize to also test with a global OperatorGroup. -### Migration (OwnNamespace to AllNamespaces) +### Phase 4: Migration Documentation -Since both install modes are supported in the same CSV, migration is a single step: swap the OperatorGroup. -No channel switch or CSV change is needed. +Since both modes coexist in the same CSV, migration is a single step: swap the OperatorGroup. 1. Delete the existing namespaced OperatorGroup. -2. Create a global OperatorGroup in `openshift-adp`. +2. Create a global OperatorGroup in `openshift-adp` (`spec: {}`). 3. OLM re-deploys the operator. Behavior is identical. -Rollback is the reverse: delete the global OperatorGroup, create a namespaced one. -No CSV downgrade is needed — the same CSV supports both modes. - -Brief operator unavailability occurs during the OperatorGroup swap (seconds to approximately one minute). +Rollback is the reverse. No CSV downgrade is needed. +Brief operator unavailability occurs during the swap (seconds to ~1 minute). In-flight backups or restores should be completed before migration. +## Upgrade Path + +Upgrading from a prior OADP version (OwnNamespace-only CSV) to the version with this change is seamless. +Adding `AllNamespaces: true` is a safe superset change — OLM does not reject the upgrade. +The customer's existing namespaced OperatorGroup continues to work. +No customer action is required unless they want to switch to AllNamespaces mode. + ## Alternatives Considered ### Two separate channels (one per install mode) -A separate AllNamespaces channel with its own CSV was considered. -This was rejected because a single CSV can support both install modes simultaneously — OLM uses the OperatorGroup to determine the active mode. -The single-CSV approach eliminates the need for two channels, two bundles, kustomize overlays, and catalog changes. -It also avoids the OLM upgrade deadlock that would occur if a future release dropped OwnNamespace support. +Rejected. A single CSV supports both install modes — OLM uses the OperatorGroup to determine the active mode. +Two channels would require separate bundles, kustomize overlays, catalog changes, and cross-channel update graphs. +It would also create an OLM upgrade deadlock if a future release dropped OwnNamespace support. ### Two separate OLM packages -A separate package (e.g., `oadp-operator-allnamespaces`) would provide clean separation but requires customers to uninstall and reinstall to migrate. -The single-CSV approach avoids this entirely — customers only swap their OperatorGroup. +Rejected. Requires customers to uninstall and reinstall to migrate. +The single-CSV approach requires only an OperatorGroup swap. ### Handle empty WATCH_NAMESPACE in Go code -Instead of switching the env var source to `metadata.namespace`, the operator could detect an empty `WATCH_NAMESPACE` and fall back to the pod's own namespace at runtime. -This was rejected because it would require Go code changes, defeating the CSV-only design goal. -It would also introduce a runtime behavior difference between the two install modes. +Rejected. Would require Go code changes and introduce a runtime behavior difference between install modes. +Sourcing from `metadata.namespace` achieves the same result at the CSV level. ## Security Considerations -The `velero` ServiceAccount's ClusterRole grants near-cluster-admin permissions (`apiGroups: ['*'], resources: ['*']`). -These permissions are declared as `clusterPermissions` in the CSV and bound via `ClusterRoleBinding`, which is cluster-scoped regardless of the OperatorGroup's install mode. -The `OwnNamespace` OperatorGroup does not restrict or contain these permissions — the RBAC posture is identical in both `OwnNamespace` and `AllNamespaces` modes. +The `velero` SA's ClusterRole grants near-cluster-admin permissions (`apiGroups: ['*'], resources: ['*']`). +These are bound via `ClusterRoleBinding`, which is cluster-scoped regardless of install mode. +The `OwnNamespace` OperatorGroup does not restrict these permissions — the RBAC posture is identical in both modes. -The new namespace-scoped `permissions` entries for `non-admin-controller` and `velero` add minimal RBAC: configmaps, leases, and events (standard leader-election permissions). -In AllNamespaces mode, OLM promotes these to ClusterRoles/ClusterRoleBindings. -These are strictly less permissive than the existing `clusterPermissions` for those SAs. +The new `permissions` entries add minimal RBAC (configmaps, leases, events) and are strictly less permissive than the existing `clusterPermissions` for those SAs. +In AllNamespaces mode, OLM promotes these to ClusterRoles. -Since the operator's runtime behavior is unchanged (still watches only its own namespace), the actual security posture does not change. -A security review of the velero SA permissions is recommended as a general hygiene item, independent of this install mode change. +A security review of the velero SA permissions is recommended as a general hygiene item, independent of this change. ## Compatibility -- Existing `OwnNamespace` installations are fully backward-compatible. The CSV supports both modes, so no OperatorGroup change is required to continue running as OwnNamespace. -- Upgrading from a prior release to the version with this change does not alter the customer's install mode — their existing namespaced OperatorGroup stays in place. +- Existing OwnNamespace installations are fully backward-compatible. No OperatorGroup change required. +- Upgrade from prior releases does not alter the install mode. - Migration to AllNamespaces is optional and requires an explicit OperatorGroup swap. -- No Go code changes are required. The operator binary is identical. -- Runtime behavior is identical in both modes: `WATCH_NAMESPACE` always resolves to the operator's own namespace. -- Under OwnNamespace, `metadata.namespace` and `olm.targetNamespaces` resolve to the same value — the change is transparent. -- Minimum OpenShift version: the operator functions in both modes on any supported OpenShift version. The `suggested-namespace` annotation in OperatorHub requires OpenShift 4.14+ for AllNamespaces namespace selection. +- Under OwnNamespace, `metadata.namespace` and `olm.targetNamespaces` resolve to the same value. +- Minimum OpenShift version for AllNamespaces namespace selection in OperatorHub: 4.14. ## Open Issues -1. **`operator-sdk generate bundle` override**: The SDK automatically replaces `metadata.namespace` with `metadata.annotations['olm.targetNamespaces']` in the generated CSV. -This substitution is hardcoded and cannot be disabled. -A post-generation patch step (e.g., `sed` in the Makefile `bundle` target) is required. -This must survive the `bundle-isupdated` CI check, which regenerates the bundle and diffs the result. +1. **`operator-sdk generate bundle` override**: The SDK hardcodes the `olm.targetNamespaces` substitution. A post-generation patch is required and must survive the `bundle-isupdated` CI check. -2. **Minimum version**: Which OADP release will include this change? +2. **Target OADP release**: Which version will include this change? -## Future Enhancements +## Validation -These are out of scope for this proposal but are natural follow-on work: +Tested on OpenShift 4.22.0-ec.3 (2026-08-13). Both install modes validated with the same bundle. +See [full test log](https://hackmd.io/MVRrs4zTTHiwGcxfRUwhBA). -### Decouple OPERATOR_NAMESPACE from WATCH_NAMESPACE +| Test | Mode | Result | +|---|---|---| +| CSV install | AllNamespaces | PASS | +| CSV install | OwnNamespace | PASS | +| WATCH_NAMESPACE value | Both | `openshift-adp` | +| DPA reconciliation + Velero deploy | Both | PASS | +| All controllers started | Both | PASS | -If a future requirement is for the operator to actually watch all namespaces (or a different namespace), `WATCH_NAMESPACE` would need to be set to empty or to a different value than the operator's own namespace. -In that case, a separate `OPERATOR_NAMESPACE` env var (sourced from `metadata.namespace`) would be needed so the operator knows its own namespace for PSA labeling, STS credential setup, CLI/VMDP downloads, and sub-controller configuration. +## Future Enhancements -### Handle Empty WATCH_NAMESPACE for Cluster-Wide Watching +### Cluster-Wide Watching -If `WATCH_NAMESPACE` is set to empty (to watch all namespaces), the controller-runtime manager's cache config must be updated to omit `DefaultNamespaces` instead of inserting an empty-string key. -The `getWatchNamespace()` function in `cmd/main.go` would also need to treat empty/unset as valid rather than logging an error. -DPA singleton enforcement scope would need a design decision: one DPA globally or one per namespace (multi-tenant Velero). +If a future requirement is for the operator to watch all namespaces, `WATCH_NAMESPACE` would need to be empty. +This requires a separate `OPERATOR_NAMESPACE` env var so the operator knows its own namespace for PSA labeling, STS, CLI/VMDP setup. +It also requires changes to the cache configuration and a design decision on DPA singleton scope (one globally or one per namespace). From 9139fb7073d6cf3ce1eb4a6e737f2ba39ceef5f3 Mon Sep 17 00:00:00 2001 From: Joseph Date: Fri, 21 Aug 2026 18:29:16 -0700 Subject: [PATCH 12/13] docs: expand background with OLMv0/OLMv1 context and improve readability Add structured OLMv0 five-resource model explanation, OLMv1 rationale for AllNamespaces-only GA support, and TP status of OwnNamespace in OLMv1. Convert dense prose sections to bullet lists throughout. Co-Authored-By: Claude Sonnet 4.6 (1M context) --- .../oadp-operator.clusterserviceversion.yaml | 70 +++++++- .../oadp-operator.clusterserviceversion.yaml | 2 +- .../allnamespaces-install-mode_design.md | 166 +++++++++++------- 3 files changed, 171 insertions(+), 67 deletions(-) diff --git a/bundle/manifests/oadp-operator.clusterserviceversion.yaml b/bundle/manifests/oadp-operator.clusterserviceversion.yaml index adb82f8e304..c23ff014e4f 100644 --- a/bundle/manifests/oadp-operator.clusterserviceversion.yaml +++ b/bundle/manifests/oadp-operator.clusterserviceversion.yaml @@ -1102,7 +1102,7 @@ spec: - name: WATCH_NAMESPACE valueFrom: fieldRef: - fieldPath: metadata.annotations['olm.targetNamespaces'] + fieldPath: metadata.namespace - name: FS_PV_HOSTPATH - name: PLUGINS_HOSTPATH - name: RELATED_IMAGE_VELERO @@ -1181,6 +1181,39 @@ spec: - emptyDir: {} name: tmp-dir permissions: + - rules: + - apiGroups: + - "" + resources: + - configmaps + verbs: + - get + - list + - watch + - create + - update + - patch + - delete + - apiGroups: + - coordination.k8s.io + resources: + - leases + verbs: + - get + - list + - watch + - create + - update + - patch + - delete + - apiGroups: + - "" + resources: + - events + verbs: + - create + - patch + serviceAccountName: non-admin-controller - rules: - apiGroups: - "" @@ -1214,6 +1247,39 @@ spec: - create - patch serviceAccountName: openshift-adp-controller-manager + - rules: + - apiGroups: + - "" + resources: + - configmaps + verbs: + - get + - list + - watch + - create + - update + - patch + - delete + - apiGroups: + - coordination.k8s.io + resources: + - leases + verbs: + - get + - list + - watch + - create + - update + - patch + - delete + - apiGroups: + - "" + resources: + - events + verbs: + - create + - patch + serviceAccountName: velero strategy: deployment installModes: - supported: true @@ -1222,7 +1288,7 @@ spec: type: SingleNamespace - supported: false type: MultiNamespace - - supported: false + - supported: true type: AllNamespaces keywords: - disaster backup diff --git a/config/manifests/bases/oadp-operator.clusterserviceversion.yaml b/config/manifests/bases/oadp-operator.clusterserviceversion.yaml index 48ecd4a15b9..e4cffe41803 100644 --- a/config/manifests/bases/oadp-operator.clusterserviceversion.yaml +++ b/config/manifests/bases/oadp-operator.clusterserviceversion.yaml @@ -467,7 +467,7 @@ spec: type: SingleNamespace - supported: false type: MultiNamespace - - supported: false + - supported: true type: AllNamespaces keywords: - disaster backup diff --git a/docs/design/allnamespaces-install-mode_design.md b/docs/design/allnamespaces-install-mode_design.md index b0828bdb6a1..b271662ebb3 100644 --- a/docs/design/allnamespaces-install-mode_design.md +++ b/docs/design/allnamespaces-install-mode_design.md @@ -1,34 +1,74 @@ -# AllNamespaces Install Mode for OADP Operator +# Enable AllNamespaces Install Mode for OADP Operator ## Abstract -Enable `AllNamespaces` install mode alongside the existing `OwnNamespace` mode in a single CSV. -The operator's runtime behavior is unchanged — it watches only the namespace it is deployed in. -No Go code changes are required. +- Enable `AllNamespaces` install mode alongside the existing `OwnNamespace` mode in a single CSV. +- The operator's runtime behavior is unchanged — it watches only the namespace it is deployed in. +- No Go code changes are required. ## Background -OADP is installed via OLM with only `OwnNamespace` install mode supported. -At runtime, the `WATCH_NAMESPACE` environment variable controls which namespace the controller-runtime cache monitors. +### OLMv0 — Current model -The OLM-deployed CSV sources `WATCH_NAMESPACE` from the `olm.targetNamespaces` annotation, which OLM sets based on the OperatorGroup. -In OwnNamespace mode this resolves to the operator's namespace. -In AllNamespaces mode this would be empty — breaking PSA labeling, STS credential flow, CLI/VMDP setup, and the cache configuration. +OLMv0 uses a five-resource model: -The fix is to source `WATCH_NAMESPACE` from `metadata.namespace` (the pod's own namespace via Kubernetes downward API) instead. -This resolves to the same value under OwnNamespace (backward-compatible) and correctly resolves to the pod's namespace under AllNamespaces. +- **`CatalogSource`** — where to find operators (catalog image) +- **`OperatorGroup`** — which namespaces the operator watches + RBAC generation +- **`Subscription`** — install request +- **`InstallPlan`** — auto-generated execution plan +- **`ClusterServiceVersion` (CSV)** — the operator's metadata, permissions, and deployment spec -Additionally, OLM requires every ServiceAccount declared in `clusterPermissions` to have a corresponding `permissions` (namespace-scoped Role) entry when running in AllNamespaces mode. -Two of the three OADP ServiceAccounts currently lack this entry. +OLMv0 supports four install modes: `AllNamespaces`, `OwnNamespace`, `SingleNamespace`, and `MultiNamespace`. This multi-tenancy model allowed multiple independent operator instances in different namespaces, each watching its own scope. OADP currently uses `OwnNamespace`, meaning the operator only watches CRs in the namespace where it is installed. + +### OLMv1 — The new model + +OLMv1 is not a version bump — it is a ground-up redesign that consolidates the five OLMv0 resources into two: + +- **`ClusterCatalog`** — replaces `CatalogSource` +- **`ClusterExtension`** — replaces `OperatorGroup` + `Subscription` + `InstallPlan` + `CSV` + +It is declarative, GitOps-friendly, and uses a cluster-admin security model instead of namespace-scoped ServiceAccount RBAC. + +**OLMv1 GA only supports AllNamespaces mode.** + +The rationale: +1. **CRDs are cluster-scoped singletons.** Only one definition of a CRD can exist per cluster. OLMv0's multi-tenancy promise — that multiple operator instances in different namespaces could each own their own CRDs — was architecturally flawed. +2. **Dependency resolution requires a global view.** OLMv1's explicit dependency model cannot work at namespace scope. +3. **PSA and security are simpler at cluster scope.** Managed platforms (ROSA, OSD) operate cluster-admin anyway. + + + +`OwnNamespace` and `SingleNamespace` are available only as unsupported Tech Preview behind the `TechPreviewNoUpgrade` feature gate — a one-way toggle that permanently blocks cluster upgrades: + +- OCP 4.21 release notes explicitly state these modes "continued as a Technology Preview feature" and are "not recommended for production use." +- A planned GA promotion in 4.22 was reversed; the Operator Framework team confirmed the feature was moved back to alpha/TP status with no current plans to re-promote a namespace-scoping model. + +They are not a viable option for production operators. + +### OADP Operator + +OADP is installed via OLM with only `OwnNamespace` install mode supported. At runtime, the `WATCH_NAMESPACE` environment variable controls which namespace the controller-runtime cache monitors. + +The OLM-deployed CSV sources `WATCH_NAMESPACE` from the `olm.targetNamespaces` annotation, which OLM sets based on the OperatorGroup: + +- In `OwnNamespace` mode this resolves to the operator's namespace. +- In `AllNamespaces` mode this would be `""` (empty) — breaking PSA labeling, STS credential flow, CLI/VMDP setup, and the cache configuration. + +The fix is to source `WATCH_NAMESPACE` from `metadata.namespace` (the pod's own namespace via Kubernetes downward API) instead. This resolves to the same value under OwnNamespace (backward-compatible) and correctly resolves to the pod's namespace under AllNamespaces. + +Additionally, OLM requires every ServiceAccount declared in `clusterPermissions` to have a corresponding `permissions` (namespace-scoped Role) entry when running in AllNamespaces mode. Two of the three OADP ServiceAccounts currently lack this entry. Both issues are CSV metadata problems, not Go code problems. ### OLMv1 Alignment -AllNamespaces is the strategically correct direction. -At OLMv1 GA (OCP 4.18), only AllNamespaces operators were installable. -OwnNamespace support was added later as backward-compatibility (Tech Preview in OCP 4.19, GA in OCP 4.22). -Enabling AllNamespaces now positions OADP for OLMv1 readiness. +AllNamespaces is the strategically correct direction: + +- OLMv1 GA (OCP 4.18) shipped with AllNamespaces-only support. +- OwnNamespace was added as Tech Preview in OCP 4.19 and remains TP. +- Enabling AllNamespaces now positions OADP for OLMv1 readiness without requiring disruptive changes to runtime behavior — the operator continues watching only its own namespace. + +> **Note:** AllNamespaces here refers to the install mode (how OLM deploys the operator), not the operator's watch scope. ## Goals @@ -76,8 +116,7 @@ installModes: type: AllNamespaces ``` -Adding installMode support is a safe superset change in OLM — it never blocks upgrades. -Existing customers with a namespaced OperatorGroup are unaffected; their install mode stays OwnNamespace. +Adding installMode support is a safe superset change in OLM — it never blocks upgrades. Existing customers with a namespaced OperatorGroup are unaffected; their install mode stays OwnNamespace. ### Change 2: WATCH_NAMESPACE Source @@ -95,20 +134,18 @@ Existing customers with a namespaced OperatorGroup are unaffected; their install fieldPath: metadata.namespace ``` -`operator-sdk generate bundle` automatically rewrites `metadata.namespace` to `olm.targetNamespaces` during bundle generation. -This substitution is hardcoded in the SDK and cannot be disabled. -A post-generation patch step is required to restore `metadata.namespace`. - -OLM still sets the `olm.targetNamespaces` annotation on the pod template regardless — it is simply unused by the operator's env var. +- `operator-sdk generate bundle` automatically rewrites `metadata.namespace` to `olm.targetNamespaces` during bundle generation. +- This substitution is hardcoded in the SDK and cannot be disabled. +- A post-generation patch step is required to restore `metadata.namespace`. +- OLM still sets the `olm.targetNamespaces` annotation on the pod template regardless — it is simply unused by the operator's env var. ### Change 3: Add Permissions for Missing ServiceAccounts -The CSV declares `clusterPermissions` for three SAs but only `openshift-adp-controller-manager` has a `permissions` entry. -Add `permissions` entries for `non-admin-controller` and `velero` with leader-election rules (configmaps, leases, events). +The CSV declares `clusterPermissions` for three SAs but only `openshift-adp-controller-manager` has a `permissions` entry. Add `permissions` entries for `non-admin-controller` and `velero` with leader-election rules (configmaps, leases, events). -The `velero` SA does not actually perform leader election — these are placeholder rules required solely to satisfy OLM's SA creation requirement. -In AllNamespaces mode, OLM promotes these to ClusterRoles/ClusterRoleBindings via `ensureSingletonRBAC`. -In OwnNamespace mode, they remain namespace-scoped Roles/RoleBindings. +- The `velero` SA does not actually perform leader election — these are placeholder rules required solely to satisfy OLM's SA creation requirement. +- In AllNamespaces mode, OLM promotes these to ClusterRoles/ClusterRoleBindings via `ensureSingletonRBAC`. +- In OwnNamespace mode, they remain namespace-scoped Roles/RoleBindings. ### OLM Behavior in AllNamespaces Mode @@ -120,9 +157,10 @@ These OLM behaviors do not affect operator functionality but should be understoo ### OperatorHub User Experience -When both install modes are enabled, OperatorHub (OpenShift 4.14+) presents install mode radio buttons and a namespace dropdown. -The existing `suggested-namespace: openshift-adp` annotation defaults the namespace to `openshift-adp` in both modes. -For fresh AllNamespaces installs, the Console automatically creates the namespace, a global OperatorGroup, and a Subscription. +When both install modes are enabled, OperatorHub (OpenShift 4.14+) presents install mode radio buttons and a namespace dropdown: + +- The existing `suggested-namespace: openshift-adp` annotation defaults the namespace to `openshift-adp` in both modes. +- For fresh AllNamespaces installs, the Console automatically creates the namespace, a global OperatorGroup, and a Subscription. This follows the pattern used by the Loki Operator (`openshift-operators-redhat`) and OpenShift Serverless (`openshift-serverless`). @@ -137,15 +175,15 @@ This follows the pattern used by the Loki Operator (`openshift-operators-redhat` ### Phase 1: CSV Changes -Apply the three changes described in Detailed Design. -The `bundle` Makefile target needs a post-generation step to replace `olm.targetNamespaces` with `metadata.namespace`. -This must survive the `bundle-isupdated` CI check. +- Apply the three changes described in Detailed Design. +- The `bundle` Makefile target needs a post-generation step to replace `olm.targetNamespaces` with `metadata.namespace`. +- This must survive the `bundle-isupdated` CI check. ### Phase 2: Deploy + CI for AllNamespaces -Add a `deploy-olm-allnamespaces` Makefile target using `operator-sdk run bundle --install-mode AllNamespaces`. -Add a Prow presubmit job that deploys with this target and runs the existing e2e suite unchanged. -This gives immediate signal before writing any new test code. +- Add a `deploy-olm-allnamespaces` Makefile target using `operator-sdk run bundle --install-mode AllNamespaces`. +- Add a Prow presubmit job that deploys with this target and runs the existing e2e suite unchanged. +- This gives immediate signal before writing any new test code. ### Phase 3: AllNamespaces Install E2E Tests @@ -165,45 +203,44 @@ Since both modes coexist in the same CSV, migration is a single step: swap the O 2. Create a global OperatorGroup in `openshift-adp` (`spec: {}`). 3. OLM re-deploys the operator. Behavior is identical. -Rollback is the reverse. No CSV downgrade is needed. -Brief operator unavailability occurs during the swap (seconds to ~1 minute). -In-flight backups or restores should be completed before migration. +Notes: +- Rollback is the reverse. No CSV downgrade is needed. +- Brief operator unavailability occurs during the swap (seconds to ~1 minute). +- In-flight backups or restores should be completed before migration. ## Upgrade Path -Upgrading from a prior OADP version (OwnNamespace-only CSV) to the version with this change is seamless. -Adding `AllNamespaces: true` is a safe superset change — OLM does not reject the upgrade. -The customer's existing namespaced OperatorGroup continues to work. -No customer action is required unless they want to switch to AllNamespaces mode. +- Upgrading from a prior OADP version (OwnNamespace-only CSV) to the version with this change is seamless. +- Adding `AllNamespaces: true` is a safe superset change — OLM does not reject the upgrade. +- The customer's existing namespaced OperatorGroup continues to work. +- No customer action is required unless they want to switch to AllNamespaces mode. ## Alternatives Considered ### Two separate channels (one per install mode) -Rejected. A single CSV supports both install modes — OLM uses the OperatorGroup to determine the active mode. -Two channels would require separate bundles, kustomize overlays, catalog changes, and cross-channel update graphs. -It would also create an OLM upgrade deadlock if a future release dropped OwnNamespace support. +Rejected: +- A single CSV supports both install modes — OLM uses the OperatorGroup to determine the active mode. +- Two channels would require separate bundles, kustomize overlays, catalog changes, and cross-channel update graphs. +- It would also create an OLM upgrade deadlock if a future release dropped OwnNamespace support. ### Two separate OLM packages -Rejected. Requires customers to uninstall and reinstall to migrate. -The single-CSV approach requires only an OperatorGroup swap. +Rejected: +- Requires customers to uninstall and reinstall to migrate. +- The single-CSV approach requires only an OperatorGroup swap. ### Handle empty WATCH_NAMESPACE in Go code -Rejected. Would require Go code changes and introduce a runtime behavior difference between install modes. -Sourcing from `metadata.namespace` achieves the same result at the CSV level. +Rejected: +- Would require Go code changes and introduce a runtime behavior difference between install modes. +- Sourcing from `metadata.namespace` achieves the same result at the CSV level. ## Security Considerations -The `velero` SA's ClusterRole grants near-cluster-admin permissions (`apiGroups: ['*'], resources: ['*']`). -These are bound via `ClusterRoleBinding`, which is cluster-scoped regardless of install mode. -The `OwnNamespace` OperatorGroup does not restrict these permissions — the RBAC posture is identical in both modes. - -The new `permissions` entries add minimal RBAC (configmaps, leases, events) and are strictly less permissive than the existing `clusterPermissions` for those SAs. -In AllNamespaces mode, OLM promotes these to ClusterRoles. - -A security review of the velero SA permissions is recommended as a general hygiene item, independent of this change. +- The `velero` SA's ClusterRole grants near-cluster-admin permissions (`apiGroups: ['*'], resources: ['*']`). These are bound via `ClusterRoleBinding`, which is cluster-scoped regardless of install mode. The `OwnNamespace` OperatorGroup does not restrict these permissions — the RBAC posture is identical in both modes. +- The new `permissions` entries add minimal RBAC (configmaps, leases, events) and are strictly less permissive than the existing `clusterPermissions` for those SAs. In AllNamespaces mode, OLM promotes these to ClusterRoles. +- A security review of the velero SA permissions is recommended as a general hygiene item, independent of this change. ## Compatibility @@ -216,7 +253,6 @@ A security review of the velero SA permissions is recommended as a general hygie ## Open Issues 1. **`operator-sdk generate bundle` override**: The SDK hardcodes the `olm.targetNamespaces` substitution. A post-generation patch is required and must survive the `bundle-isupdated` CI check. - 2. **Target OADP release**: Which version will include this change? ## Validation @@ -236,6 +272,8 @@ See [full test log](https://hackmd.io/MVRrs4zTTHiwGcxfRUwhBA). ### Cluster-Wide Watching -If a future requirement is for the operator to watch all namespaces, `WATCH_NAMESPACE` would need to be empty. -This requires a separate `OPERATOR_NAMESPACE` env var so the operator knows its own namespace for PSA labeling, STS, CLI/VMDP setup. -It also requires changes to the cache configuration and a design decision on DPA singleton scope (one globally or one per namespace). +If a future requirement is for the operator to watch all namespaces, `WATCH_NAMESPACE` would need to be empty. This requires: + +- A separate `OPERATOR_NAMESPACE` env var so the operator knows its own namespace for PSA labeling, STS, CLI/VMDP setup. +- Changes to the cache configuration. +- A design decision on DPA singleton scope (one globally or one per namespace). From 23101b017715bbc26bbaeb33b2444b35447f86dc Mon Sep 17 00:00:00 2001 From: Joseph Date: Fri, 21 Aug 2026 18:49:30 -0700 Subject: [PATCH 13/13] docs: add sources and address unsupported claims in design doc - Link OLMv1 rationale to openshift/enhancements#1849 - Source OwnNamespace TP to OCPSTRAT-1711 (OCP 4.19) - Source GA reversion to OPRUN-4514 and operator-controller#2568 - Explain TechPreviewNoUpgrade gate behavior precisely - Attribute "architecturally flawed" claim to enhancement proposal - Clarify ensureSingletonRBAC is an OLM internal reconciler - Note InterOperatorGroupOwnerConflict is a pre-existing constraint - Add transition sentence to OADP Operator subsection Co-Authored-By: Claude Sonnet 4.6 (1M context) --- .../allnamespaces-install-mode_design.md | 24 ++++++++----------- 1 file changed, 10 insertions(+), 14 deletions(-) diff --git a/docs/design/allnamespaces-install-mode_design.md b/docs/design/allnamespaces-install-mode_design.md index b271662ebb3..9d0357c87bb 100644 --- a/docs/design/allnamespaces-install-mode_design.md +++ b/docs/design/allnamespaces-install-mode_design.md @@ -29,25 +29,22 @@ OLMv1 is not a version bump — it is a ground-up redesign that consolidates the It is declarative, GitOps-friendly, and uses a cluster-admin security model instead of namespace-scoped ServiceAccount RBAC. -**OLMv1 GA only supports AllNamespaces mode.** +**OLMv1 GA only supports AllNamespaces mode.** The rationale, as documented in the [OLMv1 single/own namespace enhancement proposal](https://github.com/openshift/enhancements/pull/1849): -The rationale: 1. **CRDs are cluster-scoped singletons.** Only one definition of a CRD can exist per cluster. OLMv0's multi-tenancy promise — that multiple operator instances in different namespaces could each own their own CRDs — was architecturally flawed. 2. **Dependency resolution requires a global view.** OLMv1's explicit dependency model cannot work at namespace scope. 3. **PSA and security are simpler at cluster scope.** Managed platforms (ROSA, OSD) operate cluster-admin anyway. - - -`OwnNamespace` and `SingleNamespace` are available only as unsupported Tech Preview behind the `TechPreviewNoUpgrade` feature gate — a one-way toggle that permanently blocks cluster upgrades: +`OwnNamespace` and `SingleNamespace` were added as Tech Preview in OCP 4.19 ([OCPSTRAT-1711](https://issues.redhat.com/browse/OCPSTRAT-1711)) behind the `TechPreviewNoUpgrade` feature gate — a cluster-wide toggle that, while enabled, blocks upgrading to the next OCP minor version: - OCP 4.21 release notes explicitly state these modes "continued as a Technology Preview feature" and are "not recommended for production use." -- A planned GA promotion in 4.22 was reversed; the Operator Framework team confirmed the feature was moved back to alpha/TP status with no current plans to re-promote a namespace-scoping model. +- A planned GA promotion was subsequently reversed. [OPRUN-4514](https://issues.redhat.com/browse/OPRUN-4514) ("Revert Single/Own Namespace promotion to GA") explicitly states the goal was to return the feature to `TPNU` status; the upstream revert is [operator-controller#2568](https://github.com/operator-framework/operator-controller/pull/2568). -They are not a viable option for production operators. +`OwnNamespace` and `SingleNamespace` are not a viable option for production operators. ### OADP Operator -OADP is installed via OLM with only `OwnNamespace` install mode supported. At runtime, the `WATCH_NAMESPACE` environment variable controls which namespace the controller-runtime cache monitors. +Against this backdrop, OADP today is installed via OLM with only `OwnNamespace` install mode supported. At runtime, the `WATCH_NAMESPACE` environment variable controls which namespace the controller-runtime cache monitors. The OLM-deployed CSV sources `WATCH_NAMESPACE` from the `olm.targetNamespaces` annotation, which OLM sets based on the OperatorGroup: @@ -65,7 +62,7 @@ Both issues are CSV metadata problems, not Go code problems. AllNamespaces is the strategically correct direction: - OLMv1 GA (OCP 4.18) shipped with AllNamespaces-only support. -- OwnNamespace was added as Tech Preview in OCP 4.19 and remains TP. +- OwnNamespace was added as Tech Preview in OCP 4.19 ([OCPSTRAT-1711](https://issues.redhat.com/browse/OCPSTRAT-1711)) and remains TP. - Enabling AllNamespaces now positions OADP for OLMv1 readiness without requiring disruptive changes to runtime behavior — the operator continues watching only its own namespace. > **Note:** AllNamespaces here refers to the install mode (how OLM deploys the operator), not the operator's watch scope. @@ -134,8 +131,7 @@ Adding installMode support is a safe superset change in OLM — it never blocks fieldPath: metadata.namespace ``` -- `operator-sdk generate bundle` automatically rewrites `metadata.namespace` to `olm.targetNamespaces` during bundle generation. -- This substitution is hardcoded in the SDK and cannot be disabled. +- `operator-sdk generate bundle` automatically rewrites `metadata.namespace` to `olm.targetNamespaces` during bundle generation. This substitution is hardcoded in the SDK and cannot be disabled via flags or configuration. - A post-generation patch step is required to restore `metadata.namespace`. - OLM still sets the `olm.targetNamespaces` annotation on the pod template regardless — it is simply unused by the operator's env var. @@ -144,15 +140,15 @@ Adding installMode support is a safe superset change in OLM — it never blocks The CSV declares `clusterPermissions` for three SAs but only `openshift-adp-controller-manager` has a `permissions` entry. Add `permissions` entries for `non-admin-controller` and `velero` with leader-election rules (configmaps, leases, events). - The `velero` SA does not actually perform leader election — these are placeholder rules required solely to satisfy OLM's SA creation requirement. -- In AllNamespaces mode, OLM promotes these to ClusterRoles/ClusterRoleBindings via `ensureSingletonRBAC`. -- In OwnNamespace mode, they remain namespace-scoped Roles/RoleBindings. +- In AllNamespaces mode, OLM promotes these namespace-scoped Role/RoleBinding entries to ClusterRoles/ClusterRoleBindings (via its internal `ensureSingletonRBAC` reconciler, which merges all operator permissions into a single cluster-scoped set). +- In OwnNamespace mode, they remain namespace-scoped Roles/RoleBindings — no change to existing behavior. ### OLM Behavior in AllNamespaces Mode These OLM behaviors do not affect operator functionality but should be understood: - **CSV copies**: OLM copies the CSV to every namespace. On large clusters, `OLMConfig.spec.features.disableCopiedCSVs: true` disables this. -- **CRD ownership**: The operator globally owns its CRDs. Customers running standalone upstream Velero alongside OADP would hit `InterOperatorGroupOwnerConflict`. +- **CRD ownership**: The operator globally owns its CRDs. This is a pre-existing constraint of OADP's cluster-scoped CRD ownership — customers running standalone upstream Velero alongside OADP would hit `InterOperatorGroupOwnerConflict` regardless of install mode. - **Dual installation prevention**: OLM prevents installing the operator in two namespaces with overlapping OperatorGroups. ### OperatorHub User Experience