Skip to content

✨ Support serviceaccount pull secrets #2005

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

tmshort
Copy link
Contributor

@tmshort tmshort commented Jun 3, 2025

Serviceaccounts reference pull secrets!

  • Determine our serviceaccount (via the new internal/shared/util/sa package).
  • Use a common pull_secret_controller
  • Update the pull_secret_controller to know about the service account
  • Update the pull_secret_controller to watch the namespace-local secrets
  • Update caching to include sa, and use filters for additional secrets
  • Add RBAC to access these secrets and sa
  • Update writing the auth.json file to handle dockercfg and dockerconfigjson
  • Update writing the auth.json file to include multiple secrets

Description

Reviewer Checklist

  • API Go Documentation
  • Tests: Unit Tests (and E2E Tests, if appropriate)
  • Comprehensive Commit Messages
  • Links to related GitHub Issue(s)

@tmshort tmshort requested a review from a team as a code owner June 3, 2025 19:39
@openshift-ci openshift-ci bot requested a review from oceanc80 June 3, 2025 19:39
Copy link

netlify bot commented Jun 3, 2025

Deploy Preview for olmv1 ready!

Name Link
🔨 Latest commit 2196977
🔍 Latest deploy log https://app.netlify.com/projects/olmv1/deploys/684352fc0c3dfb0008073d6a
😎 Deploy Preview https://deploy-preview-2005--olmv1.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@openshift-ci openshift-ci bot requested a review from OchiengEd June 3, 2025 19:39
@tmshort tmshort force-pushed the use-sa-pull-secret branch from d09d1b8 to 26246b6 Compare June 3, 2025 20:10
Copy link

codecov bot commented Jun 3, 2025

Codecov Report

Attention: Patch coverage is 56.85484% with 107 lines in your changes missing coverage. Please review.

Project coverage is 69.54%. Comparing base (8f81c23) to head (2196977).
Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
...ernal/shared/controllers/pull_secret_controller.go 66.66% 38 Missing and 11 partials ⚠️
cmd/catalogd/main.go 0.00% 41 Missing ⚠️
cmd/operator-controller/main.go 65.85% 10 Missing and 4 partials ⚠️
internal/shared/util/sa/serviceaccount.go 84.21% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2005      +/-   ##
==========================================
+ Coverage   69.17%   69.54%   +0.36%     
==========================================
  Files          79       79              
  Lines        7037     7144     +107     
==========================================
+ Hits         4868     4968     +100     
- Misses       1887     1889       +2     
- Partials      282      287       +5     
Flag Coverage Δ
e2e 43.50% <40.57%> (+0.49%) ⬆️
unit 59.65% <31.45%> (-0.40%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@tmshort
Copy link
Contributor Author

tmshort commented Jun 4, 2025

/hold
Until I do some additional manual testing.

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 4, 2025
@tmshort
Copy link
Contributor Author

tmshort commented Jun 4, 2025

/approve

Copy link

openshift-ci bot commented Jun 4, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tmshort

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 4, 2025
@tmshort tmshort force-pushed the use-sa-pull-secret branch from 26246b6 to 11579bc Compare June 4, 2025 16:15
Serviceaccounts reference pull secrets!

* Determine our serviceaccount (via the new internal/shared/util/sa package).
* Use a common pull_secret_controller
* Update the pull_secret_controller to know about the service account
* Update the pull_secret_controller to watch the namespace-local secrets
* Update caching to include sa, and use filters for additional secrets
* Add RBAC to access these secrets and sa
* Update writing the auth.json file to handle dockercfg and dockerconfigjson
* Update writing the auth.json file to include multiple secrets

Signed-off-by: Todd Short <[email protected]>
@tmshort tmshort force-pushed the use-sa-pull-secret branch from 11579bc to 1c0e4dd Compare June 4, 2025 18:05
@tmshort
Copy link
Contributor Author

tmshort commented Jun 4, 2025

/unhold

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 4, 2025
AuthFilePath string
}

func (r *PullSecretReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is req used anywhere other than logging? If not, I think I'd suggest dropping it (e.g. rename to _). And changing the logging to just generically say something like "reconciling pull secrets".

The name/namespace of the request without the type info might be a little confusing to show up in the log. But maybe we could log the events that pass our predicate in our predicate where we do have type information. That way there's still detail about what is causing our reconciler to be triggered?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interestingly... the logger has a lot of additional information added to it, so you can tell what type of resource triggered the event, I wrapped the log below to make it easier to read:

I0606 18:43:42.641045       1 pull_secret_controller.go:113] "found secret"
  logger="pull-secret-reconciler"
  controller="service-account-controller"
  controllerGroup=""
  controllerKind="ServiceAccount"
  ServiceAccount="olmv1-system/operator-controller-controller-manager"
  reconcileID="26404229-a2d3-495d-86ee-0da390a1e8f4"
  name="pull-dockercfg"
  namespace="olmv1-system"

Copy link
Contributor Author

@tmshort tmshort Jun 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And when a pull secret is modified:

I0606 18:58:10.831129       1 pull_secret_controller.go:113] "found secret"
  logger="pull-secret-reconciler"
  controller="pull-secret-controller"
  controllerGroup=""
  controllerKind="Secret"
  Secret="olmv1-system/pull-dockercfg"
  reconcileID="a783ca58-0213-4fe3-a19d-3ed7ca21f5ae"
  name="pull-dockercfg"
  namespace="olmv1-system"

Comment on lines +113 to +126
_, err := ctrl.NewControllerManagedBy(mgr).
For(&corev1.Secret{}).
Named("pull-secret-controller").
WithEventFilter(newSecretPredicate(r)).
Build(r)
if err != nil {
return err
}

_, err = ctrl.NewControllerManagedBy(mgr).
For(&corev1.ServiceAccount{}).
Named("service-account-controller").
WithEventFilter(newNamespacedPredicate(r.ServiceAccountKey)).
Build(r)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I'd setup two separate controllers. IIRC, there's a way to have a single controller with multiple watches. You may need to drop down to the lower-level controller package though (can't remember if For is required).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH, having two controllers actually seems the cleanest, in that we get different invocations that recognize ServiceAccount vs. Secret, and it's clear what triggered the reconcile (i.e. the logger clearly indicates what resource is the trigger). With a single controller, we have to somehow map the ServiceAccount to a set of Secrets (which we may not yet know about yet), which feels kinda kludgy. Otherwise, the Secret controller is getting a ServiceAccount for the request, rather than a Secret.

@tmshort
Copy link
Contributor Author

tmshort commented Jun 6, 2025

lint doesn't like my logging trick...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants