Skip to content

OCPBUGS-65622: add controller for downloads service account deletion logic#1111

Open
ehearne-redhat wants to merge 8 commits intoopenshift:mainfrom
ehearne-redhat:downloads-service-account-controller
Open

OCPBUGS-65622: add controller for downloads service account deletion logic#1111
ehearne-redhat wants to merge 8 commits intoopenshift:mainfrom
ehearne-redhat:downloads-service-account-controller

Conversation

@ehearne-redhat
Copy link
Contributor

@ehearne-redhat ehearne-redhat commented Feb 14, 2026

@yapei pointed out correctly that the service account downloads is not deleted once the pod downloads completes/deletes.

This PR addresses this by adding a controller, similar to how downloads deployment has its own controller in pkg/console/controllers/downloadsdeployment/controller.go.

It also moves the service account in bindata/assets/deployments/downloads-deployment.yaml down to spec.spec so it is placed correctly.

The service account is now managed dynamically, so static reference to it in manifests/06-sa.yaml has been removed. Now, it has its own file in bindata/assets/serviceaccounts/downloads-sa.yaml .

Permissions for the controller to perform CRUD on service account objects has been given to console-operator ClusterRole.

Summary by CodeRabbit

  • New Features

    • Automated provisioning and lifecycle controllers for console and downloads service accounts.
    • New standalone service account manifests added for console and downloads; removed duplicate in the prior manifest.
  • Infrastructure & Operations

    • RBAC extended to permit operator management of service accounts.
    • Downloads pods now run with the downloads service account.
  • Tests

    • Deployment tests updated to reflect the downloads service account.

@openshift-ci openshift-ci bot requested review from jhadvig and spadgett February 14, 2026 08:46
@ehearne-redhat ehearne-redhat changed the title OCPBUGS-65622 - add controller for downloads service account deletion logic OCPBUGS-65622: add controller for downloads service account deletion logic Feb 14, 2026
@openshift-ci-robot openshift-ci-robot added jira/severity-critical Referenced Jira bug's severity is critical for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. labels Feb 14, 2026
@openshift-ci-robot
Copy link
Contributor

@ehearne-redhat: This pull request references Jira Issue OCPBUGS-65622, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.22.0) matches configured target version for branch (4.22.0)
  • bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @yapei

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

@yapei pointed out correctly that the service account downloads is not deleted once the pod downloads completes/deletes.

This PR addresses this by adding a controller, similar to how downloads deployment has its own controller in pkg/console/controllers/downloadsdeployment/controller.go.

It also moves the service account in bindata/assets/deployments/downloads-deployment.yaml down to spec.spec so it is placed correctly.

The service account is now managed dynamically, so static reference to it in manifests/06-sa.yaml has been removed. Now, it has its own file in bindata/assets/serviceaccounts/downloads-sa.yaml .

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot requested a review from yapei February 14, 2026 08:47
@ehearne-redhat ehearne-redhat force-pushed the downloads-service-account-controller branch from 5c08afe to 7ba9a63 Compare February 15, 2026 22:34
@openshift-ci-robot
Copy link
Contributor

@ehearne-redhat: This pull request references Jira Issue OCPBUGS-65622, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.22.0) matches configured target version for branch (4.22.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @yapei

Details

In response to this:

@yapei pointed out correctly that the service account downloads is not deleted once the pod downloads completes/deletes.

This PR addresses this by adding a controller, similar to how downloads deployment has its own controller in pkg/console/controllers/downloadsdeployment/controller.go.

It also moves the service account in bindata/assets/deployments/downloads-deployment.yaml down to spec.spec so it is placed correctly.

The service account is now managed dynamically, so static reference to it in manifests/06-sa.yaml has been removed. Now, it has its own file in bindata/assets/serviceaccounts/downloads-sa.yaml .

Permissions for the controller to perform CRUD on service account objects has been given to console-operator ClusterRole.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@ehearne-redhat ehearne-redhat force-pushed the downloads-service-account-controller branch from 7ba9a63 to 0504779 Compare February 16, 2026 09:37
Copy link
Member

@jhadvig jhadvig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ehearne-redhat for the PR. I would suggest to implement the SA logic into the downloadsdeployment controller since the logic is basically similar. Maybe we could rename it just to downloads folder wise.
We might need to be cautions with the DownloadsDeploymentSync condition if renaming, cause since we would need to add it into the StaleConditionsController.

@ehearne-redhat
Copy link
Contributor Author

@jhadvig thanks for the review! I may also need to add manual garbage collection for the downloads service account so that e2e-azure-ovn-upgrade passes.

@jhadvig
Copy link
Member

jhadvig commented Feb 17, 2026

@ehearne-redhat thanks for the changes. I was thinking about the change since we are changing the management process for downloads SA we should do a similar thing for the console SA.
Here is what I would suggest:

  • move the SA for console and downloads from /manifest folder into the bindata, so they will be managed by the console-operator instead of the CVO
  • create a new serviceaccountcontroller which will be responsible for creating /deleting SA. Ideally this controller should be in instantiated twice, one for console, second for download. Similarly like we are doing it with the Services where we are using the same controller.
  • leave the downloadsDeployment as is

I apologise for not suggesting this approach on the first pass, but I believe this is the idiomatic implementation.

@ehearne-redhat
Copy link
Contributor Author

@jhadvig that's okay - I have moved everything over as asked. I am trying to figure out why the controller value in objectreference is being set to true when multiple ownerrefs exist, even though the code should ensure that is not the case. 🤔

@ehearne-redhat
Copy link
Contributor Author

/retest

@ehearne-redhat ehearne-redhat force-pushed the downloads-service-account-controller branch from ab9c36f to 9a3e679 Compare February 19, 2026 10:54
@ehearne-redhat
Copy link
Contributor Author

/retest

@ehearne-redhat
Copy link
Contributor Author

Hey @jhadvig - all tests are passing. Could you take a look again, and let me know if this was what you were looking for? Then, I can tidy up commits. :)

Copy link
Member

@jhadvig jhadvig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ehearne-redhat thank you so much for addressing the comments. Adding a few suggestiongs. PTAL

@coderabbitai
Copy link

coderabbitai bot commented Mar 4, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Note

.coderabbit.yaml has unrecognized properties

CodeRabbit is using all valid settings from your configuration. Unrecognized properties (listed below) have been ignored and may indicate typos or deprecated fields that can be removed.

⚠️ Parsing warnings (1)
Validation error: Unrecognized key(s) in object: 'version', 'path_filters'
⚙️ Configuration instructions
  • Please see the configuration documentation for more information.
  • You can also validate your configuration using the online YAML validator.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Walkthrough

Adds ServiceAccount manifests and a ServiceAccountSyncController, moves the downloads Deployment's serviceAccountName into the pod template spec, extends ClusterRole permissions for serviceaccounts, exposes service account name constants, wires controllers into startup, and updates a test.

Changes

Cohort / File(s) Summary
ServiceAccount Manifests
bindata/assets/serviceaccounts/console-sa.yaml, bindata/assets/serviceaccounts/downloads-sa.yaml
Add dedicated console and downloads ServiceAccount YAMLs with include.release.openshift.io annotations and capability metadata.
Deployment Configuration
bindata/assets/deployments/downloads-deployment.yaml
Moved serviceAccountName: "downloads" from Deployment.spec into Deployment.spec.template.spec so pods use the downloads SA.
RBAC & Manifest Cleanup
manifests/03-rbac-role-cluster.yaml, manifests/06-sa.yaml
Added ClusterRole rules for serviceaccounts (list/watch/get/create; update/delete for resourceNames console,downloads); removed inline console and downloads SA entries from manifests/06-sa.yaml.
API Constants
pkg/api/api.go
Added exported constants OpenShiftConsoleDownloadsServiceAccountName and OpenShiftConsoleServiceAccountName mapped to existing deployment name constants.
ServiceAccount Controller
pkg/console/controllers/serviceaccounts/controller.go
New ServiceAccountSyncController with Sync, SyncServiceAccount, and removeServiceAccount to apply or delete SA manifests, manage ownerRefs, and react to Console management state.
Integration & Startup
pkg/console/starter/starter.go
Wired and started two ServiceAccountSyncController instances (console and downloads) in operator startup and added them to the run list.
Minor Edits & Tests
pkg/console/controllers/downloadsdeployment/controller.go, pkg/console/subresource/deployment/deployment_test.go
Small comment change in downloads deployment controller; test updated to set serviceAccountName: "downloads" in the downloads deployment PodSpec.

Sequence Diagram(s)

sequenceDiagram
    participant Operator as Operator (Run)
    participant SAController as ServiceAccountSyncController
    participant Config as Console Config
    participant SAClient as ServiceAccount Client
    participant ResourceApply as resourceapply

    Operator->>SAController: Sync(ctx, controllerContext)
    activate SAController
    SAController->>Config: Get Console config
    Config-->>SAController: Console config
    alt managementState == Managed
        SAController->>ResourceApply: Load SA manifest from assets
        ResourceApply-->>SAController: SA resource
        SAController->>SAController: Set owner refs (Console as controller)
        SAController->>SAClient: Apply ServiceAccount
        SAClient-->>SAController: Applied
    else managementState == Removed
        SAController->>SAClient: Delete ServiceAccount
        SAClient-->>SAController: Deleted / NotFound
    else managementState == Unmanaged
        SAController-->>SAController: Skip sync
    end
    SAController-->>Operator: return (error if any)
    deactivate SAController
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title specifically references the main objective of the PR: adding a controller for downloads service account deletion logic, which is directly reflected in the substantial code changes introducing the ServiceAccountSyncController.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
📝 Coding Plan
  • Generate coding plan for human review comments

Comment @coderabbitai help to get the list of available commands and usage tips.

@ehearne-redhat
Copy link
Contributor Author

Thanks Jakub - I will re-ping once tests pass. I have made the requested changes. :)

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (5)
pkg/console/subresource/deployment/deployment_test.go (1)

1612-1612: Prefer API constant for the service account name in test fixtures.

Using the shared constant avoids drift if naming changes later.

♻️ Suggested tweak
-		ServiceAccountName: "downloads",
+		ServiceAccountName: api.OpenShiftConsoleDownloadsServiceAccountName,
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/console/subresource/deployment/deployment_test.go` at line 1612, The test
fixture uses the literal service account name "downloads"; replace this literal
with the shared API constant (e.g., api.ServiceAccountNameDownloads or the
appropriate exported constant in the api package) so the test uses the canonical
service account identifier; update the ServiceAccountName field in the
deployment test (where ServiceAccountName: "downloads") to reference that API
constant.
manifests/03-rbac-role-cluster.yaml (1)

151-174: Consider namespace-scoping ServiceAccount permissions for tighter least privilege.

The console-operator ClusterRole is bound cluster-wide via ClusterRoleBinding, which means the list/watch/get/create permissions on serviceaccounts apply across all namespaces. The update/delete operations are properly least-privilege scoped to resourceNames console and downloads. If the operator only needs to manage ServiceAccounts in specific namespaces (e.g., openshift-console), move the SA list/watch/get/create rules to namespace-scoped Roles instead of the ClusterRole.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@manifests/03-rbac-role-cluster.yaml` around lines 151 - 174, The ClusterRole
"console-operator" currently grants list/watch/get/create on ServiceAccounts
cluster-wide; remove those list/watch/get/create rules from the ClusterRole and
instead create a namespace-scoped Role (e.g., "console-operator-sa-access") in
the target namespace(s) such as "openshift-console" that grants verbs list,
watch, get, create on serviceaccounts, then add a RoleBinding that binds the
operator's ServiceAccount to that Role in each namespace; keep the existing
update/delete rules with resourceNames "console" and "downloads" in the
ClusterRole for any cluster-scoped needs and ensure your ClusterRoleBinding only
grants the remaining cluster-scoped permissions to the operator.
pkg/console/controllers/serviceaccounts/controller.go (3)

134-136: Minor: Inconsistent error message formatting.

The error message uses a capital "No" (should be lowercase for consistency with Go conventions) and has an extra space before the period.

📝 Suggested fix
-		return fmt.Errorf("No service account found for name %v .", c.serviceAccountName)
+		return fmt.Errorf("no service account found for name %s", c.serviceAccountName)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/console/controllers/serviceaccounts/controller.go` around lines 134 -
136, Update the error string returned when serviceAccount.Name == "" in
controller.go (the if block referencing serviceAccount.Name and
c.serviceAccountName) to use a lowercase "no" and remove the extra space before
the period; e.g. change the fmt.Errorf message to "no service account found for
name %v." (optionally use %q for clearer quoting of c.serviceAccountName).

40-40: Unused field: infrastructureLister is stored but never accessed.

The infrastructureLister is initialized in the constructor (line 69) but no method reads from it. If infrastructure config changes should trigger reconciliation only (via the informer on line 81), consider removing the stored lister to reduce confusion. Otherwise, if it's needed for future logic, add a TODO comment.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/console/controllers/serviceaccounts/controller.go` at line 40, The struct
field infrastructureLister is unused: either remove the infrastructureLister
field from the controller struct and delete its initialization in NewController
(and any related imports) to avoid dead state, or if you intend to use it later,
keep the field but add a clear TODO comment above the infrastructureLister
declaration and a short inline comment in NewController explaining why it's
stored; reference the field name infrastructureLister and the constructor
function NewController to locate and update the code accordingly.

86-86: Replace deprecated strings.Title with golang.org/x/text/cases.Title.

strings.Title is deprecated since Go 1.18 and can trigger linter warnings. Although it works for ASCII-only controller names, use the modern alternative from the golang.org/x/text package (already a project dependency).

Add imports:

import "golang.org/x/text/cases"
import "golang.org/x/text/language"

Then update line 86:

-ToController(fmt.Sprintf("%sServiceAccountController", strings.Title(controllerName)), ...)
+ToController(fmt.Sprintf("%sServiceAccountController", cases.Title(language.English).String(controllerName)), ...)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/console/controllers/serviceaccounts/controller.go` at line 86, Replace
deprecated strings.Title usage in the ToController call: import
"golang.org/x/text/cases" and "golang.org/x/text/language", then change the name
construction in the call that uses strings.Title(controllerName) (found in the
ToController call that builds "%sServiceAccountController" and
recorder.WithComponentSuffix) to use
cases.Title(language.Und).String(controllerName) so the controller name is
formed with the modern title-casing API.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@manifests/03-rbac-role-cluster.yaml`:
- Around line 151-174: The ClusterRole "console-operator" currently grants
list/watch/get/create on ServiceAccounts cluster-wide; remove those
list/watch/get/create rules from the ClusterRole and instead create a
namespace-scoped Role (e.g., "console-operator-sa-access") in the target
namespace(s) such as "openshift-console" that grants verbs list, watch, get,
create on serviceaccounts, then add a RoleBinding that binds the operator's
ServiceAccount to that Role in each namespace; keep the existing update/delete
rules with resourceNames "console" and "downloads" in the ClusterRole for any
cluster-scoped needs and ensure your ClusterRoleBinding only grants the
remaining cluster-scoped permissions to the operator.

In `@pkg/console/controllers/serviceaccounts/controller.go`:
- Around line 134-136: Update the error string returned when serviceAccount.Name
== "" in controller.go (the if block referencing serviceAccount.Name and
c.serviceAccountName) to use a lowercase "no" and remove the extra space before
the period; e.g. change the fmt.Errorf message to "no service account found for
name %v." (optionally use %q for clearer quoting of c.serviceAccountName).
- Line 40: The struct field infrastructureLister is unused: either remove the
infrastructureLister field from the controller struct and delete its
initialization in NewController (and any related imports) to avoid dead state,
or if you intend to use it later, keep the field but add a clear TODO comment
above the infrastructureLister declaration and a short inline comment in
NewController explaining why it's stored; reference the field name
infrastructureLister and the constructor function NewController to locate and
update the code accordingly.
- Line 86: Replace deprecated strings.Title usage in the ToController call:
import "golang.org/x/text/cases" and "golang.org/x/text/language", then change
the name construction in the call that uses strings.Title(controllerName) (found
in the ToController call that builds "%sServiceAccountController" and
recorder.WithComponentSuffix) to use
cases.Title(language.Und).String(controllerName) so the controller name is
formed with the modern title-casing API.

In `@pkg/console/subresource/deployment/deployment_test.go`:
- Line 1612: The test fixture uses the literal service account name "downloads";
replace this literal with the shared API constant (e.g.,
api.ServiceAccountNameDownloads or the appropriate exported constant in the api
package) so the test uses the canonical service account identifier; update the
ServiceAccountName field in the deployment test (where ServiceAccountName:
"downloads") to reference that API constant.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: f111efba-42f6-498f-bbf8-ca1719b5be08

📥 Commits

Reviewing files that changed from the base of the PR and between 5e38222 and d75e52d.

📒 Files selected for processing (10)
  • bindata/assets/deployments/downloads-deployment.yaml
  • bindata/assets/serviceaccounts/console-sa.yaml
  • bindata/assets/serviceaccounts/downloads-sa.yaml
  • manifests/03-rbac-role-cluster.yaml
  • manifests/06-sa.yaml
  • pkg/api/api.go
  • pkg/console/controllers/downloadsdeployment/controller.go
  • pkg/console/controllers/serviceaccounts/controller.go
  • pkg/console/starter/starter.go
  • pkg/console/subresource/deployment/deployment_test.go
💤 Files with no reviewable changes (1)
  • manifests/06-sa.yaml

@ehearne-redhat ehearne-redhat force-pushed the downloads-service-account-controller branch from d75e52d to 72b2a70 Compare March 5, 2026 09:50
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
pkg/console/controllers/serviceaccounts/controller.go (1)

129-147: ⚠️ Potential issue | 🟠 Major

Re-add the Console controller ownerRef before apply.

Line 129 passes operatorConfigCopy, but Lines 137-147 only clear existing Controller flags and never attach a controller ownerRef. This can leave the ServiceAccount ownerless from a controller perspective.

Proposed fix
 func (c *ServiceAccountSyncController) SyncServiceAccount(ctx context.Context, operatorConfigCopy *operatorv1.Console, controllerContext factory.SyncContext) error {
@@
 	ownerRefs := serviceAccount.GetOwnerReferences()
 	for oR := range ownerRefs {
 		falseBool := false
 		ownerRefs[oR].Controller = &falseBool
 	}
+	ownerRef := metav1.NewControllerRef(operatorConfigCopy, operatorv1.GroupVersion.WithKind("Console"))
+	ownerRefs = append(ownerRefs, *ownerRef)
 
 	serviceAccount.SetOwnerReferences(ownerRefs)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/console/controllers/serviceaccounts/controller.go` around lines 129 -
147, ServiceAccount ownerRefs are cleared but a Console controller ownerRef is
never re-attached; in SyncServiceAccount build and append a
metav1.OwnerReference for the Console using operatorConfigCopy (set Name from
operatorConfigCopy.GetName(), UID from operatorConfigCopy.GetUID(), Kind
"Console" and the correct APIVersion for operatorv1) with Controller=true and
BlockOwnerDeletion=true, then set that ownerRef on serviceAccount before calling
the apply logic; update the code in SyncServiceAccount (serviceAccount,
ownerRefs, operatorConfigCopy) to create and append this controller ownerRef.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In `@pkg/console/controllers/serviceaccounts/controller.go`:
- Around line 129-147: ServiceAccount ownerRefs are cleared but a Console
controller ownerRef is never re-attached; in SyncServiceAccount build and append
a metav1.OwnerReference for the Console using operatorConfigCopy (set Name from
operatorConfigCopy.GetName(), UID from operatorConfigCopy.GetUID(), Kind
"Console" and the correct APIVersion for operatorv1) with Controller=true and
BlockOwnerDeletion=true, then set that ownerRef on serviceAccount before calling
the apply logic; update the code in SyncServiceAccount (serviceAccount,
ownerRefs, operatorConfigCopy) to create and append this controller ownerRef.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: a26fd2a2-0e32-41a7-8a0f-d87ee172ab7e

📥 Commits

Reviewing files that changed from the base of the PR and between d75e52d and 72b2a70.

📒 Files selected for processing (4)
  • manifests/03-rbac-role-cluster.yaml
  • pkg/api/api.go
  • pkg/console/controllers/serviceaccounts/controller.go
  • pkg/console/starter/starter.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • manifests/03-rbac-role-cluster.yaml

@ehearne-redhat
Copy link
Contributor Author

@coderabbitai I cannot add an additional ownerref to the controller using ownerreffrom() as it gives a controller:true value which resulted in this error in e2e-azure-ovn-upgrade .

event happened 193 times, something is wrong: namespace/openshift-console-operator deployment/console-operator hmsg/7e9628ad2d - reason/ServiceAccountUpdateFailed Failed to update ServiceAccount/console -n openshift-console: ServiceAccount "console" is invalid: metadata.ownerReferences: Invalid value: [{"apiVersion":"config.openshift.io/v1","kind":"ClusterVersion","name":"version","uid":"5764b29f-7749-4012-94ce-d8c3d4e72991","controller":true},{"apiVersion":"operator.openshift.io/v1","kind":"Console","name":"cluster","uid":"44ea12d6-4e25-47cb-916f-5127bf971ea2","controller":true}]: Only one reference can have Controller set to true. Found "true" in references for ClusterVersion/version and Console/cluster (17:36:30Z) result=reject 
event happened 192 times, something is wrong: namespace/openshift-console-operator deployment/console-operator hmsg/9ba732befd - reason/ServiceAccountUpdateFailed Failed to update ServiceAccount/downloads -n openshift-console: ServiceAccount "downloads" is invalid: metadata.ownerReferences: Invalid value: [{"apiVersion":"config.openshift.io/v1","kind":"ClusterVersion","name":"version","uid":"5764b29f-7749-4012-94ce-d8c3d4e72991","controller":true},{"apiVersion":"operator.openshift.io/v1","kind":"Console","name":"cluster","uid":"44ea12d6-4e25-47cb-916f-5127bf971ea2","controller":true}]: Only one reference can have Controller set to true. Found "true" in references for ClusterVersion/version and Console/cluster (17:36:30Z) result=reject }

However, the logic in the service account controller should ensure that even if we used ownerreffrom() that there should only be one controller:true.

@coderabbitai
Copy link

coderabbitai bot commented Mar 5, 2026

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

Let me examine the service account controller code to understand the ownerReference logic.


🏁 Script executed:

#!/bin/bash
# Read the service account controller
cat pkg/console/controllers/serviceaccounts/controller.go

Length of output: 6090


Let me dig deeper into how resourceapply.ApplyServiceAccount handles owner references and trace where the Console/cluster ownerRef gets set.


🏁 Script executed:

#!/bin/bash
# Find where Console/cluster ownerRef or ownerRefFrom is used in the codebase
rg -n "ownerRef\|OwnerRef\|ownerref" --type go -C3

echo "---"

# Check how resourceapply.ApplyServiceAccount is called elsewhere for patterns
rg -n "ApplyServiceAccount" --type go -C3

echo "---"

# Check library-go's ApplyServiceAccount to see if it adds ownerRefs
find . -path ./vendor/github.com/openshift/library-go/pkg/operator/resource/resourceapply -name "*.go" | head -5
cat vendor/github.com/openshift/library-go/pkg/operator/resource/resourceapply/core.go 2>/dev/null | grep -A 40 "func ApplyServiceAccount"

Length of output: 7926


Now let me check how EnsureObjectMeta merges owner references, which will reveal the full picture of this bug.


🏁 Script executed:

#!/bin/bash
# Check EnsureObjectMeta in resourcemerge
find . -path "*/resourcemerge/*.go" | xargs grep -l "EnsureObjectMeta" 2>/dev/null

echo "---"

grep -n "OwnerRef\|ownerRef\|EnsureOwner\|OwnerReference" vendor/github.com/openshift/library-go/pkg/operator/resource/resourcemerge/core.go 2>/dev/null | head -30

echo "---"

# Look at the full EnsureObjectMeta function
grep -n -A 50 "func EnsureObjectMeta" vendor/github.com/openshift/library-go/pkg/operator/resource/resourcemerge/core.go 2>/dev/null | head -80

Length of output: 195


🏁 Script executed:

#!/bin/bash
# Check object_merger.go for EnsureObjectMeta and ownerRef handling
cat vendor/github.com/openshift/library-go/pkg/operator/resource/resourcemerge/object_merger.go

Length of output: 7485


@ehearne-redhat — I've traced through the full call chain. Here's what's happening and why the fix isn't working.


Root cause

The clearing loop in SyncServiceAccount operates on the manifest object (read from bindata YAML), which has no ownerReferences defined:

ownerRefs := serviceAccount.GetOwnerReferences()  // empty — YAML has no ownerRefs
for oR := range ownerRefs {                        // iterates over nothing
    ownerRefs[oR].Controller = &falseBool          // never executed
}

So the clearing does nothing. Meanwhile, resourceapply.ApplyServiceAccountApplyServiceAccountImproved calls resourcemerge.EnsureObjectMeta, which calls MergeOwnerRefs. Looking at MergeOwnerRefs:

// It only iterates over ownerRefs in *required* (the manifest SA — empty list)
for _, o := range required {  // required is empty, loop body never runs
    ...
}

Because required has no ownerRefs, MergeOwnerRefs does nothing and the live object's ownerRefs are preserved as-is. The ClusterVersion/version controller:true (set by CVO from the old manifests/06-sa.yaml entry) and the Console/cluster controller:true (set by a previous iteration of this code that used ownerreffrom()) both survive on the live SA. Kubernetes then rejects the update since two ownerRefs have controller:true.


Fix

You need to fetch the live SA from the cluster, clear controller:true on its ownerRefs, and then pass those cleared refs as the desired state. MergeOwnerRefs will then see matching entries in required and update them to controller:false:

func (c *ServiceAccountSyncController) SyncServiceAccount(ctx context.Context, operatorConfigCopy *operatorv1.Console, controllerContext factory.SyncContext) error {
	serviceAccount := resourceread.ReadServiceAccountV1OrDie(
		bindata.MustAsset(fmt.Sprintf("assets/serviceaccounts/%s-sa.yaml", c.serviceAccountName)),
	)
	if serviceAccount.Name == "" {
		return fmt.Errorf("No service account found for name %v .", c.serviceAccountName)
	}

	// Fetch the live SA to get its actual ownerRefs (e.g. ClusterVersion/version
	// set by CVO from the old manifests/06-sa.yaml, or any previously set
	// Console/cluster ref). We need these on the *required* object so that
	// MergeOwnerRefs will update them — it only modifies refs that appear in
	// required (matched by Name+Kind+APIVersion.Group).
	existing, err := c.serviceAccountClient.ServiceAccounts(api.OpenShiftConsoleNamespace).Get(ctx, c.serviceAccountName, metav1.GetOptions{})
	if err != nil && !apierrors.IsNotFound(err) {
		return fmt.Errorf("failed to get service account %s: %w", c.serviceAccountName, err)
	}
	if err == nil {
		// Clear controller:true from all existing ownerRefs to satisfy the
		// Kubernetes invariant that only one ownerRef may have controller:true.
		// https://kubernetes.io/docs/concepts/overview/working-with-objects/owners-dependents/
		falseBool := false
		ownerRefs := existing.DeepCopy().GetOwnerReferences()
		for i := range ownerRefs {
			ownerRefs[i].Controller = &falseBool
		}
		serviceAccount.SetOwnerReferences(ownerRefs)
	}

	_, _, err = resourceapply.ApplyServiceAccount(ctx,
		c.serviceAccountClient,
		controllerContext.Recorder(),
		serviceAccount,
	)
	if err != nil {
		return fmt.Errorf("failed to apply service account %s: %w", c.serviceAccountName, err)
	}

	return nil
}

Why this works:

  1. The live SA's ownerRefs (including ClusterVersion/version controller:true and Console/cluster controller:true) are fetched and copied.
  2. All their controller flags are set to false.
  3. These are placed on the required SA object so MergeOwnerRefs finds matching entries (by Name+Kind+Group) and updates them to controller:false.
  4. No ownerRef ends up with controller:true on the SA, resolving the Kubernetes validation error.

This also makes the ownerreffrom() approach safe again if you want to re-add it later — just ensure you clear existing controller flags from the live object first.

@ehearne-redhat ehearne-redhat force-pushed the downloads-service-account-controller branch from 72b2a70 to 245427b Compare March 5, 2026 10:19
@ehearne-redhat
Copy link
Contributor Author

@coderabbitai I made that change, but I did change one thing.

Your suggestion of api.OpenShiftConsoleNamespace implies that this controller will only be used there. Since we are going to apply the service account anyways, I opted to use serviceaccount.namespace instead.

@coderabbitai
Copy link

coderabbitai bot commented Mar 5, 2026

Tip

For best results, initiate chat on the files or code changes.

@ehearne-redhat That's a great call — using serviceAccount.Namespace (derived from the manifest YAML) is strictly better than hardcoding api.OpenShiftConsoleNamespace. It keeps the controller self-contained and namespace-agnostic, so if the controller is ever instantiated for a service account in a different namespace down the line, it will just work without any changes. Good catch!


📝 Learnings were identified and not saved (knowledge base disabled). Enable

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
pkg/console/controllers/serviceaccounts/controller.go (1)

121-123: Avoid namespace drift between apply and delete paths.

Line 122 hard-codes openshift-console for deletion, while apply/get use the namespace from the manifest object. If manifest namespace changes, removed-state cleanup can miss the ServiceAccount.

♻️ Proposed adjustment
 func (c *ServiceAccountSyncController) removeServiceAccount(ctx context.Context) error {
-	err := c.serviceAccountClient.ServiceAccounts(api.OpenShiftConsoleNamespace).Delete(ctx, c.serviceAccountName, metav1.DeleteOptions{})
+	serviceAccount := resourceread.ReadServiceAccountV1OrDie(
+		bindata.MustAsset(fmt.Sprintf("assets/serviceaccounts/%s-sa.yaml", c.serviceAccountName)),
+	)
+	if serviceAccount.Namespace == "" {
+		return fmt.Errorf("service account namespace is empty for %s", c.serviceAccountName)
+	}
+	err := c.serviceAccountClient.ServiceAccounts(serviceAccount.Namespace).Delete(ctx, c.serviceAccountName, metav1.DeleteOptions{})
 	if apierrors.IsNotFound(err) {
 		return nil
 	}
 	return err
 }

Also applies to: 129-132, 142-143

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/console/controllers/serviceaccounts/controller.go` around lines 121 -
123, The delete path in removeServiceAccount uses a hard-coded
api.OpenShiftConsoleNamespace when calling
c.serviceAccountClient.ServiceAccounts(...).Delete which can cause namespace
drift versus the apply/get paths that use the manifest's namespace; change the
Delete calls (and the similar calls around lines referenced) to use the
ServiceAccount namespace derived from the manifest or controller state (e.g.,
the manifest's namespace via c.manifest.GetNamespace() or the controller's
namespace field) instead of api.OpenShiftConsoleNamespace, keeping
c.serviceAccountName as-is so deletion targets the same resource created by
apply.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@pkg/console/controllers/serviceaccounts/controller.go`:
- Around line 121-123: The delete path in removeServiceAccount uses a hard-coded
api.OpenShiftConsoleNamespace when calling
c.serviceAccountClient.ServiceAccounts(...).Delete which can cause namespace
drift versus the apply/get paths that use the manifest's namespace; change the
Delete calls (and the similar calls around lines referenced) to use the
ServiceAccount namespace derived from the manifest or controller state (e.g.,
the manifest's namespace via c.manifest.GetNamespace() or the controller's
namespace field) instead of api.OpenShiftConsoleNamespace, keeping
c.serviceAccountName as-is so deletion targets the same resource created by
apply.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 8bb2957d-ec8c-4144-a595-ec3870a9c804

📥 Commits

Reviewing files that changed from the base of the PR and between 72b2a70 and 245427b.

📒 Files selected for processing (4)
  • manifests/03-rbac-role-cluster.yaml
  • pkg/api/api.go
  • pkg/console/controllers/serviceaccounts/controller.go
  • pkg/console/starter/starter.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • manifests/03-rbac-role-cluster.yaml

@ehearne-redhat ehearne-redhat force-pushed the downloads-service-account-controller branch from 245427b to aa70bf3 Compare March 5, 2026 10:31
@ehearne-redhat
Copy link
Contributor Author

@coderabbitai I decided to keep it consistent instead.

@coderabbitai
Copy link

coderabbitai bot commented Mar 5, 2026

Tip

For best results, initiate chat on the files or code changes.

@ehearne-redhat Understood — using api.OpenShiftConsoleNamespace keeps it consistent with how the rest of the codebase references that namespace, which is perfectly reasonable. Thanks for the heads-up!

@ehearne-redhat ehearne-redhat requested a review from jhadvig March 5, 2026 14:44
@ehearne-redhat
Copy link
Contributor Author

@jhadvig - tests are now passing. Feel free to review the requested changes. :)

- list
- watch
- delete
- apiGroups:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We just need one

  - apiGroups:
      - ""
    resources:
      - serviceaccounts
    verbs:
      - get
      - list
      - watch
      - create
      - update
      - delete
    resourceNames:
      - console
      - downloads

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking here , if we did squash it all together then we may have issues with looking for service accounts. I kept it separate so we can list, watch and get any service accounts. However, there is no reason why we cannot merge the first two rules.

From what I understand, the minimum number of rules we can have is two. We can move list, watch and get to the third rule. However, we cannot restrict create .

You cannot restrict deletecollection or top-level create requests by resource name. For create, this limitation is because the name of the new object may not be known at authorization time.

As for the third rule, if we wanted to make modifications to the service account, we wouldn't want for the controller to be able to delete just any service account, so having resourceNames specified here allows us to restrict where possible.

So, the compromise we can do is keep create without restriction, and we can restrict everything else. I'll go ahead and update to reflect this, and feel free to correct me if I misunderstood anything.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also need to double check if the get used in the controller looks for these service accounts specifically or not and make adjustments.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @jhadvig - sadly we cannot use it like this. Found this in console-operator logs

026-03-11T13:25:17.260764907Z E0311 13:25:17.260731       1 reflector.go:205] "Failed to watch" err="failed to list *v1.ServiceAccount: serviceaccounts is forbidden: User \"system:serviceaccount:openshift-console-operator:console-operator\" cannot list resource \"serviceaccounts\" in API group \"\" in the namespace \"openshift-console\"" 

This is the current rule formation:

- apiGroups:
      - ""
    resources:
      - serviceaccounts
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - serviceaccounts
    verbs:
      - list
      - watch
      - get
      - update
      - delete
    resourceNames:
      - console
      - downloads

I will probably need to move it to this

- apiGroups:
      - ""
    resources:
      - serviceaccounts
    verbs:
      - create
      - list
      - watch
      - get
  - apiGroups:
      - ""
    resources:
      - serviceaccounts
    verbs:
      - update
      - delete
    resourceNames:
      - console
      - downloads

Let me know if this suits. :)

Copy link
Member

@jhadvig jhadvig Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:
its a bit weird that we have the create action but not delete.
if anything I would add the create to the second list, sicne we are only creating these two SAs

return nil
case operatorv1.Removed:
klog.V(4).Infoln("console is in a removed state: removing synced serviceaccount")
statusHandler := status.NewStatusHandler(c.operatorClient)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets not initialize the statushandler in this case

		return c.removeServiceAccount(ctx)

}

func (c *ServiceAccountSyncController) SyncServiceAccount(ctx context.Context, operatorConfigCopy *operatorv1.Console, controllerContext factory.SyncContext) error {
serviceAccount := resourceread.ReadServiceAccountV1OrDie(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let pull this block into a helper

func DefaultServiceAccount(cr *operatorv1.Console) (*corev1.ServiceAccount, error) {
	serviceAccount := resourceread.ReadServiceAccountV1OrDie(
		bindata.MustAsset(fmt.Sprintf("assets/serviceaccounts/%s-sa.yaml", c.serviceAccountName)),
	)
	if serviceAccount.Name == "" {
		return nil, fmt.Errorf("No service account found for name %v .", c.serviceAccountName)
	}
    util.AddOwnerRef(serviceAccount, util.OwnerRefFrom(cr))
    return serviceAccount, nil
}

return fmt.Errorf("No service account found for name %v .", c.serviceAccountName)
}

// Fetch the live SA to get its actual ownerRefs (e.g. ClusterVersion/version
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dont think this is quite right, eg is the SA will be missing this block will get skipped.

instead:

	required, err := DefaultServiceAccount() // declared above
    if err != nil {
        return err
    }
    _, _, err = resourceapply.ApplyServiceAccount(ctx,
		c.serviceAccountClient,
		controllerContext.Recorder(),
		serviceAccount,
	)
	if err != nil {
		return fmt.Errorf("failed to apply service account %s: %w", c.serviceAccountName, err)
	}

@ehearne-redhat ehearne-redhat force-pushed the downloads-service-account-controller branch from aa70bf3 to 9cd12ec Compare March 10, 2026 14:36
@ehearne-redhat
Copy link
Contributor Author

/retest

@ehearne-redhat ehearne-redhat force-pushed the downloads-service-account-controller branch 2 times, most recently from 3f7011f to ac6818f Compare March 12, 2026 09:03
@ehearne-redhat ehearne-redhat force-pushed the downloads-service-account-controller branch from ac6818f to 1ee9a0a Compare March 20, 2026 12:04
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Mar 20, 2026

@ehearne-redhat: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@ehearne-redhat ehearne-redhat requested a review from jhadvig March 23, 2026 15:26
Copy link
Member

@jhadvig jhadvig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

- list
- watch
- delete
- apiGroups:
Copy link
Member

@jhadvig jhadvig Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:
its a bit weird that we have the create action but not delete.
if anything I would add the create to the second list, sicne we are only creating these two SAs

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Mar 24, 2026
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Mar 24, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ehearne-redhat, jhadvig

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 24, 2026
@jhadvig
Copy link
Member

jhadvig commented Mar 24, 2026

/approved

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/severity-critical Referenced Jira bug's severity is critical for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants