Prevent spaming DEBUG messages with 403s because we try to delete orphaned resources that we can't list

[sbernauer]

Description

Reported in https://stackable-workspace.slack.com/archives/C02FZ581UCD/p1773388676214269

In

operator-rs/crates/stackable-operator/src/cluster_resources.rs

Line 672 in 992b3b2

pub async fn delete_orphaned_resources(self, client: &Client) -> Result<()> {

we try to delete all kind of resources.

It is totally expected that some operators miss RBAC permissions on certain resources (e.g. the opa-operator doesn't care about StatefulSets).
In this case the underlying kube emits DEBUGs that it received a 403 from Kubernetes (see below).

This issue is a stylistic task to prevent this 403 DEBUGs, as they could potentially confuse users.
We can achieve this by asking Kubernetes first "are we allowed to patch this resource" via a SelfSubjectAccessReview - which indicates that this operator cares about creating such resource and should there garbage-collect them.
Only in case we can patch the resource we run the GC for it.
We cache this "can I patch this resource" result with a TTL of e.g. 5 minutes to reduce k8s API calls.

A sample log looks like

opa-operator-deployment-554d4dcc7-6x89m 2026-03-13T07:52:38.840175Z DEBUG reconciling object{object.ref=OpaCluster.v1alpha2.opa.stackable.tech/opa.default object.reason=object updated}: kube_client::client: Unsuccessful: Status { status: Some(Failure), code: 403, message: "statefulsets.apps is forbidden: User \"system:serviceaccount:stackable-operators:opa-operator-serviceaccount\" cannot list resource \"statefulsets\" in API group \"apps\" in the namespace \"default\"", metadata: Some(ListMeta { continue_: None, remaining_item_count: None, resource_version: None, self_link: None }), reason: "Forbidden", details: Some(StatusDetails { name: "", group: "apps", kind: "statefulsets", uid: "", causes: [], retry_after_seconds: 0 }) }

followed by the line

opa-operator-deployment-554d4dcc7-6x89m 2026-03-13T07:52:38.840206Z DEBUG reconciling object{object.ref=OpaCluster.v1alpha2.opa.stackable.tech/opa.default object.reason=object updated}: stackable_operator::cluster_resources: Skipping deletion of orphaned statefulsets because the operator is not allowed to list them and is therefore probably not in charge of them.

[razvan]

Update 2: nvm i'll try this.

Update: I now understand. This tries to brute force delete stuff regardless of operator ownership.
These are lots of requests on the api server. I think the proper solution is to keep a list of objects created by the op and only list what is necessary.

If I understand correctly, the proposed solution just skips the deletion of orphaned resources if listing is not allowed.
This removes the error but doesn't solve the original problem: orphaned resources are not garbage collected.
The proper solution is to allow the operators to list/delete all resources they create.

[sbernauer]

I think the proper solution is to keep a list of objects created by the op and only list what is necessary.

That would be the naive brute-force solution, but I figured we can do it more clever and less error-prone: We already have a list of resources we are allowed to delete: It's in the RBAC :)

These are lots of requests on the api server

Agreed, that's why I asked Claude to please include a cache