Is the Repo per app scenario possible with gitops-connector?

[cyberjpb1]

Is the Repo per app scenario possible with gitops-connector?
Currently I have a repo for each application which also contains the manifests.

If the answer to my question is yes, how to configure it?
gitRepositoryType: AZDO
ciCdOrchestratorType: AZDO
gitOpsOperatorType: FLUX
azdoGitOpsRepoName:
azdoOrgUrl:
azdoPrRepoName:
gitOpsAppURL:
orchestratorPAT:

[eedorenko]

You need to have multiple instances of gitops-connector on your clusters. One instance per application/repo

[markphillips100]

@eedorenko would a k8 operator approach be feasible here? To clarify, I mean one installation of gitops-connector (operator + custom Connection CRD) which then handles multiple subscribe-to-notification/publish-to-specific-repo Connection resources.

[markphillips100]

@eedorenko I've added support for multiple configs in the one instance via a CRD if you are interested? It still supports the original env config via helm values type of setup, albeit it is a breaking change due to the values restructuring. The switch between modes of operation are determined by setting singleInstance: null or supplying values. All is explained in the helm chart readme.

Whilst it works for ArgoCD notifications as-is, I haven't tested with flux as my environment isn't setup to deal with it. It's not a great deal of work and can explain if this goes further.

My fork is here, and let me know if you want a PR opened.

[cyberjpb1]

@markphillips100 Very interesting, I will try your approach by doing a test with Flux. I will get back to you as soon as possible.

[markphillips100]

@cyberjpb1 For the flux_gitops_operator to filter supported messages and indicate its support for the required config name in phase_data, the is_supported_message function needs fleshing out here. See the argo_gitops_operator change for how I implemented it in that use case.

I imagine in FluxV2 use case we would need to make use of the Alert's eventMetadata to convey the required config name in the phase_data, although being unfamiliar with Flux I do not know how this results in the phase_data structurally to know what to look for.

[markphillips100]

@cyberjpb1 I checked a previous PR you opened for insight into the eventMetadata and that gave me enough info to create the is_supported_operator. I've created a new flux-multi-config-support branch with this change.

So in theory, the following should suffice:

1. add a gitops_connector_config_name: "<name of config>" to the Alert's eventMetadata,
2. set singleInstance: null in values.yaml,
3. apply a gitopsconfig manifest to the cluster where gitops-connector is running - ensure the name is the same name used in step 1.

NOTE: The helm chart creates a service account, role and role binding to support the connector watching and updating the gitopsconfig resource. The operator also automatically patches (hence the updating) a finalizer into the resource to ensure when it is deleted that a proper cleanup occurs before the manifest is removed from the cluster.

[cyberjpb1]

@markphillips100 Hello, sorry for the delay of my response but my schedule did not allow me to do a test before today.
I tried to follow your installation procedure but it does not work, I must be missing something in the configuration.

[markphillips100]

The last warning in your logs would point at some role/role-binding issue for the service account the operator is configured to run with. Maybe take a look there to ensure the watch permission is available on gitopsconfigs crd.

[markphillips100]

The helm chart should be setting that up for you here

[cyberjpb1]

Yes, I use your Helm Chart, the only thing I changed is replacing "example.com" by "apps-crc.testing".

[markphillips100]

Yeah was noticing that but wouldn't think it would be a problem....prior to the edit :-)

[markphillips100]

Did you by any chance install the helm chart prior to making the api group changes? Just wondering if there are now 2 CRDs for the same Kind.

The 3 event handlers here aren't currently checking for apiVersion so would be just chance which one is used if more than one CRD group exists. These handlers would need to change (and probably should anyway) to include the group name.

[cyberjpb1]

It's as if the callback was never sent.
The task in the pipeline wait indefinitely.

[markphillips100]

DEBUG:kopf._cogs.clients.watching:Stopping the watch-stream for customresourcedefinitions.v1.apiextensions.k8s.io cluster-wide.
WARNING:kopf._core.reactor.observation:Not enough permissions to watch for resources: changes (creation/deletion/updates) will not be noticed; the resources are only refreshed on operator restarts.

Can you delete the operator pod (without uninstalling/reinstalling helm chart) and confirm if logs still show this warning please? It's possibly a race condition re permissions not set by time pod comes up so a restart of pod should negate that.

[cyberjpb1]

OK, I deleted the pod and here is the log of the new pod:

[2024-11-24 02:36:45 +0000] [7] [INFO] Starting gunicorn 20.0.4
[2024-11-24 02:36:45 +0000] [7] [INFO] Listening at: http://0.0.0.0:8080 (7)
[2024-11-24 02:36:45 +0000] [7] [INFO] Using worker: sync
[2024-11-24 02:36:45 +0000] [8] [INFO] Booting worker with pid: 8
DEBUG:root:Detected no ENV configuration data. Running in multiple instance configuration mode via gitopsconfig resources.
INFO:root:Starting Kopf operator thread
INFO:root:Starting GitOps Operator
DEBUG:asyncio:Using selector: EpollSelector
/usr/local/lib/python3.9/site-packages/kopf/_core/reactor/running.py:179: FutureWarning: Absence of either namespaces or cluster-wide flag will become an error soon. For now, switching to the cluster-wide mode for backward compatibility.
warnings.warn("Absence of either namespaces or cluster-wide flag will become an error soon."
DEBUG:kopf._core.reactor.running:Starting Kopf 1.37.3.
INFO:kopf._core.engines.activities:Initial authentication has been initiated.
DEBUG:kopf.activities.authentication:Activity 'login_via_client' is invoked.
WARNING:kopf._core.reactor.running:OS signals are ignored: running not in the main thread.
DEBUG:kopf.activities.authentication:Client is configured in cluster with service account.
INFO:kopf.activities.authentication:Activity 'login_via_client' succeeded.
INFO:kopf._core.engines.activities:Initial authentication has finished.
WARNING:kopf._core.reactor.observation:Non-patchable resources will not be served: {gitopsconfigs.v1.apps-crc.testing}
DEBUG:kopf._cogs.clients.watching:Starting the watch-stream for customresourcedefinitions.v1.apiextensions.k8s.io cluster-wide.
DEBUG:kopf._cogs.clients.watching:Stopping the watch-stream for customresourcedefinitions.v1.apiextensions.k8s.io cluster-wide.
WARNING:kopf._core.reactor.observation:Not enough permissions to watch for resources: changes (creation/deletion/updates) will not be noticed; the resources are only refreshed on operator restarts.