Skip to content

ci: add ConfigMap-based coverage tracking#8084

Closed
bryan-cox wants to merge 2 commits intoopenshift:mainfrom
bryan-cox:add-coverage-tracking
Closed

ci: add ConfigMap-based coverage tracking#8084
bryan-cox wants to merge 2 commits intoopenshift:mainfrom
bryan-cox:add-coverage-tracking

Conversation

@bryan-cox
Copy link
Copy Markdown
Member

Summary

Add coverage comparison using a ConfigMap on the arc-runner-set cluster
to track baseline coverage and report diffs on PRs.

  • Extract total and per-directory coverage from cover.out using go tool cover
  • On push to main: update a coverage-baseline ConfigMap in arc-runners namespace
  • On PRs: compare against baseline at both levels and fail if total coverage drops by more than 1%
  • Job summary shows overall trend and a per-directory breakdown with diffs

Depends on: #8060 (Codecov integration)

Setup on runner cluster

The following was applied to the arc-runner-set cluster:

```bash
export KUBECONFIG=/path/to/github-actions-kubeconfig

Create the ConfigMap

kubectl create configmap coverage-baseline -n arc-runners --from-literal=total=0

Grant the runner ServiceAccount access to the ConfigMap

kubectl create role coverage-baseline-access -n arc-runners
--verb=get,patch
--resource=configmaps
--resource-name=coverage-baseline

kubectl create rolebinding coverage-baseline-access -n arc-runners
--role=coverage-baseline-access
--serviceaccount=arc-runners:arc-runner-set-gha-rs-no-permission

Seed with actual baseline (run tests locally and extract coverage):

go test -race -count=1 -timeout=30m ./... -coverprofile cover.out

Build patch with total and per-directory coverage

TOTAL=$(go tool cover -func=cover.out | grep '^total:' | awk '{print $NF}' | tr -d '%')
PATCH="{"data":{"total":"${TOTAL}""
while IFS='=' read -r dir cov; do
PATCH="${PATCH},"dir.${dir}":"${cov}""
done < <(awk '
NR > 1 {
split($1, a, ":"); file = a[1]
sub("github.com/openshift/hypershift/", "", file)
split(file, parts, "/"); dir = parts[1]
stmts = $2; count = $3
total_stmt[dir] += stmts
if (count > 0) covered_stmt[dir] += stmts
}
END {
for (dir in total_stmt)
printf "%s=%.1f\n", dir, (total_stmt[dir] > 0 ? (covered_stmt[dir]/total_stmt[dir])*100 : 0)
}
' cover.out | sort)
PATCH="${PATCH}}}"
kubectl patch configmap coverage-baseline -n arc-runners -p "$PATCH"
```

The workflow updates this ConfigMap automatically on each push to main.

Test plan

  • Verify coverage extraction step succeeds on arc-runner-set
  • Verify RBAC allows runner SA to access ConfigMap
  • Verify baseline ConfigMap seeded with total and per-directory data
  • Verify ConfigMap gets updated on push to main
  • Verify PR comparison reports diff in job summary with per-directory breakdown

🤖 Generated with Claude Code

bryan-cox and others added 2 commits March 26, 2026 12:17
Upload coverage data to Codecov after each unit test run and add
codecov.yml to set the default branch to main.

- Uses codecov/codecov-action@v5 to upload cover.out
- Token passed via env so fork PRs fall back to tokenless upload
- codecov.yml sets default branch to main (Codecov defaults to master)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add coverage comparison using a ConfigMap on the arc-runner-set cluster
to track baseline coverage and report diffs on PRs.

- Extract total and per-directory coverage from cover.out
- On push to main: update coverage-baseline ConfigMap with current data
- On PRs: compare against baseline, fail if coverage drops >1%
- Job summary shows trend indicators and per-directory breakdown

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@openshift-ci-robot
Copy link
Copy Markdown

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Mar 26, 2026

Important

Review skipped

Auto reviews are limited based on label configuration.

🚫 Review skipped — only excluded labels are configured. (1)
  • do-not-merge/work-in-progress

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 68386a3d-674d-4979-8e4f-82095cdc96a4

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 26, 2026
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Mar 26, 2026

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Mar 26, 2026

Please specify an area label

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Mar 26, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bryan-cox

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added approved Indicates a PR has been approved by an approver from all required OWNERS files. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Mar 26, 2026
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Mar 26, 2026

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@bryan-cox
Copy link
Copy Markdown
Member Author

/close

@openshift-ci openshift-ci Bot closed this Mar 30, 2026
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Mar 30, 2026

@bryan-cox: Closed this PR.

Details

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/needs-area do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants