-
Notifications
You must be signed in to change notification settings - Fork 1
Add GH Action workflow to update downstream repos #73
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,126 @@ | ||||||
| name: Update downstream repos | ||||||
|
|
||||||
| on: | ||||||
| push: | ||||||
| branches: | ||||||
| - main | ||||||
| workflow_dispatch: | ||||||
|
|
||||||
| concurrency: | ||||||
| group: ${{ github.workflow }} | ||||||
| cancel-in-progress: true | ||||||
|
|
||||||
| jobs: | ||||||
| build-downstream-matrix: | ||||||
| runs-on: ubuntu-latest | ||||||
| outputs: | ||||||
| downstream-matrix: ${{ steps.downstream-matrix.outputs.downstream-matrix }} | ||||||
| steps: | ||||||
| - id: downstream-matrix | ||||||
| env: | ||||||
| GH_TOKEN: ${{ secrets.GH_TOKEN_NEXTSTRAIN_BOT_REPO }} | ||||||
| # Create an array of potential repos to update that will be used as the | ||||||
| # matrix to the next job. | ||||||
| # [ | ||||||
| # { "repo": "nextstrain/zika", "path": "shared/vendored"}, | ||||||
| # { "repo": "nextstrain/mpox", "path": "shared/vendored"}, | ||||||
| # ... | ||||||
| # ] | ||||||
| run: | | ||||||
| shared_matrix=$(gh api -X GET search/code \ | ||||||
| -f q='org:nextstrain filename:.gitrepo "remote = https://github.com/nextstrain/shared"' \ | ||||||
| | jq -c ' | ||||||
| .items | ||||||
| | map({ | ||||||
| "repo": "\(.repository.full_name)", | ||||||
| "path": "\(.path | split("/")[0:-1] | join("/"))" | ||||||
| }) | ||||||
| ') | ||||||
|
|
||||||
| # I was unable to get the 'OR' syntax to work with the search/code API, | ||||||
| # so making a separate query for the old nextstrain/ingest repo name. | ||||||
| # -Jover, 14 Apr 2026. | ||||||
| ingest_matrix=$(gh api -X GET search/code \ | ||||||
| -f q='org:nextstrain filename:.gitrepo "remote = https://github.com/nextstrain/ingest"' \ | ||||||
| | jq -c ' | ||||||
| .items | ||||||
| | map({ | ||||||
| "repo": "\(.repository.full_name)", | ||||||
| "path": "\(.path | split("/")[0:-1] | join("/"))" | ||||||
| }) | ||||||
| ') | ||||||
|
|
||||||
| # Deduplicate by repo since each repo should only have a single copy | ||||||
| # of the vendored repo. In cases where a repo has both, | ||||||
| # we are prioritizing the nextstrain/shared remote since that is | ||||||
| # the newer repo. | ||||||
| # -Jover, 15 Apr 2026. | ||||||
| matrix=$(jq -n \ | ||||||
| --argjson matrix1 "$shared_matrix" \ | ||||||
| --argjson matrix2 "$ingest_matrix" \ | ||||||
| -c '$matrix1 + $matrix2 | unique_by(.repo)') | ||||||
|
|
||||||
| echo "downstream-matrix=$matrix" | tee -a "$GITHUB_OUTPUT" | ||||||
| update-downstream: | ||||||
| name: update-downstream (${{ matrix.repo }}, ${{ matrix.path }}) | ||||||
| needs: [build-downstream-matrix] | ||||||
| strategy: | ||||||
| fail-fast: false | ||||||
| matrix: | ||||||
| include: ${{ fromJson(needs.build-downstream-matrix.outputs.downstream-matrix) }} | ||||||
| env: | ||||||
| GIT_SUBREPO_DIR: .git/git-subrepo | ||||||
| VENDORED_PATH: ${{ matrix.path }} | ||||||
| branch: nextstrain-bot/update-vendored | ||||||
| runs-on: ubuntu-latest | ||||||
| steps: | ||||||
| - name: Checkout ${{ matrix.repo }} | ||||||
| uses: actions/checkout@v6 | ||||||
| with: | ||||||
| repository: ${{ matrix.repo }} | ||||||
| token: ${{ secrets.GH_TOKEN_NEXTSTRAIN_BOT_REPO }} | ||||||
| # Checkout git-subrepo _after_ the downstream repo to ensure that we | ||||||
| # keep it in a path within the downstream repo that does not interefere | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| # with the subrepo changes | ||||||
| - name: Checkout git-subrepo | ||||||
| uses: actions/checkout@v6 | ||||||
| with: | ||||||
| repository: "ingydotnet/git-subrepo" | ||||||
| path: ${{ env.GIT_SUBREPO_DIR }} | ||||||
| - name: Add git-subrepo to PATH | ||||||
| run: echo "$GIT_SUBREPO_DIR/lib" >> "$GITHUB_PATH" | ||||||
| - name: Update vendored path | ||||||
| run: | | ||||||
| git config user.name "${{ vars.GIT_USER_NAME_NEXTSTRAIN_BOT }}" | ||||||
| git config user.email "${{ vars.GIT_USER_EMAIL_NEXTSTRAIN_BOT }}" | ||||||
|
|
||||||
| git switch -c "$branch" | ||||||
| git subrepo pull "$VENDORED_PATH" --force | ||||||
| - name: Create pull request | ||||||
| env: | ||||||
| GH_TOKEN: ${{ secrets.GH_TOKEN_NEXTSTRAIN_BOT_REPO }} | ||||||
| title: '[bot] Update ${{ env.VENDORED_PATH }}' | ||||||
| body: | | ||||||
| This PR was automaticaly created by http://github.com/nextstrain/shared/actions/runs/${{ github.run_id }} | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| to update the vendored subrepo in ${{ env.VENDORED_PATH }}. | ||||||
|
|
||||||
| Subrepo updates were made with the `--force` flag so it overwrites any local changes in the subrepo. | ||||||
| run: | | ||||||
| default_branch=$(git remote show origin | sed -n '/HEAD branch/s/.*: //p') | ||||||
| changes=$(git rev-list --count "$default_branch".."$branch") | ||||||
| if [[ "$changes" == "1" ]]; then | ||||||
| git push --force origin HEAD | ||||||
|
Comment on lines
+111
to
+112
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How about
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 👍 I exclusively use |
||||||
| pr_url=$(gh pr list --head "$branch" --json url | jq -r '.[0].url') | ||||||
|
|
||||||
| if [[ "$pr_url" == "null" ]]; then | ||||||
| pr_url="$(gh pr create --head "$branch" --title "$title" --body "$body")" | ||||||
| echo "Pull request created: $pr_url" >> "$GITHUB_STEP_SUMMARY" | ||||||
| else | ||||||
| echo "Pull request updated: $pr_url" >> "$GITHUB_STEP_SUMMARY" | ||||||
| fi | ||||||
| elif [[ "$changes" == "0" ]]; then | ||||||
| echo "No pull request created or updated because no changes were made" >> "$GITHUB_STEP_SUMMARY" | ||||||
| else | ||||||
| echo "ERROR: Encountered an unexpected number of changes: $changes" | ||||||
| exit 1 | ||||||
| fi | ||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
GitHub API calls are limited to 30 results by default.
--paginate+--slurpshould work well to get everything. Something like this (copied from LLM output, untested):Same goes for the other API call.