Add create-check-run safe output type for multi-agent PR analysis#33852
Add create-check-run safe output type for multi-agent PR analysis#33852Copilot wants to merge 16 commits into
create-check-run safe output type for multi-agent PR analysis#33852Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
create-check-run safe output type for multi-agent PR analysis
|
@copilot merge main and recompile add support for github-token and github-app custom configuration fields, similar to other safe-outputs. |
…eck-run-safe-output # Conflicts: # .github/workflows/ai-moderator.lock.yml # .github/workflows/changeset.lock.yml # .github/workflows/codex-github-remote-mcp-test.lock.yml # .github/workflows/daily-cache-strategy-analyzer.lock.yml # .github/workflows/daily-fact.lock.yml # .github/workflows/daily-observability-report.lock.yml # .github/workflows/duplicate-code-detector.lock.yml # .github/workflows/grumpy-reviewer.lock.yml # .github/workflows/issue-arborist.lock.yml # .github/workflows/necromancer.lock.yml # .github/workflows/schema-feature-coverage.lock.yml # .github/workflows/smoke-call-workflow.lock.yml # .github/workflows/smoke-codex.lock.yml Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
- Add GitHubApp field to BaseSafeOutputConfig (all safe-output types inherit it) - Parse github-app in parseBaseSafeOutputConfig for manual-parse handlers - Wire create-check-run handler to use minted token expression when github-app is set - Add per-handler token minting/invalidation steps in buildHandlerManagerStep - Update JSON schema to allow github-app in create-check-run config block Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Done in 77543ad:
|
|
@copilot update the smoke-copilot, and smoke-claude prompt and frontmatter to include a check run status safe-outputs. |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Done in the latest commit. Both
Lock files for both workflows were recompiled (233 total). |
|
@copilot merge main and recompile |
…eck-run-safe-output Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Applied in the latest commit. |
|
📰 BREAKING: Smoke Copilot is now investigating this pull request. Sources say the story is developing... |
|
|
|
🚀 Smoke Gemini MISSION COMPLETE! Gemini has spoken. ✨ Caution agentic threat detected Verifying connectivity to safeoutputs |
|
🎬 THE END — Smoke Claude MISSION: ACCOMPLISHED! The hero saves the day! ✨ |
|
🌑 The shadows whisper... Smoke Codex failed. The oracle requires further meditation... |
|
Caution agentic threat detected Smoke Test: Gemini - 26269860165
Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "localhost"See Network Configuration for more information.
|
Agent Container Tool Check ✅All required development tools are available:
Result: 12/12 tools available ✅ PASS
|
Smoke Claude — Run §26269860246Core #1–#13: ✅ all pass · PR-review #14–#19: ✅ all pass · #20 Overall: ✅ PASS Warning Firewall blocked 6 domainsThe following domains were blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "accounts.google.com"
- "android.clients.google.com"
- "clients2.google.com"
- "contentautofill.googleapis.com"
- "safebrowsingohttpgateway.googleapis.com"
- "www.google.com"See Network Configuration for more information.
|
There was a problem hiding this comment.
💥 Automated smoke test review - all systems nominal!
Warning
Firewall blocked 6 domains
The following domains were blocked by the firewall during workflow execution:
accounts.google.comandroid.clients.google.comclients2.google.comcontentautofill.googleapis.comsafebrowsingohttpgateway.googleapis.comwww.google.com
To allow these domains, add them to the
network.allowedlist in your workflow frontmatter:
network:
allowed:
- defaults
- "accounts.google.com"
- "android.clients.google.com"
- "clients2.google.com"
- "contentautofill.googleapis.com"
- "safebrowsingohttpgateway.googleapis.com"
- "www.google.com"See Network Configuration for more information.
💥 [THE END] — Illustrated by Smoke Claude · ● 5.8M
| // CreateCheckRunConfig holds configuration for creating GitHub Check Runs from agent output | ||
| type CreateCheckRunConfig struct { | ||
| BaseSafeOutputConfig `yaml:",inline"` | ||
| Name string `yaml:"name,omitempty"` // Check run name shown in the GitHub Checks UI |
There was a problem hiding this comment.
Nit: consider adding a brief godoc comment on the Name field clarifying that it appears in the GitHub Checks UI (you already mention it inline, but a // Name is ... style doc would also surface via go doc).
| } else { | ||
| // If configData is nil or not a map (e.g., "create-check-run:" with no value), | ||
| // still set the default max of 1 | ||
| createCheckRunLog.Print("No config map provided, using defaults (max=1)") |
There was a problem hiding this comment.
Consider unifying the two branches: even when configData is a map, the explicit Max = defaultIntStr(1) fallback could be centralized in parseBaseSafeOutputConfig so the nil/non-map path doesn't need to duplicate that logic.
✅ Smoke Test PASSEDRun: §26269860137 Results: 16/16 tests ✅
Author: All Copilot engine features validated successfully! 🎉 Warning Firewall blocked 6 domainsThe following domains were blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "accounts.google.com"
- "android.clients.google.com"
- "clients2.google.com"
- "contentautofill.googleapis.com"
- "safebrowsingohttpgateway.googleapis.com"
- "www.google.com"See Network Configuration for more information.
|
|
🎭 The smoke test agent has arrived! 🎭 Just passing through on my automated rounds, validating all the wonderful MCP servers and safe-output tools! Everything's working beautifully! Here's a little something I created while testing: Smoke test was here ✨ Keep building amazing things! ✨ Warning Firewall blocked 6 domainsThe following domains were blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "accounts.google.com"
- "android.clients.google.com"
- "clients2.google.com"
- "contentautofill.googleapis.com"
- "safebrowsingohttpgateway.googleapis.com"
- "www.google.com"See Network Configuration for more information.
|
Comment MemoryNote This comment is managed by comment memory.It stores persistent context for this thread in the code block at the top of this comment. Warning Firewall blocked 6 domainsThe following domains were blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "accounts.google.com"
- "android.clients.google.com"
- "clients2.google.com"
- "contentautofill.googleapis.com"
- "safebrowsingohttpgateway.googleapis.com"
- "www.google.com"See Network Configuration for more information.
|
There was a problem hiding this comment.
Smoke tests look comprehensive! Tests cover MCP servers, file ops, GitHub integration, and safe outputs. All required functionality appears well-tested.
Warning
Firewall blocked 6 domains
The following domains were blocked by the firewall during workflow execution:
accounts.google.comandroid.clients.google.comclients2.google.comcontentautofill.googleapis.comsafebrowsingohttpgateway.googleapis.comwww.google.com
To allow these domains, add them to the
network.allowedlist in your workflow frontmatter:
network:
allowed:
- defaults
- "accounts.google.com"
- "android.clients.google.com"
- "clients2.google.com"
- "contentautofill.googleapis.com"
- "safebrowsingohttpgateway.googleapis.com"
- "www.google.com"See Network Configuration for more information.
📰 BREAKING: Report filed by Smoke Copilot · ● 1.2M
Comments that could not be inline-anchored
.github/workflows/smoke-copilot.md:10
Nice smoke test setup! The workflow structure looks solid.
actions/setup/js/create_check_run.cjs:5
Good addition of upload_artifact to the test suite!
|
📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤 |
|
@copilot add support in safe_output to specify output.title/output.summary. Setup max sizes, sanitize markdown as with other safe outputs. Optional. |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
…k-run Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Done. The safe-outputs:
create-check-run:
name: "My Agent"
output:
title: "Static fallback title" # max 256 chars
summary: "Static fallback summary" # max 65535 charsBehaviour:
Changes: |
|
`` `` Merge state is UNSTABLE — please update/rebase the branch to re-run checks and request reviewer attention. If CI still fails, run tests locally and report results.
|
Workflows can now surface agent results as GitHub Check Runs on the triggering commit/PR — a first-class signal in the PR checks UI, distinct from comments, that supports pass/fail state with title/summary/details.
Config
Requires
checks: write+contents: readpermissions (for SHA resolution).Agent output
{"type": "create_check_run", "conclusion": "failure", "title": "3 issues found", "summary": "...", "text": "..."}Valid conclusions:
success,failure,neutral,cancelled,timed_out,action_required,skipped.Changes
create_check_run.go,compiler_types.go,safe_outputs_config.go) —CreateCheckRunConfigstruct withName+BaseSafeOutputConfig; wired intoSafeOutputsConfigparsersafe_output_handlers.go,compiler_safe_outputs_handlers.go,safe_outputs_state.go,safe_outputs_tools_computation.go,safe_outputs_max_validation.go,compiler_safe_outputs_job.go) — follows the same registration pattern ascreate_issue/create_code_scanning_alertpermissions_factory.go) — newNewPermissionsContentsReadChecksWrite()factorymain_workflow_schema.json) —create-check-runblock withname,max,github-token,stagedsafe_outputs_tools.jsoncopies) —create_check_runtool with requiredconclusion/title/summary+ optionaltextcreate_check_run.cjs) — callsrest.checks.create(); usesGITHUB_SHAforhead_sha; truncates fields at 65535 chars; classified asTHREAT_WARNING_REVIEWABLE_TYPESunified_prompt_step.go,tool_description_enhancer.go) — tools budget entry + constraint descriptions (max count, check run name)✨ PR Review Safe Output Test - Run 26267023456
Warning
Firewall blocked 6 domains
The following domains were blocked by the firewall during workflow execution:
accounts.google.comandroid.clients.google.comclients2.google.comcontentautofill.googleapis.comsafebrowsingohttpgateway.googleapis.comwww.google.comSee Network Configuration for more information.
✨ PR Review Safe Output Test - Run 26267820166
Warning
Firewall blocked 6 domains
The following domains were blocked by the firewall during workflow execution:
accounts.google.comandroid.clients.google.comclients2.google.comcontentautofill.googleapis.comsafebrowsingohttpgateway.googleapis.comwww.google.comSee Network Configuration for more information.
✨ PR Review Safe Output Test - Run 26269860246
Warning
Firewall blocked 6 domains
The following domains were blocked by the firewall during workflow execution:
accounts.google.comandroid.clients.google.comclients2.google.comcontentautofill.googleapis.comsafebrowsingohttpgateway.googleapis.comwww.google.comSee Network Configuration for more information.
pr-sous-chef: automatic branch update via run https://github.com/github/gh-aw/actions/runs/26271114276