Summary
Audit of safe-output validation errors over the last 24 hours surfaced one tool-description issue where an agent passed "#aw_summary" (with the # prefix) as add_labels.item_number, causing the validator to reject the entry. The agent's workflow prompt was correct; the confusion comes from the standard safe-outputs preamble teaching #aw_X for body references without disambiguating that field values must omit the #.
A secondary tool-description inconsistency was found: add_labels.item_number declares type: "number" in safe_outputs_tools.json, but the validator accepts strings matching the temporary-id pattern ^aw_[A-Za-z0-9]{3,12}$. The string form is undocumented in the schema, yet the validator's error message acknowledges it ("must be a positive integer or temporary ID").
(A separate cluster of 7 runs hit a CLI-bridge bug where printf '{json}' | safeoutputs <tool> . produced an empty {"type":"<tool>"} entry — see the bottom "Out of scope" section. Those are not tool-description issues and are filed for reference only.)
Evidence — the tool-description error
Workflow: Contribution Check
Run: §25967977306 — claude engine
The agent created an issue with temporary_id: "aw_summary" (correct), then attempted to add labels referencing it. What got emitted:
{"item_number":"#aw_summary","labels":["lgtm","needs-work"],"type":"add_labels"}Validator response in agent_output.json:
Line 3: add_labels 'item_number' must be a positive integer or temporary ID (got: #aw_summary)
The correct payload would have been:
{"item_number":"aw_summary","labels":["lgtm","needs-work"],"type":"add_labels"}Why the agent got confused
The standard safe-outputs preamble (included in every workflow prompt) says:
temporary_id: optional cross-reference field (e.g. use #aw_abc1 in a body).
This teaches the #aw_X notation for in-body markdown references (which get replaced with real issue numbers after creation) but never explicitly states that when the same temporary_id is used as a field value (e.g. item_number, issue_number, parent), the # prefix must be omitted.
The agent generalized incorrectly: "I created an issue with temporary_id aw_summary and I reference it in bodies as #aw_summary, so the item_number field must also take #aw_summary."
Root cause analysis
-
Schema type does not match validator behavior — add_labels.item_number is declared type: "number" only. The validator's error message implies strings are accepted ("must be a positive integer or temporary ID"). Either the schema should accept ["number", "string"] with the temporary_id pattern, or the validator should reject strings outright.
-
Preamble teaches one syntax, doesn't disambiguate the other — The preamble line cited above is the only mention of #aw_X and aw_X. There is no example showing the unprefixed form being used as a field value, nor a sentence explicitly contrasting body-reference vs field-value forms.
-
Tool description silent on the convention — The item_number description in add_comment, add_labels, and parent in create_issue mentions "Can also be a temporary_id (e.g., 'aw_abc123')" but does not warn against the # prefix or against confusing it with body references.
Recommended improvements
A. Fix pkg/workflow/js/safe_outputs_tools.json
add_labels.item_number — change schema to match validator behavior:
- "type": ["number", "string"],
- "pattern": "^(\d+|aw_[A-Za-z0-9]{3,12})$",
"description": "Issue or PR number ...; OR a temporary_id from a previously created issue in the same run (e.g. 'aw_abc123' — without leading '#')."
}
2. **`add_comment.item_number`, `add_comment.pr_number`, `add_comment.pr`, `create_issue.parent`** — same `(e.g. 'aw_abc123' — without leading '#')` clarification in the description. Today they say `"Can also be a temporary_id (e.g., 'aw_abc123')"` which is close but not explicit.
3. **Sweep the rest** of `safe_outputs_tools.json` for any field whose description mentions temporary IDs (`assign_to_agent.issue_number`, `update_issue.issue_number`, etc.) and apply the same clarification.
#### B. Improve the safe-outputs preamble
The preamble line is currently emitted by the workflow compiler — likely from a template under `pkg/workflow/` or `actions/setup/js/`. Replace:
> `temporary_id: optional cross-reference field (e.g. use #aw_abc1 in a body).`
with something that contrasts both forms explicitly. Suggested wording:
> `temporary_id: optional cross-reference field. Format: aw_ + 3–8 alphanumeric chars (/^aw_[A-Za-z0-9]{3,8}$/). When **referencing** a temporary_id in a body or markdown text, use #aw_abc1 — the # is replaced with the real issue/PR number after creation. When using a temporary_id as a **field value** (item_number, issue_number, parent, etc.), pass the raw id without the #: 'aw_abc1'. Omit entirely when not needed.`
### Affected workflows in this audit window
| Workflow | Run | Tool | Issue |
|---|---|---|---|
| Contribution Check | [§25967977306](https://github.com/github/gh-aw/actions/runs/25967977306) | `add_labels` | `"#aw_summary"` instead of `"aw_summary"` |
Only one workflow tripped this in the 24h window, but the underlying ambiguity affects every safe-output tool that accepts a temporary_id in a field — so the fix is preventive across the whole tool catalog.
### Implementation checklist
- [ ] Update `add_labels.item_number` schema in `pkg/workflow/js/safe_outputs_tools.json` to accept `["number", "string"]` with the temporary_id pattern, OR explicitly state in the description that temporary IDs are accepted despite the type being `number`
- [ ] Add the `(e.g. 'aw_abc123' — without leading '#')` clarification to `item_number`/`issue_number`/`parent` field descriptions across all tools that accept temporary IDs
- [ ] Replace the single-line `temporary_id` preamble entry with the two-clause version that contrasts body-reference (`#aw_X`) vs field-value (`aw_X`) usage
- [ ] `make recompile` to regenerate compiled workflows so the new preamble lands in every workflow's prompt
- [ ] `make test` to ensure no regressions
- [ ] Monitor next 3 daily audits for recurrence of `must be a positive integer or temporary ID` validation errors
### Out of scope (filed for reference, not part of this issue)
- **CLI-bridge bug** — 7 runs (1 Bot Detection update_issue + 6 Test Quality Sentinel submit_pull_request_review) had `printf '{...}' | safeoutputs <tool> .` produce an empty `{"type":"<tool>"}` entry. The agent retried via heredoc/cat and succeeded each time, proving the schema was understood. Affected runs: [§25969843722](https://github.com/github/gh-aw/actions/runs/25969843722), [§25970175071](https://github.com/github/gh-aw/actions/runs/25970175071), [§25968749492](https://github.com/github/gh-aw/actions/runs/25968749492), [§25964660443](https://github.com/github/gh-aw/actions/runs/25964660443), [§25969164447](https://github.com/github/gh-aw/actions/runs/25969164447), [§25966197731](https://github.com/github/gh-aw/actions/runs/25966197731), [§25964635944](https://github.com/github/gh-aw/actions/runs/25964635944). This is an MCP CLI bridge issue, not a tool-description issue. Worth a separate investigation.
- **Count-limit overflows** (2 runs): PR Sous Chef [§25970430595](https://github.com/github/gh-aw/actions/runs/25970430595) emitted 2 noop entries; Daily Code Metrics [§25969831244](https://github.com/github/gh-aw/actions/runs/25969831244) emitted 6 upload_asset entries (max 5). Both are workflow-prompt issues, not tool-description issues.
- **Deprecated alias usage** (2 runs, PR Sous Chef): used `pr_number` (deprecated alias) instead of `item_number` for `add_comment`. Documented as alias, validates fine — not an error, but worth migrating.
### References
- Tool schema: `pkg/workflow/js/safe_outputs_tools.json`
- Validator error site: `actions/setup/js/safe_output_validator.cjs` (error string `"must be a positive integer or temporary ID"`)
- Validation error in `agent_output.json` of [§25967977306](https://github.com/github/gh-aw/actions/runs/25967977306)
> Generated by [⚡ Daily Safe Output Tool Optimizer](https://github.com/github/gh-aw/actions/runs/25972929656) · ● 28.6M · [◷](https://github.com/search?q=repo%3Agithub%2Fgh-aw+is%3Aissue+%22gh-aw-workflow-call-id%3A+github%2Fgh-aw%2Fdaily-safe-output-optimizer%22&type=issues)
> - [x] expires <!-- gh-aw-expires: 2026-05-18T21:19:58.841Z --> on May 18, 2026, 9:19 PM UTC
<!-- gh-aw-agentic-workflow: Daily Safe Output Tool Optimizer, engine: claude, model: auto, id: 25972929656, workflow_id: daily-safe-output-optimizer, run: https://github.com/github/gh-aw/actions/runs/25972929656 -->
<!-- gh-aw-workflow-id: daily-safe-output-optimizer -->
<!-- gh-aw-workflow-call-id: github/gh-aw/daily-safe-output-optimizer -->
Summary
Audit of safe-output validation errors over the last 24 hours surfaced one tool-description issue where an agent passed
"#aw_summary"(with the#prefix) asadd_labels.item_number, causing the validator to reject the entry. The agent's workflow prompt was correct; the confusion comes from the standard safe-outputs preamble teaching#aw_Xfor body references without disambiguating that field values must omit the#.A secondary tool-description inconsistency was found:
add_labels.item_numberdeclarestype: "number"insafe_outputs_tools.json, but the validator accepts strings matching the temporary-id pattern^aw_[A-Za-z0-9]{3,12}$. The string form is undocumented in the schema, yet the validator's error message acknowledges it ("must be a positive integer or temporary ID").(A separate cluster of 7 runs hit a CLI-bridge bug where
printf '{json}' | safeoutputs <tool> .produced an empty{"type":"<tool>"}entry — see the bottom "Out of scope" section. Those are not tool-description issues and are filed for reference only.)Evidence — the tool-description error
Workflow: Contribution Check
Run: §25967977306 — claude engine
The agent created an issue with
temporary_id: "aw_summary"(correct), then attempted to add labels referencing it. What got emitted:{"item_number":"#aw_summary","labels":["lgtm","needs-work"],"type":"add_labels"}Validator response in
agent_output.json:The correct payload would have been:
{"item_number":"aw_summary","labels":["lgtm","needs-work"],"type":"add_labels"}Why the agent got confused
The standard safe-outputs preamble (included in every workflow prompt) says:
This teaches the
#aw_Xnotation for in-body markdown references (which get replaced with real issue numbers after creation) but never explicitly states that when the same temporary_id is used as a field value (e.g.item_number,issue_number,parent), the#prefix must be omitted.The agent generalized incorrectly: "I created an issue with temporary_id
aw_summaryand I reference it in bodies as#aw_summary, so the item_number field must also take#aw_summary."Root cause analysis
Schema type does not match validator behavior —
add_labels.item_numberis declaredtype: "number"only. The validator's error message implies strings are accepted ("must be a positive integer or temporary ID"). Either the schema should accept["number", "string"]with the temporary_id pattern, or the validator should reject strings outright.Preamble teaches one syntax, doesn't disambiguate the other — The preamble line cited above is the only mention of
#aw_Xandaw_X. There is no example showing the unprefixed form being used as a field value, nor a sentence explicitly contrasting body-reference vs field-value forms.Tool description silent on the convention — The
item_numberdescription inadd_comment,add_labels, andparentincreate_issuementions "Can also be a temporary_id (e.g., 'aw_abc123')" but does not warn against the#prefix or against confusing it with body references.Recommended improvements
A. Fix
pkg/workflow/js/safe_outputs_tools.jsonadd_labels.item_number— change schema to match validator behavior:"item_number": {"description": "Issue or PR number ...; OR a temporary_id from a previously created issue in the same run (e.g. 'aw_abc123' — without leading '#')."
}