Phase 3 smoketest — auto-fix pipeline verification#280
Open
scszcoder wants to merge 1 commit into
Open
Conversation
scszcoder
pushed a commit
that referenced
this pull request
May 9, 2026
A direct-invoke (not AppSync-routed) Lambda field that exercises the
Phase 3 GitHub pipeline without involving the LLM. Modes:
dryrun (default):
1. _get_github_pat (reports found/missing + boto3 diagnostics on miss)
2. GET /user (auth check)
3. GET /repos/{o}/{r} (repo + permission check)
pr:
Steps 1-3 above, then stages a single harmless file
(agent/.cra_smoketest_{epoch}.md) in /tmp/cra_smoketest_{epoch}/
and calls _submit_fix_pr to create branch + commit + PR. Returns
pr_url so the operator can verify the full chain end-to-end.
Diagnostics on PAT lookup failure include:
- which env var(s) are set (without leaking values)
- direct boto3 call to surface the real exception when GITHUB_PAT_SECRET_ARN
is set but lookup fails (catches paste typos, wrong ARN, missing IAM,
invalid JSON shape, etc.)
Invocation:
aws lambda invoke --function-name skill_editor_agent \\
--cli-binary-format raw-in-base64-out \\
--payload '{"info":{"fieldName":"reqCodeAgentSmoketest"},
"arguments":{"input":{"mode":"pr"}}}' \\
/tmp/out.json && cat /tmp/out.json
Verified end-to-end: PR #280 opened on scszcoder/eCan.ai during the
initial test run (2026-05-08). Cleanup is a single 'close + delete branch'
click on the PR.
Bug fixed in same commit: the ship_ok detector previously evaluated False
on success because _submit_fix_pr's output legitimately starts with
'[1/3]'. Replaced the prefix check with '[3/3] ✅' presence + absence of
' error]' tokens.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated test of the Code Review Agent → branch → commit → PR flow. No code changes — adds a single marker file under
agent/.cra_smoketest_*.md. Safe to close + delete branch.