Skip to content

Setup wake hook E2E test loop #100

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jaruesink wants to merge 6 commits into from
Closed

Setup wake hook E2E test loop #100

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
a62e6a0
Setup wake hook E2E test loop
jaruesink Feb 3, 2026
931ae6b
Fix epic ID to use prefixed name
jaruesink Feb 3, 2026
f283ec5
chore(wake-hook): add marker file for on-iteration hook test [skip ci]
jaruesink Feb 3, 2026
fcc7a12
chore(wake-hook-test): add verify script for marker.txt (devagent-wak…
jaruesink Feb 3, 2026
81446a1
chore(wake-hook-test): add verification results summary (devagent-wak…
jaruesink Feb 3, 2026
a80280e
chore(wake-hook-test): add epic revise report (devagent-wake-hook-tes…
jaruesink Feb 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions .devagent/plugins/ralph/runs/fix-cursor-logs-pr-feedback-2026-01-31.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
{
"$schema": "../core/schemas/loop.schema.json",
"run": {
"git": {
"base_branch": "main",
"working_branch": "feature/fix-cursor-agent-logs"
},
"execution": {
"max_iterations": 5,
"log_dir": ".devagent/plugins/ralph/logs/fix-cursor-logs-pr-feedback"
}
},
"epic": {
"id": "devagent-pr94-feedback",
"title": "PR #94 CodeRabbit Feedback Fixes",
"description": "Address the 4 actionable review comments from CodeRabbit on PR #94 (fix-cursor-agent-logs).\n\nPR: https://github.com/lambda-curry/devagent/pull/94\n\nBranch: feature/fix-cursor-agent-logs"
},
"availableAgents": ["engineering"],
"tasks": [
{
"id": "1",
"title": "Skip ensureLogDirectoryExists when stored path exists",
"role": "engineering",
"description": "In api.logs.$taskId.stream.ts, skip the ensureLogDirectoryExists() call when storedLogPath is available.\n\nCurrently the route always creates the default log directory before using the stored path, which can throw and fail the stream even when the stored path is valid elsewhere.\n\n**Fix:** Only call ensureLogDirectoryExists() when we don't have a storedLogPath override.\n\nBranch: feature/fix-cursor-agent-logs",
"acceptance_criteria": [
"ensureLogDirectoryExists is only called when no storedLogPath is present",
"Stream route works correctly when stored path points to a different location than default"
],
"dependencies": [],
"labels": ["bugfix", "pr-feedback"]
},
{
"id": "2",
"title": "Move resolveLogPathForRead inside validation try/catch",
"role": "engineering",
"description": "In api.logs.$taskId.ts, move the resolveLogPathForRead call inside the validation try/catch block.\n\nCurrently it can throw INVALID_TASK_ID before validation and produce a 500 instead of the intended 400 response.\n\n**Fix:** Call resolveLogPathForRead only after taskId is validated.\n\nBranch: feature/fix-cursor-agent-logs",
"acceptance_criteria": [
"Invalid task IDs return 400 response, not 500",
"resolveLogPathForRead is only called after taskId is validated"
],
"dependencies": [],
"labels": ["bugfix", "pr-feedback"]
},
{
"id": "3",
"title": "Add optional logPath param to logFileExists",
"role": "engineering",
"description": "In logs.server.ts, update logFileExists to accept an optional logPath parameter instead of always recalculating.\n\n**Signature:** `logFileExists(taskId: string, logPath?: string)`\n**Implementation:** `pathToCheck = logPath || getLogFilePath(taskId)`\n\nBranch: feature/fix-cursor-agent-logs",
"acceptance_criteria": [
"logFileExists accepts optional logPath parameter",
"When logPath is provided, it uses that path directly",
"When logPath is not provided, it calculates path as before",
"All call sites updated to pass resolved path where available"
],
"dependencies": [],
"labels": ["refactor", "pr-feedback"]
},
{
"id": "4",
"title": "Update docs metadata and fix markdown formatting",
"role": "engineering",
"description": "Update Last Updated to 2026-01-31 in task AGENTS.md, and fix markdown table formatting (MD060) in the plan doc.\n\n**Files:**\n- .devagent/workspace/tasks/active/2026-01-30_fix-cursor-agent-logs/AGENTS.md\n- .devagent/workspace/tasks/active/2026-01-30_fix-cursor-agent-logs/plan/2026-01-30_fix-cursor-agent-logs-plan.md\n\n**Fixes:**\n- Change Last Updated date to 2026-01-31\n- Ensure table separator row has spaces around pipes (MD060)\n\nBranch: feature/fix-cursor-agent-logs",
"acceptance_criteria": [
"Last Updated shows 2026-01-31",
"Markdown tables pass MD060 lint"
],
"dependencies": [],
"labels": ["docs", "pr-feedback"]
}
]
}
84 changes: 84 additions & 0 deletions .devagent/plugins/ralph/runs/ralph-iteration-hooks-2026-01-31.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
{
"$schema": "../core/schemas/loop.schema.json",
"run": {
"git": {
"base_branch": "main",
"working_branch": "feature/ralph-iteration-hooks"
},
"execution": {
"max_iterations": 6,
"log_dir": ".devagent/plugins/ralph/logs/ralph-iteration-hooks"
}
},
"epic": {
"id": "devagent-iteration-hooks",
"title": "Add completion and iteration hooks to Ralph",
"description": "Add callback hooks to Ralph for loop completion and per-iteration updates.\n\nBranch: feature/ralph-iteration-hooks"
},
"availableAgents": ["engineering"],
"tasks": [
{
"id": "1",
"title": "Add --on-complete CLI arg to ralph.sh",
"role": "engineering",
"description": "Add a new --on-complete argument to ralph.sh that accepts a path to a script.\n\n**File:** .devagent/plugins/ralph/tools/ralph.sh\n\n**Changes:**\n1. Add ON_COMPLETE_HOOK variable parsing\n2. Accept --on-complete <path> in the CLI args loop\n3. Pass it to ralph.ts via --on-complete flag\n\nBranch: feature/ralph-iteration-hooks",
"acceptance_criteria": [
"ralph.sh accepts --on-complete <script-path>",
"The value is passed to bun ralph.ts"
],
"dependencies": [],
"labels": ["feature"]
},
{
"id": "2",
"title": "Add --on-complete support to ralph.ts",
"role": "engineering",
"description": "Modify ralph.ts to call the on-complete hook when the loop finishes.\n\n**File:** .devagent/plugins/ralph/tools/ralph.ts\n\n**Changes:**\n1. Parse --on-complete CLI arg\n2. At the end of the loop (after all iterations complete or max reached), call the hook\n3. Payload should include: status, epicId, iterations, maxIterations, exitReason, durationSec, branch, logTail\n\n**Example:**\n```typescript\nif (onCompleteHook) {\n const payload = JSON.stringify({\n status: allTasksComplete ? 'completed' : 'blocked',\n epicId,\n iterations: currentIteration,\n maxIterations,\n exitReason,\n durationSec: (Date.now() - startTime) / 1000,\n branch: currentBranch,\n logTail: getLastNCharsOfLog(3000)\n });\n try {\n execSync(`echo '${payload}' | ${onCompleteHook}`, { stdio: 'inherit' });\n } catch (e) {\n console.error('Warning: on-complete hook failed:', e);\n }\n}\n```\n\nBranch: feature/ralph-iteration-hooks",
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Feb 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Command injection risk in example code.

The example uses shell string interpolation which is vulnerable to command injection:

execSync(`echo '${payload}' | ${onCompleteHook}`, { stdio: 'inherit' });

If payload contains a single quote (valid in JSON string values), or if onCompleteHook contains shell metacharacters, this could break or be exploited. When implementing, use spawn with stdin piping instead of shell interpolation.

🔒 Safer implementation pattern 
// Instead of shell interpolation, pipe JSON via spawn stdin:
import { spawn } from 'child_process';

const child = spawn(onCompleteHook, [], { stdio: ['pipe', 'inherit', 'inherit'] });
child.stdin.write(payload);
child.stdin.end();

This avoids shell parsing entirely and safely passes arbitrary JSON content.

🤖 Prompt for AI Agents 
In @.devagent/plugins/ralph/runs/ralph-iteration-hooks-2026-01-31.json at line
36, The current example in ralph.ts builds a shell string and calls execSync
with interpolation (execSync(`echo '${payload}' | ${onCompleteHook}`)), which is
vulnerable to command injection and breaks on quotes; replace this with
child_process.spawn and pipe the JSON payload to the child process's stdin
instead: treat onCompleteHook as the executable (or split it into
executable+args if you accept args), call spawn(onCompleteHook, argsArray, {
stdio: ['pipe','inherit','inherit'] }), write the JSON payload to child.stdin
and end it, and handle errors/exit codes (log warnings on failure) so the
onCompleteHook invocation is safe; update the code paths around parsing
--on-complete, the hook invocation, and use getLastNCharsOfLog, epicId,
currentIteration, maxIterations, exitReason, startTime/currentBranch to build
the payload as described.

"acceptance_criteria": [
"ralph.ts parses --on-complete argument",
"Hook is called when loop ends with proper JSON payload",
"Hook failures are logged but don't crash Ralph"
],
"dependencies": ["1"],
"labels": ["feature"]
},
{
"id": "3",
"title": "Add --on-iteration CLI arg to ralph.sh",
"role": "engineering",
"description": "Add a new --on-iteration argument to ralph.sh that accepts a path to a script.\n\n**File:** .devagent/plugins/ralph/tools/ralph.sh\n\n**Changes:**\n1. Add ON_ITERATION_HOOK variable parsing (similar to --on-complete)\n2. Accept --on-iteration <path> in the CLI args loop\n3. Pass it to ralph.ts via --on-iteration flag\n\nBranch: feature/ralph-iteration-hooks",
"acceptance_criteria": [
"ralph.sh accepts --on-iteration <script-path>",
"The value is passed to bun ralph.ts"
],
"dependencies": ["1"],
"labels": ["feature"]
},
{
"id": "4",
"title": "Add --on-iteration support to ralph.ts",
"role": "engineering",
"description": "Modify ralph.ts to call the on-iteration hook after each iteration completes.\n\n**File:** .devagent/plugins/ralph/tools/ralph.ts\n\n**Changes:**\n1. Parse --on-iteration CLI arg\n2. After each iteration (when a task completes), call the hook with JSON payload\n3. Payload: epicId, iteration, maxIterations, taskId, taskTitle, taskStatus, tasksCompleted, tasksRemaining, iterationDurationSec\n\n**Example:**\n```typescript\nif (onIterationHook) {\n const payload = JSON.stringify({\n epicId,\n iteration: currentIteration,\n maxIterations,\n taskId: task.id,\n taskTitle: task.title,\n taskStatus: taskResult.status, // 'completed' | 'failed' | 'blocked'\n tasksCompleted,\n tasksRemaining,\n iterationDurationSec\n });\n try {\n execSync(`echo '${payload}' | ${onIterationHook}`, { stdio: 'inherit' });\n } catch (e) {\n console.error('Warning: on-iteration hook failed:', e);\n }\n}\n```\n\nBranch: feature/ralph-iteration-hooks",
"acceptance_criteria": [
"ralph.ts parses --on-iteration argument",
"Hook is called after each iteration with proper JSON payload",
"Hook failures are logged but don't stop the loop"
],
"dependencies": ["2", "3"],
"labels": ["feature"]
},
{
"id": "5",
"title": "Test hooks work end-to-end",
"role": "engineering",
"description": "Verify both hooks work by running a quick test.\n\n**Steps:**\n1. Create a test hook script that appends to a file\n2. Run ralph with both --on-iteration and --on-complete pointing to that hook\n3. Verify the file contains both iteration and completion payloads\n\nBranch: feature/ralph-iteration-hooks",
"acceptance_criteria": [
"Both hooks are called during loop execution",
"Payloads contain all expected fields",
"Loop continues even if hooks fail"
],
"dependencies": ["4"],
"labels": ["test"]
}
]
}
56 changes: 56 additions & 0 deletions .devagent/plugins/ralph/runs/wake-hook-test-2026-02-03.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
{
"extends": "generic-ralph-loop.json",
"run": {
"git": {
"base_branch": "main",
"working_branch": "feature/wake-hook-test"
},
"execution": {
"max_iterations": 10,
"log_dir": "logs/ralph"
}
},
"epic": {
"id": "devagent-wake-hook-test",
"title": "Wake Hook E2E Test",
"description": "Minimal loop to validate that Ralph iteration/completion hooks correctly wake the main Clawdbot agent via /hooks/wake."
},
"tasks": [
{
"id": "1",
"title": "Create test marker file",
"description": "Create a file at .devagent/workspace/tests/wake-hook-test/marker.txt with the content: 'Wake hook test - task 1 complete'. This validates that the on-iteration hook fires after a simple file creation task.",
"role": "engineering",
"acceptance_criteria": [
"File exists at .devagent/workspace/tests/wake-hook-test/marker.txt",
"File contains the expected text"
],
"dependencies": [],
"labels": ["test"]
},
{
"id": "2",
"title": "Create verification script",
"description": "Create a script at .devagent/workspace/tests/wake-hook-test/verify.sh that reads marker.txt and exits 0 if it contains the expected text, 1 otherwise. Make it executable.",
"role": "engineering",
"acceptance_criteria": [
"Script exists and is executable",
"Running it returns exit code 0"
],
"dependencies": ["1"],
"labels": ["test"]
},
{
"id": "3",
"title": "Run verification and document results",
"description": "Run the verify.sh script and create a summary file at .devagent/workspace/tests/wake-hook-test/results.md documenting that all tasks passed. This is the final task — the on-complete hook should fire after this.",
"role": "engineering",
"acceptance_criteria": [
"verify.sh exits 0",
"results.md exists with pass summary"
],
"dependencies": ["2"],
"labels": ["test"]
}
]
}
6 changes: 3 additions & 3 deletions .devagent/plugins/ralph/tools/config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
"beads_payload": ".devagent/plugins/ralph/output/beads-payload.json",
"git": {
"base_branch": "main",
"working_branch": "feat/multi-project-support"
"working_branch": "feature/wake-hook-test"
},
"roles": {
"engineering": "Code Wizard",
Expand All @@ -34,12 +34,12 @@
},
"execution": {
"require_confirmation": false,
"max_iterations": 25
"max_iterations": 10
},
"agents": {
"engineering": "engineering-agent.json",
"qa": "qa-agent.json",
"design": "design-agent.json",
"project-manager": "project-manager-agent.json"
}
}
}
50 changes: 50 additions & 0 deletions .devagent/plugins/ralph/tools/verify-on-iteration-e2e.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
#!/usr/bin/env bash
# E2E verification for --on-iteration: run ralph with the test hook and verify the output file.
# Requires: repo root, epic devagent-iteration-hooks exists, branch feature/ralph-iteration-hooks,
# at least one ready task for the hook to be called.
# Usage: from repo root: .devagent/plugins/ralph/tools/verify-on-iteration-e2e.sh

set -e
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../../../../.." && pwd)"
cd "$REPO_ROOT"

OUT_FILE="$(mktemp)"
trap 'rm -f "$OUT_FILE"' EXIT

SCRIPT_DIR="$(dirname "${BASH_SOURCE[0]}")"
RUN_FILE="${SCRIPT_DIR}/../runs/ralph-iteration-hooks-2026-01-31.json"
HOOK_SCRIPT="${SCRIPT_DIR}/test-on-iteration-hook.sh"

if [ ! -f "$RUN_FILE" ]; then
echo "Error: Run file not found at $RUN_FILE" >&2
exit 1
fi
if [ ! -x "$HOOK_SCRIPT" ]; then
echo "Error: Hook script not executable at $HOOK_SCRIPT" >&2
exit 1
fi

export OUT_FILE
export RALPH_MAX_ITERATIONS=1

echo "Running ralph with --on-iteration (max_iterations=1)..."
"${SCRIPT_DIR}/ralph.sh" --run "$RUN_FILE" --on-iteration "$HOOK_SCRIPT" || true

# Ralph may exit non-zero (e.g. no ready tasks, or agent failure); we only care that the hook was called
if [ ! -s "$OUT_FILE" ]; then
echo "Error: Hook output file is empty or missing. Ensure epic has at least one ready task." >&2
exit 1
fi

EXPECTED_KEYS='epicId,iteration,maxIterations,taskId,taskTitle,taskStatus,tasksCompleted,tasksRemaining,iterationDurationSec'
while IFS= read -r line; do
[ -z "$line" ] && continue
for key in epicId iteration maxIterations taskId taskTitle taskStatus tasksCompleted tasksRemaining iterationDurationSec; do
if ! echo "$line" | jq -e ".$key" >/dev/null 2>&1; then
echo "Error: Payload missing key '$key': $line" >&2
exit 1
fi
done
done < "$OUT_FILE"

echo "OK: Hook was called; payload(s) contain all expected fields ($EXPECTED_KEYS)."
10 changes: 10 additions & 0 deletions .devagent/plugins/ralph/tools/vitest.config.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
import { defineConfig } from 'vitest/config';

export default defineConfig({
test: {
globals: true,
environment: 'node',
include: ['**/*.test.ts'],
exclude: ['**/node_modules/**'],
},
});
Loading