Fix grader compatibility with OpenClaw transcripts

[jijivski]

Improve grader compatibility with current OpenClaw transcripts

The grader currently assumes a narrower transcript format than the one produced by current OpenClaw runtime, which can lead to false negatives.

Changes:

• read tool inputs from toolCall.arguments
• support file alongside path / file_path
• improve judge score parsing robustness

These changes do not alter task requirements; they only make grading align with real transcript output.

[ScuttleBot]

ScuttleBot review 🦀

Solid defensive fix. The grader was too rigid about transcript formats, causing false negatives on valid runs.

What's good:

• _coerce_score_value() handles the full zoo of judge response formats (nested dicts, string numbers, boolean rejection)
• Supporting file alongside path/file_path aligns with how OpenClaw actually emits tool calls
• The refactor into _extract_named_scores() and _extract_total_score() is cleaner than the previous inline conditionals

One question:

• Task file changes (task_08, task_10, task_18) — are these tested against transcripts from multiple agents? The file param support looks correct but I want to confirm this doesn't break Cursor/Windsurf/Claude Code grading.

Otherwise LGTM. This will reduce the "score 0 but the agent clearly did the work" cases.

[ScuttleBot]

Merge conflict resolution available

I've rebased this PR onto main and resolved the conflict in lib_grading.py. The conflict was between the new _parse_judge_text() function (added in main via #87) and the helper functions in this PR (_coerce_score_value, _extract_named_scores, _extract_total_score).

Resolution: Keep both — _parse_judge_text() first, then the helper functions. Both are needed.

@jijivski — could you rebase your branch onto main? The resolution is straightforward:

git fetch upstream
git rebase upstream/main
# Resolve lib_grading.py by keeping both function sets
git add scripts/lib_grading.py
git rebase --continue
git push --force-with-lease

Alternatively, @olearycrew has admin access and can use GitHub's "Update branch" button if the repo allows maintainer edits on this PR.

[olearycrew]

@jijivski can you take a look at the conflicts here?

[olearycrew]

@jijivski can you fix the issues from the linter as well as fix the conflicts with main? Thanks!

[kilo-code-bot]

Code Review Summary

Status: No Issues Found | Recommendation: Merge

This PR is well-implemented. The refactoring of _parse_judge_response to prefer parsing the most recent assistant text (skipping NO_REPLY and noise) is a robust improvement. The extracted helpers (_coerce_score_value, _extract_named_scores, _extract_total_score) are clean and well-tested. The task grader fixes (arguments fallback, file param support) are correct and backwards-compatible.

Files Reviewed (4 files)

• scripts/lib_grading.py
• tasks/task_market_research.md
• tasks/task_memory.md
• tasks/task_workflow.md
• tests/test_lib_grading.py

---

_{Reviewed by claude-4.6-sonnet-20260217 · 157,987 tokens}

[jijivski]

@olearycrew Hi, I've synced with latest main in one commit, then fixed judge transcript parsing in a follow-up commit.

For Claude Opus 4.5 multi-part judge transcripts, we now prefer the final assistant judgment JSON over earlier echoed tool JSON / waiting messages. Verified locally on the previously failing transcripts.