Summary
When using CopilotKit's useHumanInTheLoop, the frontend sends two AG-UI requests for a single user interaction:
-
First leg — a normal agent/run request whose last message has role: "user". This matches extractLastUserMessage correctly and records/replays cleanly.
-
Second leg (continuation) — after the user approves/rejects, CopilotKit sends another agent/run request whose last message has role: "tool" (the approval result). This contains no user text, so extractLastUserMessage returns "" and the recorder writes match.message: "__NO_USER_MESSAGE__" — producing a fixture that can never match in replay.
What we'd like
A first-class way to record and replay HITL continuation requests. Concretely:
For recording — when the last message in the request is role: "tool", stamp the fixture with a match key derived from that message rather than falling back to the sentinel. The natural key is either the tool_call_id or the tool name, both of which are stable and unique per interaction.
For replay — matchesFixture (and the on-disk format) would need a parallel match field, e.g. match.toolCallId or match.toolResult, so aimock can route continuation requests to the right fixture without relying on user text.
Why this matters
Without this, any flow involving HITL (approval dialogs, human-in-the-loop confirmation steps) can only be recorded for the first leg. The continuation — which often triggers the most interesting state change can't be replayed deterministically, and the response shown after approval is always proxied to the live backend.
Proposed API sketch
Extend AGUIFixtureMatch with an optional toolCallId field:
interface AGUIFixtureMatch {
message?: string | RegExp;
toolCallId?: string; // ← new: matches when last message has this tool_call_id
toolName?: string;
stateKey?: string;
predicate?: (input: RunAgentInput) => boolean;
}
The recorder would write { match: { toolCallId: "<id>" } } for continuation requests. matchesFixture would check input.messages.at(-1)?.toolCallId === match.toolCallId (or similar). The on-disk sentinel (__NO_USER_MESSAGE__) would be replaced by an explicit __NO_MATCH_KEY__ or simply omitted.
Context
- CopilotKit version:
@copilotkit/react-core@1.56.3
- aimock version:
@copilotkit/aimock@1.18.0
- AG-UI version:
@ag-ui/client@0.0.52
We ran into this while adding aimock-based E2E specs for a useHumanInTheLoop agent. The first leg records and replays perfectly; the continuation does not.
Summary
When using CopilotKit's
useHumanInTheLoop, the frontend sends two AG-UI requests for a single user interaction:First leg — a normal
agent/runrequest whose last message hasrole: "user". This matchesextractLastUserMessagecorrectly and records/replays cleanly.Second leg (continuation) — after the user approves/rejects, CopilotKit sends another
agent/runrequest whose last message hasrole: "tool"(the approval result). This contains no user text, soextractLastUserMessagereturns""and the recorder writesmatch.message: "__NO_USER_MESSAGE__"— producing a fixture that can never match in replay.What we'd like
A first-class way to record and replay HITL continuation requests. Concretely:
For recording — when the last message in the request is
role: "tool", stamp the fixture with a match key derived from that message rather than falling back to the sentinel. The natural key is either thetool_call_idor the tool name, both of which are stable and unique per interaction.For replay —
matchesFixture(and the on-disk format) would need a parallel match field, e.g.match.toolCallIdormatch.toolResult, so aimock can route continuation requests to the right fixture without relying on user text.Why this matters
Without this, any flow involving HITL (approval dialogs, human-in-the-loop confirmation steps) can only be recorded for the first leg. The continuation — which often triggers the most interesting state change can't be replayed deterministically, and the response shown after approval is always proxied to the live backend.
Proposed API sketch
Extend
AGUIFixtureMatchwith an optionaltoolCallIdfield:The recorder would write
{ match: { toolCallId: "<id>" } }for continuation requests.matchesFixturewould checkinput.messages.at(-1)?.toolCallId === match.toolCallId(or similar). The on-disk sentinel (__NO_USER_MESSAGE__) would be replaced by an explicit__NO_MATCH_KEY__or simply omitted.Context
@copilotkit/react-core@1.56.3@copilotkit/aimock@1.18.0@ag-ui/client@0.0.52We ran into this while adding aimock-based E2E specs for a
useHumanInTheLoopagent. The first leg records and replays perfectly; the continuation does not.