Conversation
Filter out 'running' status in the alarm pre-phase before calling upsertContainerStatus(). Running is the steady-state for healthy agents and a no-op in applyEvent(), so recording it just bloats the event table (~720 events/hour/agent). Non-running statuses (stopped, error, unknown) still get inserted for reconciler detection.
Add a debug endpoint that runs the reconciler against current live state and returns the actions it would emit without applying them. This enables inspecting what the reconciler thinks should happen at any given moment. - Add debugDryRun() method to TownDO that calls reconciler.reconcile() and returns actions + metrics without calling applyAction() - Add POST /debug/towns/:townId/reconcile-dry-run route following the same unauthenticated debug pattern as GET /debug/towns/:townId/status - Response includes actions array, actionsEmitted count, actionsByType breakdown, and pendingEventCount
* feat(claw): evaluate button-vs-card feature flag for PostHog experiment tracking
* fix(claw): move button-vs-card flag eval to CreateInstanceCard
Moves useFeatureFlagVariantKey('button-vs-card') from ClawDashboard
(which renders for all users including those with existing instances)
to CreateInstanceCard (which only renders for users who haven't
provisioned yet). This scopes the experiment exposure to users who
can actually see the create CTA, avoiding population dilution.
* feat(gastown): add POST /debug/reconcile-dry-run endpoint
Add a debug endpoint that runs the reconciler against current live state
and returns the actions it would emit without applying them. This enables
inspecting what the reconciler thinks should happen at any given moment.
- Add debugDryRun() method to TownDO that calls reconciler.reconcile()
and returns actions + metrics without calling applyAction()
- Add POST /debug/towns/:townId/reconcile-dry-run route following the
same unauthenticated debug pattern as GET /debug/towns/:townId/status
- Response includes actions array, actionsEmitted count, actionsByType
breakdown, and pendingEventCount
* fix(gastown): drain pending events in debugDryRun() before reconciling
Wrap debugDryRun() in a SQLite savepoint so it can drain and apply
pending town_events (Phase 0) before running reconcile (Phase 1),
matching the real alarm loop behavior. The savepoint is rolled back
in a finally block so the endpoint remains fully side-effect-free.
Adds eventsDrained to the returned metrics.
---------
Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>
Co-authored-by: Pedro Heyerdahl <pedro@kilocode.ai>
Co-authored-by: Pedro Heyerdahl <61753986+pedroheyerdahl@users.noreply.github.com>
…y debugging
Adds debugReplayEvents(from, to) method to Town.do.ts that queries all
town_events in a time range (regardless of processed_at), applies them
to reconstruct state transitions, runs the reconciler, and returns the
computed actions and a state snapshot. Uses a SQLite SAVEPOINT that is
rolled back so the endpoint remains fully side-effect-free.
Route: POST /debug/towns/:townId/replay-events
Body: { from: ISO, to: ISO }
Response: { eventsReplayed, actions, stateSnapshot }
![kilo-code-bot[bot]](https://avatars.githubusercontent.com/in/2193792?s=60&v=4){kind=small}
Contributor
Code Review SummaryStatus: 2 Issues Found | Recommendation: Address before merge Overview
Fix these issues in Kilo Cloud Issue Details (click to expand)WARNING
Other Observations (not in diff)Issues found in unchanged code that cannot receive inline comments:
Files Reviewed (1 files)
Reviewed by gpt-5.4-20260305 · 455,888 tokens |
…afana dashboard panels (#1372) - Extend writeEvent() to support double3-double10 fields for reconciler metrics - Emit reconciler_tick event after each alarm tick with all 9 metrics - Add Reconciler row to Grafana dashboard with 6 panels: 1. Events drained per tick (timeseries) 2. Actions emitted per tick by type (stacked bar) 3. Side effects attempted/succeeded/failed (timeseries) 4. Invariant violations (stat with >0 alert threshold) 5. Reconciler wall clock time (timeseries with >500ms threshold) 6. Pending event queue depth (gauge with >50 threshold)
…query Add a caveat comment and response field to debugReplayEvents explaining that events are re-applied on top of live state, not from a pre-window snapshot — results are approximate, useful for debugging event flow but not faithful historical reconstruction. Fix the Grafana 'Pending Event Queue Depth' gauge to show the latest row's double8 value instead of averaging across the time window.
…afa/4763028e/head
pandemicsyn
approved these changes
Mar 24, 2026
Each invariant violation now triggers Sentry.captureMessage with structured context (invariant number, message, townId) as both extra data and tags. Existing analytics event emission is preserved. Added TODO for future auto-recovery of invariant #7 (working agent with no hook).
| }); | ||
|
|
||
| for (const violation of violations) { | ||
| Sentry.captureMessage( |
Contributor
There was a problem hiding this comment.
WARNING: Persistent invariants will flood Sentry on every alarm tick
alarm() runs on the 5s active interval, so a long-lived invariant now calls Sentry.captureMessage() every tick until the town goes idle. One stuck town can easily generate ~17k duplicate Sentry events per day, which will drown the real signal and burn quota. Please dedupe or rate-limit these reports (for example, only emit when the invariant set changes) before sending them to Sentry.
jrf0110
deleted the
convoy/reconciler-phase-5-debug-endpoints-grafa/4763028e/head
branch
Contributor
Author
|
Refinery code review passed. All quality gates pass. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
POST /debug/towns/:townId/replay-eventsendpoint that replays town events from a given time range for debugging purposes. The endpoint:from/toISO timestamps, queries alltown_eventsin that range (regardless ofprocessed_at)reconciler.applyEvent()to reconstruct state transitionsreconciler.reconcile()against the resulting state to compute what actions would be emittedAlso includes the preceding commits on this convoy branch: dry-run reconciler endpoint, debug dry-run with event draining, and a fix for skipping container_status events.
Verification
debugDryRunendpoint conventions (SAVEPOINT/ROLLBACK, parameterized queries, Zod validation, eslint-disable comments)town_events,TownEventRecord,reconciler,query,Actionall correctly imported?placeholdersVisual Changes
N/A
Reviewer Notes
/debug/route, consistent with existing debug endpoints marked for removal after debuggingdebugDryRun, this endpoint does NOT callevents.markProcessed()— this is intentional since it replays historical (already-processed) events rather than draining pending onesSAVEPOINT→ try/finally →ROLLBACK TO→RELEASE) is identical to the existingdebugDryRunmethod