Skip to content

feat: add agent graph tracker#89

Merged
andrewklatzke merged 5 commits intomainfrom
aklatzke/REL-12055/agent-graph-tracking
Mar 2, 2026
Merged

feat: add agent graph tracker#89
andrewklatzke merged 5 commits intomainfrom
aklatzke/REL-12055/agent-graph-tracking

Conversation

@andrewklatzke
Copy link
Contributor

@andrewklatzke andrewklatzke commented Feb 5, 2026

Requirements

  • I have added test coverage for new or changed functionality
  • I have followed the repository's pull request submission guidelines
  • I have validated my changes against all supported platform versions

Describe the solution you've provided

Implements the AIGraphTracker class to handle tracking events for agent graphs in the SDK.

Describe alternatives you've considered

This is the primary implementation of the tracker. It's broken into its own tracker rather than polluting the interface of the BaseTracker or AiConfigTracker since this is a separate entity that has its own interaction & metric patterns.

Additional context

This implements the following events to be tracked:

Edge-level metrics

$ld:ai:graph:redirect<{ sourceKey, redirectedTarget }>
$ld:ai:graph:handoff_success<{ sourceKey, targetKey }>
$ld:ai:graph:handoff_failure<{ sourceKey, targetKey }>

Node level metrics

$ld:ai:graph:node_invocation<{ graphKey, configKey }>
$ld:ai:graph:tool_call<{ graphKey, configKey, toolKey }>
<judge_metrics><{ graphKey, configKey }> (judge on a specific node)

Graph metrics

$ld:ai:graph:invocation_success
$ld:ai:graph:invocation_failure
$ld:ai:graph:latency (total latency of the entire graph invocation)
$ld:ai:graph:total_tokens (total token usage of the graph invocation)
$ld:ai:graph:path<{ graphKey, ...configKey }>
<judge_metrics><{ graphKey }> (judges on final output)

Note

Medium Risk
Adds new LaunchDarkly track() event emission for agent graph executions and wires it into LDAIClient.agent_graph, increasing telemetry volume and coupling to variation metadata; incorrect usage could affect metrics quality but not core evaluation behavior.

Overview
Adds a new AIGraphTracker to emit LaunchDarkly tracking events for agent graph executions (graph invocation success/failure, latency, total tokens, execution path, node invocations/tool calls, redirects/handoffs, and judge metric events).

LDAIClient.agent_graph() now constructs an AIGraphTracker from flag _ldMeta (variation key/version) and returns it on the AgentGraphDefinition (including disabled/invalid graph returns), which also gains a tracker field plus get_tracker() accessor. AIGraphTracker is exported from ldai.__init__ for SDK consumers.

Written by Cursor Bugbot for commit e6c1ff3. This will update automatically on new commits. Configure here.

@andrewklatzke andrewklatzke requested a review from a team as a code owner February 5, 2026 00:53
@@ -1,11 +1,11 @@
"""Graph implementation for managing AI agent graphs."""

from dataclasses import dataclass
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused import here, not sure why it wasn't picked up in previous lint

Co-authored-by: Jason Bailey <jbailey@launchdarkly.com>
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

self._context,
track_data,
eval_score.score,
)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicated judge response tracking logic within same class

Low Severity

track_judge_response and track_node_judge_response within AIGraphTracker contain nearly identical logic — the only difference is that track_node_judge_response adds "configKey" to the track data. Extracting the shared eval-iteration logic into a private helper (accepting an optional config_key) would reduce duplication and the risk of the two methods drifting out of sync during future changes.

Additional Locations (1)

Fix in Cursor Fix in Web

… github.com:launchdarkly/python-server-sdk-ai into aklatzke/REL-12055/agent-graph-tracking
@andrewklatzke andrewklatzke merged commit 9d371fa into main Mar 2, 2026
35 checks passed
@andrewklatzke andrewklatzke deleted the aklatzke/REL-12055/agent-graph-tracking branch March 2, 2026 22:06
@github-actions github-actions bot mentioned this pull request Mar 2, 2026
jsonbailey pushed a commit that referenced this pull request Mar 4, 2026
🤖 I have created a release *beep* *boop*
---


<details><summary>launchdarkly-server-sdk-ai: 0.15.0</summary>

##
[0.15.0](launchdarkly-server-sdk-ai-0.14.2...launchdarkly-server-sdk-ai-0.15.0)
(2026-03-02)


### Features

* add agent graph tracker
([#89](#89))
([9d371fa](9d371fa))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Low Risk**
> Low risk: this PR is a release/version bump and documentation updates
with no functional code changes beyond updating the exported package
version string.
> 
> **Overview**
> Bumps `packages/sdk/server-ai` from `0.14.2` to `0.15.0` across the
release manifest, `pyproject.toml`, and `ldai.__version__`.
> 
> Updates release documentation by adding the `0.15.0` entry to
`CHANGELOG.md` (noting the *agent graph tracker* feature) and updating
`PROVENANCE.md`’s verification example to reference `0.15.0`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
b148be8. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants