Add MCP endpoint for audit and failed messages#5391
Add MCP endpoint for audit and failed messages#5391WilliamBZA wants to merge 24 commits intomasterfrom
Conversation
| host.UseTestRemoteIp(); | ||
| host.UseServiceControlAuthentication(settings.OpenIdConnectSettings.Enabled); | ||
| host.UseServiceControl(settings.ForwardedHeadersSettings, settings.HttpsSettings); | ||
| host.UseServiceControl(settings.ForwardedHeadersSettings, settings.HttpsSettings, settings.EnableMcpServer); |
There was a problem hiding this comment.
Are we better off passing the settings object in at this point?
There was a problem hiding this comment.
Hmm... Maybe yeah but I don't know what a reasonable threshold would be to make that flip but I agree it starts to smell
src/ServiceControl.Audit.AcceptanceTests/Mcp/When_mcp_server_is_enabled.cs
Outdated
Show resolved
Hide resolved
src/ServiceControl.Audit.UnitTests/Mcp/AuditMessageMcpToolsTests.cs
Outdated
Show resolved
Hide resolved
| Assert.That(store.LastAuditCountsEndpointName, Is.EqualTo("Sales")); | ||
| } | ||
|
|
||
| class StubAuditDataStore : IAuditDataStore |
There was a problem hiding this comment.
Worth moving this to a more general stub / fake or util?
There was a problem hiding this comment.
It's not used anywhere else as far as I can tell, so I wasn't sure if it was worth it. I have no problem with moving it out though. Does it make it more readable?
src/ServiceControl.Audit/Infrastructure/WebApi/HostApplicationBuilderExtensions.cs
Show resolved
Hide resolved
| [McpServerToolType] | ||
| public class AuditMessageTools(IAuditDataStore store) | ||
| { | ||
| [McpServerTool, Description("Get a list of successfully processed audit messages. Supports paging and sorting. Returns message metadata including endpoints, timing information, and message type.")] |
There was a problem hiding this comment.
I think we need to rework these
What is there now is mostly a parameter reference for a human developer, not a great tool contract for an LLM agent.
For MCP tools, the description should help the model answer three things fast:
- When should I use this tool?
- What kind of question is this tool good at?
- How should I call it sensibly without overthinking all parameters?
Right now the descriptions are too low-level and too focused on pagination/sorting mechanics.
There was a problem hiding this comment.
Example
[McpServerTool, Description(
"Use this tool to browse successfully processed audit messages when the user wants an overview rather than a text search. " +
"Good for questions like: 'show recent audit messages', 'what messages were processed today?', 'list messages from endpoint X', or 'show slow messages'. " +
"Returns message metadata such as message type, endpoints, sent time, processed time, and timing metrics. " +
"For broad requests, use the default paging and sorting. " +
"Prefer this tool over SearchAuditMessages when the user does not provide a specific keyword or phrase. " +
"If the user is looking for a specific term, id, or text fragment, use SearchAuditMessages instead."
)]
There was a problem hiding this comment.
I've taken a stab at updating all the descriptions, let me know if that's any better
| }, McpJsonOptions.Default); | ||
| } | ||
|
|
||
| [McpServerTool, Description("Search audit messages by a keyword or phrase. Searches across message content and metadata.")] |
There was a problem hiding this comment.
Similar here
[McpServerTool, Description(
"Use this tool when the user is looking for audit messages containing a specific keyword, phrase, message type, business identifier, endpoint name, or other known text. " +
"Good for questions like: 'find messages about order 123', 'search for timeout', 'show messages containing CustomerUpdated', or 'find audit messages mentioning endpoint X'. " +
"Searches across both message content and metadata. " +
"Prefer this tool over GetAuditMessages when the user provides a concrete search term. " +
"For broad exploration without a search term, use GetAuditMessages instead."
)]
| using ModelContextProtocol.Server; | ||
| using Persistence; | ||
|
|
||
| [McpServerToolType] |
There was a problem hiding this comment.
From Mnemonic I also learned that giving some workflow guidance really helps. Like for example on the class level describe what should be done with useful hints like
[McpServerToolType, Description(
"Tools for exploring audit messages.\n\n" +
"Agent guidance:\n" +
"1. For broad requests like 'show recent messages', start with GetAuditMessages using defaults.\n" +
"2. For requests containing a concrete text term, identifier, or phrase, use SearchAuditMessages.\n" +
"3. Keep page=1 unless the user asks for more results.\n" +
"4. Keep perPage modest, such as 20 to 50, unless the user asks for a larger batch.\n" +
"5. Use time filters when the user mentions a date or time window like 'today' or 'last hour'.\n" +
"6. Only change sorting when the user explicitly asks for it."
)]
| [McpServerToolType] | ||
| public class FailedMessageTools(IErrorMessageDataStore store) | ||
| { | ||
| [McpServerTool, Description("Get a list of failed messages. Supports filtering by status (unresolved, resolved, archived, retryissued), modified date, and queue address. Returns paged results.")] |
There was a problem hiding this comment.
Similar comments as before
|
I see you can spec agent behavior. It would be interesting if this can included something like:
@WilliamBZA Is something like this possible? |
ramonsmits
changed the title
|
@ramonsmits MCPs can even inject prompts but that comes with a cost because it is applied to every session. But having some workflow guidance usually helps |
|
@ramonsmits You probably could, but it would depend on the agent being able to do time-based triggers. It would also have to know wait for the successful/failed audit, so maybe... But I'm not sure we want to include that in the V1 part of this PR, or if that's more of a next step |
3 participants
Adds support for MCP servers within ServiceControl and ServiceControl.Audit