-
Notifications
You must be signed in to change notification settings - Fork 186
docs(rfd): prompt queueing RFD #484
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
SteffenDE
wants to merge
1
commit into
agentclientprotocol:main
Choose a base branch
from
tidewave-ai:sd-prompt-queueing
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+57
−0
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| --- | ||
| title: "Prompt Queueing" | ||
| --- | ||
|
|
||
| Author(s): [SteffenDE](https://github.com/SteffenDE) | ||
|
|
||
| ## Elevator pitch | ||
|
|
||
| > What are you proposing to change? | ||
|
|
||
| Add a new `promptQueueing` session capability, that tells client if the agent supports sending prompts while another prompt is already being processed for that session. | ||
| The Claude Code CLI supports writing a user message while the agent is working and injects that message at the next opportunity (i.e. after the current turn is finished). | ||
|
|
||
| Since ACP prompts can consist of many agent turns, having an officially supported way to adjust running prompts is very useful. | ||
|
|
||
| ## Status quo | ||
|
|
||
| > How do things work today and what problems does this cause? Why would we change things? | ||
|
|
||
| Today, the specification does not mention what should happen when sending a concurrent prompt request. | ||
| Clients cannot assume that this is supported, so running prompts need to be canceled when a client wants to allow users to send a new prompt while an existing one is running. | ||
|
|
||
| ## What we propose to do about it | ||
|
|
||
| > What are you proposing to improve the situation? | ||
|
|
||
| Add a capability that tells clients if parallel prompt requests are supported. | ||
|
|
||
| ## Shiny future | ||
|
|
||
| > How will things will play out once this feature exists? | ||
|
|
||
| If an agent supports prompt queuing, clients can send parallel prompt requests and be sure that the agent will handle them properly. | ||
|
|
||
| ## Implementation details and plan | ||
|
|
||
| > Tell me more about your implementation. What is your detailed implementation plan? | ||
|
|
||
| The idea is to use whatever mechanism the agent has to queue incoming prompts and if a new prompt arrives and is processed at the next opportunity, | ||
| the existing is finished early `end_turn` (or optionally a new stop reason). This allows the client to inject the new user message at the correct place | ||
| in the conversation, since session updates are not tied to a particular prompt request. | ||
|
|
||
| ## Frequently asked questions | ||
|
|
||
| > What questions have arisen over the course of authoring this document or during subsequent discussions? | ||
|
|
||
| None so far. | ||
|
|
||
| ### What alternative approaches did you consider, and why did you settle on this one? | ||
|
|
||
| I was thinking that maybe clients could first send a `session/cancel` with a new parameter `afterCurrentTurn: true` to indicate that the current turn should be finished. | ||
| Right now, the protocol tells clients to immediately mark tool calls as canceled when sending `session/cancel`, but for queued prompts, currently running tool calls | ||
| should be able to finish. | ||
|
|
||
| ## Revision history | ||
|
|
||
| <!-- If there have been major updates to this RFD, you can include the git revisions and a summary of the changes. --> | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A few thoughts on this that make it tricky:
As an FYI, I am planning to propose soon a completely notification-based turn lifecycle (bidirectional stream of events from both sides updating the session) that will hopefully make it easier to model this sort of thing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am happy to wait for a better proposal. This is more of a band-aid solution for sure. It does not allow editing a sent message in any way. Maybe queued is actually the wrong word, since as soon as the prompt is sent, it is assumed to be processed by the agent in some way.
There are two responses: