fix: avoid IndexOutOfBoundsException when stream choice index is non-contiguous#6056
Open
suryateja-g13 wants to merge 1 commit into
Open
Conversation
When processing streaming responses, each ChatCompletionChunk.Choice carries an index field that represents its API-level choice index. The previous code used this value directly as a position in the choices list (chunk.choices().get((int) choice.index())), which throws IndexOutOfBoundsException when the choice index is non-zero but only one choice is present in the chunk. This happens with OpenAI-compatible gateways that emit chunks where a single choice carries index=1 (e.g. when include_usage is enabled). Fix: build a Map<Long, ChatCompletionChunk.Choice> keyed by the API index before the stream map, then look up by index instead of using list position. Closes spring-projects#6023 Signed-off-by: Gorre Surya <suryateja.g13@gmail.com>
suryateja-g13
force-pushed
the
gh-6023-stream-choice-index-oob
branch
from
904f8af to
181a53f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #6023
chunk.choices().get((int) choice.index())with aMap<Long, ChatCompletionChunk.Choice>lookup keyed by API indexchunk.choices()before the stream mapping loop ininternalStream()index=1Background
OpenAiChatModel.internalStream()usedchoice.index()as a list position to look up the raw chunk choice. This works when choices are zero-based and contiguous, but some OpenAI-compatible gateways (especially whenstream_options.include_usage: trueis set) emit chunks where a single choice carries a non-zero index. In that case,choices.get(1)on a one-element list throwsIndexOutOfBoundsException.Test plan
OpenAiChatModelTests#streamingDoesNotThrowWhenChoiceIndexIsNonZero— new test with mocked async client that emits a single chunk withchoice.index()=1OpenAiChatModelTestspass