fix(langchain): populate gen_ai.response.model for Responses API (useResponsesApi)#151
Open
JacksonWeber wants to merge 3 commits into
Open
fix(langchain): populate gen_ai.response.model for Responses API (useResponsesApi)#151JacksonWeber wants to merge 3 commits into
JacksonWeber wants to merge 3 commits into
Conversation
Read response_metadata.model (canonical OpenAI Responses API field) before falling back to response_metadata.model_name (LangChain's backwards-compat alias) on both v1 and v0 message paths. This makes gen_ai.response.model populate reliably when useResponsesApi: true, and keeps working if upstream LangChain ever drops the model_name alias. Fixes microsoft#128 Refs microsoft#150 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes LangChain Responses API (useResponsesApi: true) spans to reliably populate gen_ai.response.model by reading the canonical response_metadata.model field (with fallbacks), preventing silent attribute loss if upstream drops the model_name alias.
Changes:
- Update
getResponseModelto prefermessage.response_metadata.model(v1) /message.kwargs.response_metadata.model(v0), then fall back tomodel_name, thenllmOutput. - Add unit tests covering RAPI v1/v0 response-model extraction and AzureChatOpenAI workaround interaction.
- Add an end-to-end tracer test exercising a realistic RAPI
Runshape and asserting key GenAI span attributes.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
src/genai/instrumentations/langchain/utils.ts |
Extends response-model extraction to support canonical RAPI response_metadata.model with ordered fallbacks. |
test/internal/unit/genai/langchain/utils.test.ts |
Adds unit coverage for RAPI response-model extraction paths (v1/v0) and Azure workaround. |
test/internal/unit/genai/langchain/tracer.test.ts |
Adds an end-to-end shape test for RAPI runs through LangChainTracer and _endTrace. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Pin distinct sentinel values for response_metadata.model_name (alias) and message.id so the assertions actually prove the canonical fields (response_metadata.model, response_metadata.id) win over the fallbacks, rather than passing tautologically when all sources carry the same value. - Switch the RAPI end-to-end test in tracer.test.ts to use ATTR_GEN_AI_* semconv constants instead of raw attribute string literals. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes
gen_ai.response.modelbeing missing on LangChain runs whenuseResponsesApi: true(OpenAI Responses API path).Root cause
getResponseModelonly readresponse_metadata.model_name. LangChain'slangchain-openai/converters/responses.tssets bothresponse_metadata.model(canonical OpenAI Responses-API field) andresponse_metadata.model_name(backwards-compat alias for Chat Completions). Relying solely on the alias is fragile — if upstream ever drops it, our attribute silently disappears.Fix
Update
getResponseModelto preferresponse_metadata.modeland fall back toresponse_metadata.model_nameon both v1 (message.response_metadata) and v0 (message.kwargs.response_metadata) paths. Final lookup order:message.response_metadata.model(v1, canonical)message.response_metadata.model_name(v1, alias)message.kwargs.response_metadata.model(v0, canonical)message.kwargs.response_metadata.model_name(v0, alias)run.outputs.llmOutput.model_namerun.outputs.llmOutput.modelTests
utils.test.tscovering RAPI v1 (model only), RAPI v1 (both fields → canonical wins), RAPI v0, and AzureChatOpenAI + RAPI workaround interaction.tracer.test.tsdriving a realistic RAPIRunthroughonRunCreate+_endTraceand asserting operation name, request/response model, response id, and token attrs.All 84 langchain unit tests pass. Lint + build clean.
Out of scope
isAzureChatOpenAIRunrequest-model workaround — tied to upstreamlangchain-ai/langchainjs#10874; will be retired oncelangchainjs#10886ships.gen_ai.response.model(per @PengF on [Bug]: LangChain instrumentation mis-attributes response model into gen_ai.request.model; gen_ai.response.model never set #128). This fix is still valuable for OTLP / Foundry / console exporters and for raw-span validation.Related