Skip to content

feat: add /v1/models fallback for LiteLLM when /v1/model/info is inaccessible#11899

Draft
roomote-v0[bot] wants to merge 1 commit intomainfrom
feature/litellm-v1-models-fallback
Draft

feat: add /v1/models fallback for LiteLLM when /v1/model/info is inaccessible#11899
roomote-v0[bot] wants to merge 1 commit intomainfrom
feature/litellm-v1-models-fallback

Conversation

@roomote-v0
Copy link
Contributor

@roomote-v0 roomote-v0 bot commented Mar 9, 2026

Related GitHub Issue

Closes: #11898

Description

This PR attempts to address Issue #11898. Feedback and guidance are welcome.

When /v1/model/info is inaccessible (e.g. 403 Forbidden due to API gateways or internal policies), the getLiteLLMModels function now automatically falls back to /v1/models (the OpenAI-compatible endpoint) to retrieve the model list.

Key implementation details:

  • Extracted URL building and header construction into reusable helpers (buildHeaders, buildUrl)
  • Extracted response parsing into two exported functions: parseModelInfoResponse (for /v1/model/info) and parseModelsListResponse (for /v1/models)
  • The main getLiteLLMModels function now tries /v1/model/info first. If it fails for any reason (HTTP error, network error, unexpected format), it catches the error, logs it, and retries with /v1/models
  • The /v1/models fallback uses sensible defaults (8192 max tokens, 200k context window) since that endpoint provides less metadata
  • If both endpoints fail, a descriptive error is thrown mentioning both endpoints failed
  • Fully backward compatible: users whose /v1/model/info works will see no behavior change

Test Procedure

  • All 30 existing and new tests pass via cd src && npx vitest run api/providers/fetchers/__tests__/litellm.spec.ts
  • New test cases cover:
    • Fallback to /v1/models when /v1/model/info returns 403
    • Fallback to /v1/models when /v1/model/info returns 500
    • Fallback to /v1/models when /v1/model/info returns unexpected format
    • Both endpoints failing (HTTP errors, network errors, generic errors)
    • URL structure preserved in fallback requests
    • Invalid model IDs skipped in fallback response
    • Unit tests for parseModelInfoResponse and parseModelsListResponse parsers

Pre-Submission Checklist

  • Issue Linked: This PR is linked to an approved GitHub Issue (see "Related GitHub Issue" above).
  • Scope: My changes are focused on the linked issue (one major feature/fix per PR).
  • Self-Review: I have performed a thorough self-review of my code.
  • Testing: New and/or updated tests have been added to cover my changes.
  • Documentation Impact: No documentation updates are required for this change.
  • Contribution Guidelines: I have read and agree to the Contributor Guidelines.

Documentation Updates

  • No documentation updates are required.

Additional Notes

This change helps enterprise/managed LiteLLM users where /v1/model/info may be blocked by API gateways, VPNs, or internal policies, while /v1/models remains accessible.

Interactively review PR in Roo Code Cloud

… LiteLLM

When /v1/model/info returns a non-200 status (e.g. 403 Forbidden), the
function now automatically falls back to /v1/models to retrieve the model
list. This helps enterprise/managed LiteLLM users where /v1/model/info
may be blocked by API gateways or internal policies.

Closes #11898
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] LiteLLM Integration Fails When /v1/model/info Is Inaccessible

1 participant