LoRA support for LLM & VLM#4051
Open
dkalinowski wants to merge 22 commits intomainfrom
Open
Conversation
dkalinowski
commented
Apr 13, 2026
dkalinowski
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
Adds initial LoRA adapter support to OVMS GenAI-based LLM/VLM pipelines by extending MediaPipe node options and wiring adapter configuration into GenAI plugin configuration during servable initialization.
Changes:
- Introduces
LoraAdapteroptions inLLMCalculatorOptionsand applies adapters during servable initialization. - Extends
GenAiServablePropertiesto carry LoRA adapter configuration state. - Updates GenAI dependency pin and wires in a new (currently missing) unit test target.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
versions.mk |
Updates GenAI source pin and changes GenAI org (now pointing to a fork). |
src/llm/llm_calculator.proto |
Adds LoraAdapter message and repeated lora_adapter option. |
src/llm/servable.hpp |
Adds ov::genai::AdapterConfig to servable properties. |
src/llm/servable_initializer.hpp |
Declares initializeLoraAdapters(...) helper. |
src/llm/servable_initializer.cpp |
Implements LoRA adapter loading and injects adapters into pluginConfig. |
src/llm/language_model/legacy/servable_initializer.cpp |
Calls initializeLoraAdapters(...) during LM legacy init. |
src/llm/language_model/continuous_batching/servable_initializer.cpp |
Calls initializeLoraAdapters(...) during LM CB init. |
src/llm/visual_language_model/legacy/servable_initializer.cpp |
Calls initializeLoraAdapters(...) during VLM legacy init. |
src/BUILD |
Adds test/llm/lora_adapter_test.cpp to cc_test sources. |
atobiszei
reviewed
Apr 15, 2026
| optional LLMCalculatorOptions ext = 113473750; | ||
| } | ||
|
|
||
| message LoraAdapter { |
Collaborator
There was a problem hiding this comment.
https://github.com/openvinotoolkit/model_server/pull/4084/changes#diff-47030398a28f7f3e040fd364573901cacfb90de81ab6b6e3ad56416f782af8f2R54
Is there possibility/need o use the same model with/without lora?
Collaborator
Author
There was a problem hiding this comment.
We dont need alias in LLM/VLM. It's not possible to interact with it via request. What do you think?
atobiszei
reviewed
Apr 23, 2026
| "test/multipart_calculator_test.cpp", | ||
| "test/llm/assisted_decoding_test.cpp", | ||
| "test/llm/llmnode_test.cpp", | ||
| "test/llm/lora_adapter_test.cpp", |
atobiszei
reviewed
Apr 23, 2026
atobiszei
reviewed
Apr 23, 2026
atobiszei
approved these changes
May 7, 2026
| } | ||
| } | ||
| // since it is only applied once at initialization, static mode is sufficient and more efficient. | ||
| adapterConfig.set_mode(ov::genai::AdapterConfig::MODE_STATIC); |
| #include <vector> | ||
|
|
||
| #include <gtest/gtest.h> | ||
| #include <openvino/genai/lora_adapter.hpp> |
Comment on lines
+328
to
+332
| SPDLOG_INFO("Processing LoRA adapter number {} with model path: {} alpha: {}", i, loraAdapterOption.model_path(), loraAdapterOption.alpha()); | ||
| if (loraAdapterOption.alpha() <= 0.0f || loraAdapterOption.alpha() > 1.0f) { | ||
| SPDLOG_ERROR("LoRA adapter alpha value {} is out of valid range (0.0, 1.0]", loraAdapterOption.alpha()); | ||
| return StatusCode::LLM_NODE_RESOURCE_STATE_INITIALIZATION_FAILED; | ||
| } |
Comment on lines
+333
to
+337
| auto fsLoraPath = std::filesystem::path(loraAdapterOption.model_path()); | ||
| std::string loraPath; | ||
| if (fsLoraPath.is_relative()) { | ||
| loraPath = (std::filesystem::path(graphPath) / fsLoraPath).string(); | ||
| } else { |
Comment on lines
+79
to
86
| status = initializeLoraAdapters(nodeOptions, graphPath, properties); | ||
| if (!status.ok()) { | ||
| return status; | ||
| } | ||
|
|
||
| status = JsonParser::parsePluginConfig(nodeOptions.plugin_config(), properties->pluginConfig); | ||
| if (!status.ok()) { | ||
| SPDLOG_ERROR("Error during llm node plugin_config option parsing to JSON: {}", nodeOptions.plugin_config()); |
Comment on lines
+201
to
208
| status = initializeLoraAdapters(nodeOptions, graphPath, properties); | ||
| if (!status.ok()) { | ||
| return status; | ||
| } | ||
|
|
||
| status = JsonParser::parsePluginConfig(nodeOptions.plugin_config(), properties->pluginConfig); | ||
| if (!status.ok()) { | ||
| SPDLOG_ERROR("Error during llm node plugin_config option parsing to JSON: {}", nodeOptions.plugin_config()); |
Comment on lines
+78
to
85
| status = initializeLoraAdapters(nodeOptions, graphPath, properties); | ||
| if (!status.ok()) { | ||
| return status; | ||
| } | ||
|
|
||
| status = JsonParser::parsePluginConfig(nodeOptions.plugin_config(), properties->pluginConfig); | ||
| if (!status.ok()) { | ||
| SPDLOG_ERROR("Error during llm node plugin_config option parsing to JSON: {}", nodeOptions.plugin_config()); |
| a2->set_model_path(loraFilePath); | ||
| a2->set_alpha(0.7f); | ||
| ASSERT_EQ(initializeLoraAdapters(nodeOptions, "", properties), StatusCode::OK); | ||
| EXPECT_EQ(properties->pluginConfig.count("adapters"), 1); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🛠 Summary
CVS-182580