remove GeneralLlm initialization from method definition to suppress cost warning#244
Open
remove GeneralLlm initialization from method definition to suppress cost warning#244
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adjusts QuestionDecomposer.decompose_into_questions_fast to avoid constructing a GeneralLlm instance at function-definition time (module import), deferring initialization until the method is called—reducing import-time side effects and suppressing the cost estimation warning described in the PR.
Changes:
- Replace the default argument value
GeneralLlm.search_context_model(...)withNone. - Lazily create the default
GeneralLlm.search_context_model("openrouter/perplexity/sonar")inside the method whenmodelis not provided.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
very quick PR to remove initialized class in method definition - pushing it off to method call
This suppresses cost estimation warning on import.
I made a quick pass at reducing total module load time when importing anything. I could get it down from 4.5 to 2.7 seconds by turning top level imports
forecasting_tools/__init__.pyinto lazy imports. This avoided initializing some deprecated and likely-not-always-used files with heavy dependencies like scipy etc. This loses a bit of the click-through IDE support and I don't know the prioritization of keeping that so I left it well enough alone.The 2.7 seconds left was pretty much entirely the
litellmimporting which is more annoying to circumnavigate because it's required for ?all? bots so I expect it to pretty much always want to be imported when you do anything in this codebase. The solution here could be to only import all the litellm packages the first time a bot gets initialized. This backloads the load time to execution time.I do think the ultimate solution will be to move the locations of the submodules outside of the forecasting_tools folder so it doesn't run the
forecasting_tools/__init__.pyfile unless you specifically do something likefrom forecasting_tools import <blah>. (when you do something likefrom forecasting_tools.forecast_bots.template_bot import TemplateBotit will run (if they exist) theforecasting_tools/__init__.pyandforecasting_tools/forecast_bots/__init__.pywhich means you currently import EVERYTHING when you import anything).