Skip to content

remove GeneralLlm initialization from method definition to suppress cost warning#244

Open
lsabor wants to merge 1 commit intomainfrom
quickfix/load-delay
Open

remove GeneralLlm initialization from method definition to suppress cost warning#244
lsabor wants to merge 1 commit intomainfrom
quickfix/load-delay

Conversation

@lsabor
Copy link
Copy Markdown
Contributor

@lsabor lsabor commented Apr 25, 2026

very quick PR to remove initialized class in method definition - pushing it off to method call
This suppresses cost estimation warning on import.

I made a quick pass at reducing total module load time when importing anything. I could get it down from 4.5 to 2.7 seconds by turning top level imports forecasting_tools/__init__.py into lazy imports. This avoided initializing some deprecated and likely-not-always-used files with heavy dependencies like scipy etc. This loses a bit of the click-through IDE support and I don't know the prioritization of keeping that so I left it well enough alone.
The 2.7 seconds left was pretty much entirely the litellm importing which is more annoying to circumnavigate because it's required for ?all? bots so I expect it to pretty much always want to be imported when you do anything in this codebase. The solution here could be to only import all the litellm packages the first time a bot gets initialized. This backloads the load time to execution time.

I do think the ultimate solution will be to move the locations of the submodules outside of the forecasting_tools folder so it doesn't run the forecasting_tools/__init__.py file unless you specifically do something like from forecasting_tools import <blah>. (when you do something like from forecasting_tools.forecast_bots.template_bot import TemplateBot it will run (if they exist) the forecasting_tools/__init__.py and forecasting_tools/forecast_bots/__init__.py which means you currently import EVERYTHING when you import anything).

Copilot AI review requested due to automatic review settings April 25, 2026 17:31
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adjusts QuestionDecomposer.decompose_into_questions_fast to avoid constructing a GeneralLlm instance at function-definition time (module import), deferring initialization until the method is called—reducing import-time side effects and suppressing the cost estimation warning described in the PR.

Changes:

  • Replace the default argument value GeneralLlm.search_context_model(...) with None.
  • Lazily create the default GeneralLlm.search_context_model("openrouter/perplexity/sonar") inside the method when model is not provided.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants