Bmx_base_perpetual_trades performance#9442
Merged
Conversation
…prove transaction join - Add incremental_predicates config for merge partition pruning on destination table - Add block_number and block_date to transactions join for partition pruning - Filter transactions source directly (trx.block_time) instead of redundantly filtering events - Prevents full scan of base.transactions on incremental runs Co-authored-by: jeff-dude <jeff-dude@users.noreply.github.com>
|
Cursor Agent can help with this pull request. Just |
github-actions
Bot
added
WIP
github-actions
Bot
added
ready-for-review
jeff-dude
added
ready-for-merging
and removed
ready-for-review
tomfutago
approved these changes
Mar 10, 2026
tomfutago
deleted the
cursor/CUR2-1635-bmx-base-perpetual-trades-performance-cd33
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thank you for contributing to Spellbook 🪄
Please open the PR in draft and mark as ready when you want to request a review.
Description:
Optimizes the
bmx_base_perpetual_tradesmodel to significantly reduce its runtime in hourly jobs.The model was running longer than expected due to:
incremental_predicates: No partition pruning on the destination table during merge operations.base.transactions: Only joining ontx_hashwithoutblock_numberorblock_date, leading to full table scans.base.transactionssource, preventing its pruning.This PR addresses these issues by:
incremental_predicatesto enable destination table pruning.base.transactionswithblock_numberandblock_dateto enable partition pruning on the source table.base.transactionsbased ontrx.block_time.These changes ensure more efficient data reads and joins, drastically improving the model's performance.
quick links for more information:
Linear Issue: CUR2-1635