HBASE-30161 Add paginated, single-RPC RegionLocator.getRegionLocationsPage(startKey, limit) API for bulk meta-cache warmup (branch-2.6)#8265
Open
sanjeet006py wants to merge 7 commits into
Conversation
added 7 commits
May 22, 2026 01:00
…s(startKey, limit) API for bulk meta-cache warmup
… and add default-throws Review comments by haridsv on apache#8236: - Adding an abstract method to RegionLocator on a stable branch breaks external implementers. Convert the new method to a default that throws UnsupportedOperationException, with javadoc instructing callers to fall back to getAllRegionLocations(). - Reusing the getRegionLocations name overloads a method with completely different semantic (row -> all replicas of containing region). Rename to getRegionLocationsPage(byte[] startKey, int limit) to make the pagination/range intent explicit. - SnapshotRegionLocator no longer needs an override; it inherits the default-throws and callers fall back to getAllRegionLocations(). Tests renamed to call getRegionLocationsPage; semantics and assertions unchanged.
Address the canvas review point that the paged meta scan was making ceil(limit / hbase.meta.scanner.caching) ScannerNext RPCs whenever the limit exceeded the configured caching, contradicting the "completes in a single RPC" javadoc on MetaTableAccessor.scanMetaForTableRegions and the "at most one RPC per invocation" claim on RegionLocator.getRegionLocationsPage. Plumb a private isPagedScan flag through MetaTableAccessor.scanMeta and getMetaScan. When true, set setCaching to the caller-supplied rowUpperLimit so the slice returns in a single ScannerNext RPC regardless of the configured caching default. All other existing scanMeta callers (scanByRegionEncodedName, getClosestRegionInfo, scanMetaForTableRegions without rowLimit) keep their existing behavior by passing isPagedScan = false.
…scans Mirrors the master-PR test (TestRegionLocatorPagedScanRpcCount on apache#8237). On the sync path we proxy: Connection.getTable(META_TABLE_NAME) -> Table.getScanner(Scan) -> ResultScanner.close() so we can flip Scan.setScanMetricsEnabled(true) on the way in and read ScanMetrics.countOfRPCcalls before the underlying scanner is closed. The RPC count is the natural sync analogue of the AdvancedScanResultConsumer.onNext invocation count we used on master. Cluster runs with hbase.meta.scanner.caching = 2 against a table of 5 user regions, asserting: - limit <= caching: 1 ScannerNext RPC (baseline) - limit > caching: 1 ScannerNext RPC (regression check; without the isPagedScan fix this would be ceil(5/2) = 3) - unbounded scan: ceil(NUM_REGIONS / caching) = 3 ScannerNext RPCs (proves the isPagedScan flag does not leak into the non-paged callers) Verified the test catches the regression: reverting the isPagedScan branch in MetaTableAccessor.getMetaScan makes testSingleRpcWhenLimitExceedsCaching fail with countOfRPCcalls = 3.
Drop the mini-cluster integration test that proxies the meta Table to count ScannerNext RPCs. Replace with a SmallTests UT that mocks Connection / Table / ResultScanner and uses ArgumentCaptor<Scan> to assert the property directly: scan.getCaching() == rowLimit on the paged path, and scan.getCaching() == configured caching with scan.getLimit() == Integer.MAX_VALUE on the unbounded path.
hbase-client was migrated to JUnit 5 in HBASE-30153 / branch-2; bring this test in line: switch org.junit.* imports to org.junit.jupiter.api.*, replace @category({...}) with @tag(...TAG), drop the now-unused HBaseClassTestRule @ClassRule, and replace @before with @beforeeach.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
JIRA: HBASE-30161
Backport of #8236 to branch-2.6. Cherry-picked the 7 non-merge commits from the source PR; all applied cleanly without modification (including the JUnit 5 migration of
TestMetaTableAccessorPagedScanCaching, since branch-2.6'shbase-clientis already on JUnit 5).Verified locally with
mvn clean install -DskipTests -Dhadoop.profile=3.0— full reactor BUILD SUCCESS.