Merged
Conversation
Change omitted-sort non-scoring search requests to use offset:desc instead of primary timestamp sorting. Text-scoring queries keep _score, timestamp, offset ordering, and docs now describe the default split. Perf repro measurements (SEARCH_PERF_REPRO=1 bun test test/search_perf_repro.test.ts): before default-sort/broad=2233.69ms parseCalls=32817; after=85.20ms parseCalls=119. Offset-desc repro stayed ~2338.68ms before / 2291.94ms after. Small no-L0 repro stayed ~2290.26ms before / 2319.66ms after. Verification before commit: bun run typecheck; bun run check:result-policy; bun test. Also fixed the aggregate uncompanioned-prefix test helper race so the full suite passes consistently.
Use the DSB3 segment footer for offset-desc search scans so the reader walks blocks newest-to-oldest and stops once the requested page is full. The old no-footer fallback remains for legacy/corrupt segment shapes. Perf repro measurements (SEARCH_PERF_REPRO=1 bun test test/search_perf_repro.test.ts): before default/offset-small=85.20ms/2291.94ms/2319.66ms; after=25.13ms/64.41ms/64.16ms. Offset-desc newest-segment indexed time dropped from ~2290ms to ~63ms. Verification before commit: bun run typecheck; bun run check:result-policy; bun test.
Add coverage fields and response headers for candidate doc IDs, decoded records, JSON parse time, segment payload bytes fetched, sort time, and peak hits held. The perf repro now prints these counters so broad scans can be distinguished from index-only or block-limited paths. Perf repro measurements (SEARCH_PERF_REPRO=1 bun test test/search_perf_repro.test.ts): before metrics latency=25.13ms/64.41ms/64.16ms; after=22.98ms/51.53ms/68.11ms. Benefit is diagnostic: after metrics reported default candidateDocIds=2048 decodedRecords=113 segmentBytesFetched=5801497 peakHitsHeld=100; offset-desc candidateDocIds=32768 decodedRecords=1 segmentBytesFetched=195859446 peakHitsHeld=1; small-no-L0 candidateDocIds=32768 decodedRecords=106 segmentBytesFetched=195891695 peakHitsHeld=100. Verification before commit: bun run typecheck; bun run check:result-policy; bun test.
Measurements: - Existing repros before (after metrics): default=25.13ms, offset-desc=64.41ms, small-no-L0=64.16ms. - Existing repros after: default=21.25ms, offset-desc=61.15ms, small-no-L0=65.70ms. - Added WAL-tail rare exact repro: cold cache=866.18ms, warm cache=0.65ms, warm candidate_doc_ids=1, warm scanned_tail_docs=1. Verification: - bun run typecheck - bun run check:result-policy - bun test
Measurements: - Existing repros before: default=21.25ms, offset-desc=61.15ms, small-no-L0=65.70ms, WAL warm=0.65ms. - Existing repros after: default=23.97ms, offset-desc=43.33ms, small-no-L0=62.55ms, WAL warm=0.49ms. - Added exact-only sealed repro: elapsed=1183.96ms, candidate_doc_ids=1, parseCalls=5, families=[exact]. This demonstrates doc-level exact postings avoid parsing every source candidate; block decode remains for later index-only work. Verification: - bun run typecheck - bun run check:result-policy - bun test
Measurements: - Existing repros before: default=23.97ms, offset-desc=43.33ms, small-no-L0=62.55ms, exact-only sealed=1183.96ms. - After: default=22.39ms, offset-desc=49.63ms, small-no-L0=71.71ms, exact-only sealed=27.48ms. - Exact-only decoded_records dropped from 32768 to 15 while candidate_doc_ids stayed 1; this is the expected top-k/late-block-decode improvement for indexed offset-desc queries. Verification: - bun run typecheck - bun run check:result-policy - bun test
Measurements: - Before: default=22.39ms / 5.8MB fetched, offset-desc=49.63ms / 195.9MB fetched, small-no-L0=71.71ms / 195.9MB fetched, exact-only=27.48ms / 99.7MB fetched. - After: default=27.48ms / 325489B fetched, offset-desc=17.61ms / 193625B fetched, small-no-L0=35.25ms / 820836B fetched, exact-only=5.60ms / 133119B fetched. Verification: - bun run typecheck - bun run check:result-policy - bun test
Measurements: - Before: default=27.48ms, offset-desc=17.61ms, small-no-L0=35.25ms, WAL warm=0.86ms, exact-only=5.60ms. - After: default=27.06ms, offset-desc=17.65ms, small-no-L0=33.15ms, WAL warm=0.52ms, exact-only=5.37ms. - Existing local repros are flat because decoded sections are already tiny; this adds the bounded decoded-section cache advertised by config for repeated interactive searches. Verification: - bun run typecheck - bun run check:result-policy - bun test
Measurements: - New explicit timestamp-desc repro before: 2327.85ms, parseCalls=32881, indexedSegments=16, decodedRecords=32768, segmentBytesFetched=92935801, sortTimeMs=12, peakHitsHeld=32768. - New explicit timestamp-desc repro after: 172.87ms, parseCalls=2071, indexedSegments=1, decodedRecords=2048, segmentBytesFetched=5801497, sortTimeMs=0, peakHitsHeld=100. - Existing repros after: default=27.56ms, offset-desc=17.45ms, small-no-L0=34.67ms, WAL warm=0.56ms, exact-only=5.77ms. Verification: - SEARCH_PERF_REPRO=1 bun test test/search_perf_repro.test.ts - bun run typecheck - bun run check:result-policy - bun test
The deployed demo server had next_offset=100000 but segment_count=0, sealed_through=-1, uploaded_through=-1, pending_bytes=102255073. _search therefore reported possible_missing_wal_rows=100000 and scanned_tail_docs=0 for environment:"staging". The segmenter worker path was resolved from the bundled compute/entry.js module. ./segment/segmenter_worker.js pointed under compute/segment, while the bundle emits the worker at ../segment/segmenter_worker.js. Verification: - bun test test/compute/worker_module_url.test.ts test/compute/bundle_build.test.ts - bun run typecheck - bun run check:result-policy - bun test
Enforce the documented 500-key page limit for routing key listing and add a regression test. Also align docs with current runtime defaults, profile/schema behavior, coverage fields, package requirements, and portable links.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| payload: unknown, | ||
| headers?: HeadersInit, | ||
| ): Response { | ||
| return new Response(JSON.stringify(payload), { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
@prisma/streams-localcan load the worker from the published package layoutMemory/stress evidence
Verification
bun run verifybun run test:conformancebun run test:conformance:localbun run test:node-local-packagebun run test:bun-local-packagebun run test:bun-server-packagebun run build:npm-packagesbun pm pack --dry-runindist/npm/streams-localbun pm pack --dry-runindist/npm/streams-serverNote: the first local conformance attempt collided with the server-mode conformance run on
127.0.0.1:8787; rerunning local conformance serially passed.