feat: add Source Map Scopes Proposal support by alexander-akait · Pull Request #210 · webpack/webpack-sources

alexander-akait · 2026-04-22T18:52:53Z

Summary

Port the Source Map Scopes Proposal plumbing from the stale sokra/sources-proposal branch onto the current main. The proposal branch is about a year old and shares no merge base with main, so this is a fresh re-application rather than a merge.

The goal of this PR is narrow and forward-compatible: let an input SourceMapSource carry originalScopes/generatedRanges, thread them through the streamChunks pipeline (with coordinate shifts where needed), and emit them back out in the map produced by getFromStreamChunks / streamAndGetSourceAndMap. Every existing streamChunks signature gains two optional trailing callbacks (onOriginalScope, onGeneratedRange) — nothing else changes for callers that don't care about scopes.

What's new

New helpers under lib/helpers/:

vlq.js — shared VLQ token reader (readTokens) and valueAsToken encoder, extracted so both mappings and scopes/ranges use one decoder
readOriginalScopes.js / readAllOriginalScopes.js — decode originalScopes per-source VLQ strings
readGeneratedRanges.js — decode the generatedRanges VLQ string (definitions, callsites, bindings, subranges)
createOriginalScopesSerializer.js / createGeneratedRangesSerializer.js — stateful serializers used by getFromStreamChunks / streamAndGetSourceAndMap

Source-class plumbing:

SourceMapSource reads originalScopes/generatedRanges from its source map and emits them via the new callbacks
ConcatSource forwards scopes verbatim and shifts generated-range line/column coordinates by the cumulative offset of each child
PrefixSource shifts generated-range columns by the prefix length on non-line-start positions
CachedSource passes the callbacks through both the cache-miss (streamAndGetSourceAndMap) and cache-hit (streamChunksOfSourceMap) paths
ReplaceSource forwards original scopes (their positions refer to original sources, not generated columns, so they're stable across replacements) but drops generated ranges (their columns would need remapping through each replacement — out of scope here)
RawSource, OriginalSource, streamChunksOfCombinedSourceMap accept the new callbacks as no-ops since they have no scope data to emit

Types:

Source.js gains OnOriginalScope, OnGeneratedRange, DefinitionReference, Callsite, Binding typedefs, and RawSourceMap picks up optional originalScopes/generatedRanges fields
types.d.ts regenerated via fix:special to surface the new optional parameters on every streamChunks

What's intentionally not in this PR

streamChunksOfCombinedSourceMap does not remap scopes/ranges through the combined coordinate transform — it silently drops them. Supporting that is a larger design question (and was marked as throw on the proposal branch).
ReplaceSource drops generated ranges for the same reason.
This PR only plumbs scope data that arrives via an input source map; there's no user-facing API yet to construct scopes/ranges from scratch. That can come later once the TC39 spec stabilizes.

Test plan

All 89,825 pre-existing tests pass unchanged
19 new tests in test/scopes.js cover:
- VLQ encode/decode round-trip across positive/negative values and ,/; controls
- originalScopes serializer ↔ reader round-trip (nested scope with named function + variables)
- generatedRanges serializer ↔ reader round-trip (range with HAS_DEFINITION flag)
- readAllOriginalScopes walks each per-source string; ignores non-array input
- SourceMapSource.map() preserves originalScopes/generatedRanges verbatim when no inner map
- SourceMapSource.streamChunks fires the scope/range callbacks with correct positions
- ConcatSource shifts generated-range line offsets for non-first children
npm run lint (eslint, tsc, tsc -p tsconfig.types.test.json, lint:special) clean
types.d.ts regenerated and committed

https://claude.ai/code/session_014ZzvnYVFk8TSyuPA643z3v

Generated by Claude Code

Port the Scopes Proposal plumbing from the `sokra/sources-proposal` branch on top of current `main`. Adds two optional `streamChunks` callbacks — `onOriginalScope` and `onGeneratedRange` — and threads them through every derived source so that `originalScopes`/`generatedRanges` on an input sourceMap round-trip out to `map()`/`sourceAndMap()`: - SourceMapSource reads scopes/ranges from its source map and emits them - ConcatSource forwards them with generated-line offsets per child - PrefixSource shifts generated-range column offsets for the prefix - CachedSource passes them through cache-miss and cache-hit paths - ReplaceSource forwards original scopes as-is (positions are stable) and drops generated ranges (their generated columns would need remapping) - RawSource / OriginalSource accept the callbacks as no-ops (no scopes) - streamChunksOfCombinedSourceMap accepts and drops them (combined maps don't remap scope/range coordinates today) - getFromStreamChunks / streamAndGetSourceAndMap collect the emitted scopes/ranges and emit them back onto the output source map New helpers: `vlq` (extracted token reader/encoder), `readOriginalScopes` / `readAllOriginalScopes` / `readGeneratedRanges`, `createOriginalScopesSerializer` / `createGeneratedRangesSerializer`. Typedefs for `OnOriginalScope`, `OnGeneratedRange`, `DefinitionReference`, `Callsite`, `Binding`, and the `originalScopes`/`generatedRanges` fields on `RawSourceMap` live in Source.js and are surfaced in the generated `types.d.ts`. Adds `test/scopes.js` covering VLQ round-trip, serializer↔reader round-trip, `readAllOriginalScopes`, SourceMapSource scope/range pass-through via `map()` and `streamChunks`, and ConcatSource generated range line-offset shifting. All 89,825 pre-existing tests continue to pass; 19 new tests added. https://claude.ai/code/session_014ZzvnYVFk8TSyuPA643z3v

linux-foundation-easycla · 2026-04-22T18:52:58Z

❌ - login: @claude / name: Claude . The commit (3bfb891, 626dfde, 6788cd2, 91217e1) is not authorized under a signed CLA. Please click here to be authorized. For further assistance with EasyCLA, please submit a support request ticket.

changeset-bot · 2026-04-22T18:52:59Z

🦋 Changeset detected

Latest commit: 3bfb891

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
webpack-sources	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

codecov · 2026-04-22T18:55:20Z

Codecov Report

❌ Patch coverage is 93.27146% with 29 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.88%. Comparing base (ccfbc65) to head (3bfb891).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/ConcatSource.js	75.55%	10 Missing and 1 partial ⚠️
lib/helpers/getFromStreamChunks.js	81.25%	9 Missing ⚠️
lib/helpers/scopes.js	97.52%	7 Missing ⚠️
lib/helpers/streamAndGetSourceAndMap.js	93.10%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #210      +/-   ##
==========================================
- Coverage   97.77%   96.88%   -0.89%     
==========================================
  Files          25       26       +1     
  Lines        1932     2346     +414     
  Branches      606      734     +128     
==========================================
+ Hits         1889     2273     +384     
- Misses         41       70      +29     
- Partials        2        3       +1

Flag	Coverage Δ
integration	`96.88% <93.27%> (-0.89%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

codspeed-hq · 2026-04-22T18:58:30Z

Merging this PR will degrade performance by 13.27%

⚠️

Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 3 improved benchmarks
❌ 1 regressed benchmark
✅ 157 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Benchmark	`BASE`	`HEAD`	Efficiency
⚡	`compat-source: source() (wrapping SourceLike)`	154.5 µs	130.4 µs	+18.5%
⚡	`compat-source: sourceAndMap()`	237.6 µs	184 µs	+29.13%
❌	`compat-source: map()`	167.5 µs	193.2 µs	-13.27%
⚡	`source-map-source: streamChunks({finalSource:true})`	7.1 ms	6.1 ms	+17.54%

_{Comparing claude/investigate-sources-proposal-nxQ3b (3bfb891) with main (e130653)}

@experimental

Based on review feedback, flatten the six scope helpers into a single `lib/helpers/scopes.js` module, optimize the VLQ encoder/decoder hot paths, expand the JSDoc, mark the API as experimental in TypeScript- visible typedefs (@experimental), and add a dedicated experimental section to the README. Simplifications - `lib/helpers/vlq.js`, `readOriginalScopes.js`, `readAllOriginalScopes.js`, `readGeneratedRanges.js`, `createOriginalScopesSerializer.js`, and `createGeneratedRangesSerializer.js` are gone - All their exports (plus HAS_NAME_FLAG / HAS_DEFINITION_FLAG / HAS_CALLSITE_FLAG) now live in `scopes.js` - The three internal consumers (streamChunksOfSourceMap, getFromStreamChunks, streamAndGetSourceAndMap) import from the consolidated module - Net diff: -384 lines across the port Performance - Decoders (`readOriginalScopes`, `readGeneratedRanges`) inline the VLQ state machine instead of going through a per-token `onToken` closure, saving one closure call per sextet on the hot path - Encoder (`valueAsToken`) has a single-sextet fast path for the common case of |value| < 16, skipping the continuation loop entirely - Generated-ranges serializer uses a plain `";"` literal for single-line gaps instead of `.repeat(1)` (a test pins this) Docs & experimental status - Source.js typedefs (RawSourceMap.originalScopes/generatedRanges, DefinitionReference, Callsite, Binding, OnOriginalScope, OnGeneratedRange) gain prominent "Experimental — Source Map Scopes Proposal" prefixes and links to the TC39 proposal - Dedicated "Source Map Scopes Proposal (experimental)" section in README.md explaining the fields, propagation rules across derived sources, the two optional streamChunks callbacks, and the low-level helpers — all with a clear warning that the wire format may change - Changeset updated to call out the experimental status - `test/scopes.js` drops the internal VLQ-alphabet unit tests (they were exercising private implementation) and gains three targeted round-trip tests for the fast/slow VLQ paths and the single-`;` optimization All 89,844 tests still pass; npm run lint / tsc / tsc types test / format-file-header / generate-types all clean. https://claude.ai/code/session_014ZzvnYVFk8TSyuPA643z3v

Add 24 new tests (34 total, up from 10) covering the previously under-exercised paths in lib/helpers/scopes.js and in the scope propagation through every derived Source. Encoder / decoder edges - Single-sextet fast path (|value| <= 15) - Multi-sextet continuation loop (|value| up to 2_000_000) - Same-line comma separator (",") path in the serializer - Multi-line ";" gap branch via `.repeat` - HAS_CALLSITE_FLAG round-trip including source-change reset - HAS_DEFINITION_FLAG source-change reset - Simple (no-subrange) bindings round-trip - Subrange line-delta wire format (pinned behavior) - INVALID char (space) skipped silently in both readers - Out-of-table char (charCode > 'z') skipped silently - Tail flush when input ends without a terminator - Non-string / null / number input is a no-op - HAS_NAME_FLAG=0 path skips the name field Source propagation - PrefixSource shifts non-line-start generated-range columns by the prefix length and leaves column-0 starts at column 0 - CachedSource preserves scope/range data on both the cache-miss and cache-hit paths - ReplaceSource keeps originalScopes but drops generatedRanges - SourceMapSource with an inner map silently drops scopes (no throw, no emitted events, no retained map fields) - RawSource and OriginalSource accept the callbacks as no-ops - ConcatSource remaps scope sourceIndex when a second child contributes a distinct source, and remaps scope variable name indices to the global name table - The `streamChunks` dispatcher fallback (for sources without their own streamChunks method) routes scope data through sourceAndMap + streamChunksOfSourceMap Coverage deltas - scopes.js: 72% -> 98.5% lines, 52% -> 88% branches - ConcatSource.js: 46% -> 50% lines - PrefixSource.js: 0% -> 60% lines - ReplaceSource.js: 0% -> 46% lines - streamAndGetSourceAndMap: 43% -> 85% lines - streamChunks.js: 55% -> 88% lines - streamChunksOfSourceMap: 24% -> 57% lines - streamChunksOfCombinedSourceMap: 3% -> 48% lines All 89,859 tests pass; lint + tsc + tsc-types-test + tooling lint:special clean. https://claude.ai/code/session_014ZzvnYVFk8TSyuPA643z3v

The helpers in lib/helpers/scopes.js are internal and not part of the stable package surface. Documenting them in the README implied they were user-facing; remove the section to avoid that implication. https://claude.ai/code/session_014ZzvnYVFk8TSyuPA643z3v

claude added 3 commits April 22, 2026 19:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: add Source Map Scopes Proposal support#210

feat: add Source Map Scopes Proposal support#210
alexander-akait wants to merge 4 commits into
mainfrom
claude/investigate-sources-proposal-nxQ3b

alexander-akait commented Apr 22, 2026

Uh oh!

linux-foundation-easycla Bot commented Apr 22, 2026 •

edited

Loading

Uh oh!

changeset-bot Bot commented Apr 22, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Apr 22, 2026 •

edited

Loading

Uh oh!

codspeed-hq Bot commented Apr 22, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

alexander-akait commented Apr 22, 2026

Summary

What's new

What's intentionally not in this PR

Test plan

Uh oh!

linux-foundation-easycla Bot commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

changeset-bot Bot commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

codecov Bot commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

codspeed-hq Bot commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will degrade performance by 13.27%

Performance Changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

linux-foundation-easycla Bot commented Apr 22, 2026 •

edited

Loading

changeset-bot Bot commented Apr 22, 2026 •

edited

Loading

codecov Bot commented Apr 22, 2026 •

edited

Loading

codspeed-hq Bot commented Apr 22, 2026 •

edited

Loading