Skip to content

fix: correct AES-CTR counter byte mask from 0xf to 0xff in _parseBigEndian#271

Open
RobertoLuzanilla wants to merge 3 commits intogoogle:masterfrom
RobertoLuzanilla:fix/aes-ctr-counter-byte-mask
Open

fix: correct AES-CTR counter byte mask from 0xf to 0xff in _parseBigEndian#271
RobertoLuzanilla wants to merge 3 commits intogoogle:masterfrom
RobertoLuzanilla:fix/aes-ctr-counter-byte-mask

Conversation

@RobertoLuzanilla
Copy link
Copy Markdown

Fixes #270

Change

lib/src/impl_ffi/impl_ffi.aesctr.dart line 47:

- value = (value << 8) | BigInt.from(data[i] & 0xf);
+ value = (value << 8) | BigInt.from(data[i] & 0xff);

Why

_parseBigEndian reads the current AES-CTR counter block to compute how
many AES blocks remain before the counterLength-bit counter wraps.

The 4-bit mask (0xf) truncates the upper nibble of each byte, so any
byte where the upper nibble is non-zero is misread — e.g. 0xff is
parsed as 0x0f, 0x10 as 0x00. This causes the wrap-around point
to be computed incorrectly, and BoringSSL's internal 128-bit counter
carries into bytes that the W3C WebCrypto spec requires to remain as
the nonce — producing keystream reuse across messages.

Lines 45 and 176 in the same file correctly use 0xff, confirming this
is a typo introduced in commit 5881400 (2020-07-05).

impl_js is not affected as it delegates to window.crypto.

Verification

  • Regression test added: test/aes_ctr_counter_wrap_test.dart
  • Covers the 32-bit counter wrap boundary (nonce || 0xffffffff
    (nonce+1) || 0x00000000)
  • Verified locally: 00:01 +1: All tests passed!

@jonasfj jonasfj requested a review from HamdaanAliQuatil May 6, 2026 07:36
Copy link
Copy Markdown
Member

@jonasfj jonasfj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect it's worth us locally validating that the test case fails before this, but works when running in chrome.

But assuming that's true, I suggest we land this.

@RobertoLuzanilla
Copy link
Copy Markdown
Author

Thanks for the review.

I pushed a formatting-only follow-up commit for test/aes_ctr_counter_wrap_test.dart. The previous failing check was due to dart format --set-exit-if-changed detecting formatting changes in that test file.

The PR now only changes the AES-CTR mask fix and the regression test.

@RobertoLuzanilla RobertoLuzanilla requested a review from jonasfj May 6, 2026 08:27
@RobertoLuzanilla RobertoLuzanilla force-pushed the fix/aes-ctr-counter-byte-mask branch from 0abece6 to 10fc6bd Compare May 6, 2026 08:47
@RobertoLuzanilla
Copy link
Copy Markdown
Author

Apologies for the noise — the format check failed because my local dart was resolving to a broken Flutter SDK path on WSL. Fixed by installing dart standalone and running dart format properly. Should be clean now.

Copy link
Copy Markdown
Collaborator

@HamdaanAliQuatil HamdaanAliQuatil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Locally validated the regression behavior.

Using master plus only the new regression test:

  • dart test test/aes_ctr_counter_wrap_test.dart fails on native VM
  • dart test -p chrome test/aes_ctr_counter_wrap_test.dart passes

The native failure on master was:

Expected: not [66, 108, 118, 143, 170, 65, 11, 114, 171, 16, 57, 81, 37, 155, 161, 74]
Actual:            [66, 108, 118, 143, 170, 65, 11, 114, 171, 16, 57, 81, 37, 155, 161, 74]

On this PR branch:

  • the native VM test passes
  • the Chrome test still passes

reviewed the PR. Great catch, and thanks for the focused fix and regression test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: AES-CTR counter byte mask truncated to 4 bits (0xf vs 0xff) in impl_ffi

3 participants