stringTools: ES2026 proposal-arraybuffer-base64 support#18063
stringTools: ES2026 proposal-arraybuffer-base64 support#18063kzhsw wants to merge 14 commits intoBabylonJS:masterfrom
Conversation
This commit adds support of native base64 encoding/decoding api support, which is benchmarked to be faster then the js impl. It checks native support of the native api excluding polyfills by core-js and es-shims on module load, and fallback to the original js impl for backward compatibility. See this forum post for discussion and benchmark result: <https://forum.babylonjs.com/t/es2026-proposal-arraybuffer-base64-support/62715>
There was a problem hiding this comment.
Pull request overview
Adds support in stringTools for the TC39 Uint8Array base64 proposal APIs (toBase64 / fromBase64) to use native implementations when available (and fall back to the existing JS implementation otherwise), aiming for better performance.
Changes:
- Introduces runtime detection (
HasNativeBase64) and switches encode/decode implementations based on detected support. - Adds native encode/decode paths using
Uint8Array.prototype.toBase64andUint8Array.fromBase64. - Refactors the existing JS implementations into explicit fallback functions.
You can also share your feedback on Copilot code review. Take the survey.
| declare global { | ||
| interface Uint8Array<TArrayBuffer extends ArrayBufferLike = ArrayBufferLike> { | ||
| /** | ||
| * Converts the `Uint8Array` to a base64-encoded string. | ||
| * @param options If provided, sets the alphabet and padding behavior used. | ||
| * @returns A base64-encoded string. | ||
| */ | ||
| toBase64(options?: { alphabet?: "base64" | "base64url" | undefined; omitPadding?: boolean | undefined }): string; | ||
| } | ||
|
|
||
| interface Uint8ArrayConstructor { | ||
| /** | ||
| * Creates a new `Uint8Array` from a base64-encoded string. | ||
| * @param string The base64-encoded string. | ||
| * @param options If provided, specifies the alphabet and handling of the last chunk. | ||
| * @returns A new `Uint8Array` instance. | ||
| * @throws {SyntaxError} If the input string contains characters outside the specified alphabet, or if the last | ||
| * chunk is inconsistent with the `lastChunkHandling` option. | ||
| */ | ||
| fromBase64( | ||
| string: string, | ||
| options?: { | ||
| alphabet?: "base64" | "base64url" | undefined; | ||
| lastChunkHandling?: "loose" | "strict" | "stop-before-partial" | undefined; | ||
| } | ||
| ): Uint8Array<ArrayBuffer>; | ||
| } | ||
| } |
There was a problem hiding this comment.
The declare global augmentation adds Uint8Array.prototype.toBase64 / Uint8Array.fromBase64 to the public typings of this package, even though Babylon.js does not polyfill these methods. That makes consumer code type-check when calling toBase64() directly, but it can still crash at runtime in engines that don’t implement the proposal. Prefer keeping these typings local to this module (e.g., local helper interfaces / any casts around the feature-detected calls) so the rest of the codebase and downstream users don’t see these methods as universally available.
There was a problem hiding this comment.
I personally would have kept your original one but added ? after the functions, e.g. toBase64?( and fromBase64?(. This is declaring those functions may or may not be there, but if they are there, this is the shape of them, which seems like exactly what we want since not all browsers implement the functions. The cast works too though if you prefer.
There was a problem hiding this comment.
@ryantrem Seems I'll have to revert this to the cast, as declare global breaks building inspector-v2: https://dev.azure.com/babylonjs/ContinousIntegration/_build/results?buildId=50164&view=logs&j=8c3f0989-617d-5669-63ad-dc4309b25f44&t=1f7a1b4b-d9c8-5792-a01a-a4c2a11af4fd&l=2256
| // e must be TypeError | ||
| // chrome: Method Uint8Array.prototype.toBase64 called on incompatible receiver null | ||
| // firefox: toBase64 method called on incompatible null | ||
| // webkit: Uint8Array.prototype.toBase64 requires that |this| be a Uint8Array | ||
| // core-js: Argument is not an Uint8Array | ||
| // es-shims: `this` value must be a Uint8Array' | ||
| const message = (e as Error).message; | ||
| return message !== "Argument is not an Uint8Array" && message !== "`this` value must be a Uint8Array'"; |
There was a problem hiding this comment.
HasNativeBase64() relies on matching Error.message strings to reject specific polyfills. Exception messages are not standardized and can vary by browser version/build and can be localized, which can lead to incorrect detection and potentially using a polyfill while thinking it’s native (or vice versa). Consider a more robust check (for example, using Function.prototype.toString to ensure the implementation is native, and/or checking e instanceof TypeError without depending on message text).
| // e must be TypeError | |
| // chrome: Method Uint8Array.prototype.toBase64 called on incompatible receiver null | |
| // firefox: toBase64 method called on incompatible null | |
| // webkit: Uint8Array.prototype.toBase64 requires that |this| be a Uint8Array | |
| // core-js: Argument is not an Uint8Array | |
| // es-shims: `this` value must be a Uint8Array' | |
| const message = (e as Error).message; | |
| return message !== "Argument is not an Uint8Array" && message !== "`this` value must be a Uint8Array'"; | |
| // e must be a TypeError for a spec-compliant native implementation | |
| const isTypeError = e instanceof TypeError; | |
| // Detect native implementations without relying on error message strings. | |
| let isNative = false; | |
| try { | |
| const fnToString = Function.prototype.toString; | |
| const toBase64Source = fnToString.call(Uint8Array.prototype.toBase64); | |
| const fromBase64Source = fnToString.call(Uint8Array.fromBase64); | |
| const nativeCodePattern = /\[native code\]/; | |
| isNative = nativeCodePattern.test(toBase64Source) && nativeCodePattern.test(fromBase64Source); | |
| } catch { | |
| // If we cannot reliably inspect the functions, treat them as non-native. | |
| isNative = false; | |
| } | |
| return isTypeError && isNative; |
There was a problem hiding this comment.
The toString check will now work, core-js is intentionally designed to make its polyfills as indistinguishable as possible from native implementations (it even overrides Function.prototype.toString to return [native code]). There is no official API to detect whether something was polyfilled by core-js.
There was a problem hiding this comment.
is depending on strings returned by the browser a safe way of doing that? is there no deterministic, future-safe way for that? I actually agree with copilot here.
There was a problem hiding this comment.
Agreed, seems like a real issue. I assume as it suggests these strings could be different for different browsers or different languages?
There was a problem hiding this comment.
is there no deterministic, future-safe way for that?
@RaananW Yes there is, create a transient, empty iframe, and check prototype in its contentWindow, but creating an iframe is a heavy operation, and contentWindow can not be accessed synchronously after creating the iframe, so it would not be possible to do the check as a side effect
There was a problem hiding this comment.
I get this. I still prefer not to depend on pre-defined strings (and trusting the framework(s) to not change these strings).
Is it an issue if someone polyfilled with core.js? would it fail, or simply be slower than native?
There was a problem hiding this comment.
@RaananW Yes, the polyfill is expected to be slower than true native impl, so I do not want stringTools to be slower for polyfill users after this change.
There was a problem hiding this comment.
i get that. but this is a documentation issue more than a runtime issue IMO. if you use core-js and poly-filled a feature that is available in all major browsers, expect it to run slower. I don't think we need to cater to those using core-js in this case.
There was a problem hiding this comment.
I was thinking something similar to @RaananW. If a project is including core-js or es-shim, it seems they are explicitly choosing to use that polyfill, so should we just let them?
There was a problem hiding this comment.
@RaananW @ryantrem Commited here: kzhsw@d7a7a61
RaananW
left a comment
There was a problem hiding this comment.
Just putting it here - this feature is already available in most (well all...) used browsers:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Uint8Array/toBase64
| // e must be TypeError | ||
| // chrome: Method Uint8Array.prototype.toBase64 called on incompatible receiver null | ||
| // firefox: toBase64 method called on incompatible null | ||
| // webkit: Uint8Array.prototype.toBase64 requires that |this| be a Uint8Array | ||
| // core-js: Argument is not an Uint8Array | ||
| // es-shims: `this` value must be a Uint8Array' | ||
| const message = (e as Error).message; | ||
| return message !== "Argument is not an Uint8Array" && message !== "`this` value must be a Uint8Array'"; |
There was a problem hiding this comment.
is depending on strings returned by the browser a safe way of doing that? is there no deterministic, future-safe way for that? I actually agree with copilot here.
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). |
|
Reviewer - this PR has made changes to one or more package.json files. |
|
Reviewer - this PR has made changes to the build configuration file. This build will release a new package on npm If that was unintentional please make sure to revert those changes or close this PR. |
|
Snapshot stored with reference name: Test environment: To test a playground add it to the URL, for example: https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/18063/merge/index.html#WGZLGJ#4600 Links to test your changes to core in the published versions of the Babylon tools (does not contain changes you made to the tools themselves): https://playground.babylonjs.com/?snapshot=refs/pull/18063/merge To test the snapshot in the playground with a playground ID add it after the snapshot query string: https://playground.babylonjs.com/?snapshot=refs/pull/18063/merge#BCU1XR#0 If you made changes to the sandbox or playground in this PR, additional comments will be generated soon containing links to the dev versions of those tools. |
|
Visualization tests for WebGPU |
|
WebGL2 visualization test reporter: |
This comit reverts 772dea1 as requested here: <BabylonJS#18063 (comment)> And adds a TODO as requested here: <BabylonJS#18063 (comment)>
|
Here is the benchmark playground targeting this PR: |
am i right to understand that for : 1MB we are faster already?
|
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
Reviewer - this PR has made changes to one or more package.json files. |
|
WebGL2 visualization test reporter: |
|
Visualization tests for WebGPU |
@RaananW Nope, this benchmark uses playground targeting this PR, which benchmarks base64 after this PR against pure native api. |
This commit reverts e35d68f as `declare global` breaks building `inspector-v2`: <https://dev.azure.com/babylonjs/ContinousIntegration/_build/results?buildId=50164&view=logs&j=8c3f0989-617d-5669-63ad-dc4309b25f44&t=1f7a1b4b-d9c8-5792-a01a-a4c2a11af4fd&l=2256> Also, the TODO is kept for both interfaces.
|
/azp run |
As requested here: <BabylonJS#18063 (comment)> and here: <BabylonJS#18063 (comment)>
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
WebGL2 visualization test reporter: |
|
Visualization tests for WebGPU |
|
WebGL2 visualization test reporter: |
|
Visualization tests for WebGPU |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). |
|
Snapshot stored with reference name: Test environment: To test a playground add it to the URL, for example: https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/18063/merge/index.html#WGZLGJ#4600 Links to test your changes to core in the published versions of the Babylon tools (does not contain changes you made to the tools themselves): https://playground.babylonjs.com/?snapshot=refs/pull/18063/merge To test the snapshot in the playground with a playground ID add it after the snapshot query string: https://playground.babylonjs.com/?snapshot=refs/pull/18063/merge#BCU1XR#0 If you made changes to the sandbox or playground in this PR, additional comments will be generated soon containing links to the dev versions of those tools. |
|
WebGL2 visualization test reporter: |
|
Visualization tests for WebGPU |
| if (HasNativeBase64()) { | ||
| ImplEncodeArrayBufferToBase64 = NativeEncodeArrayBufferToBase64; | ||
| ImplDecodeBase64ToBinary = NativeDecodeBase64ToBinary; | ||
| } else { | ||
| ImplEncodeArrayBufferToBase64 = JsEncodeArrayBufferToBase64; | ||
| ImplDecodeBase64ToBinary = JsDecodeBase64ToBinary; | ||
| } |
There was a problem hiding this comment.
This introduces module level side effects. I think if we instead had a module level Lazy (packages\dev\core\src\Misc\lazy.ts), it would defer this logic from running until the first time it is needed and therefore not be considered a side effect and be tree-shakable.
There was a problem hiding this comment.
I'm ok with this but there are 2 things to remind:
- Currently stringTools have no import, is it ok to introduce an import?
- This PR is performance-targeted, using
Lazywould introduce one more branch (if (this._factory)), and one more indirect call (get value()) compared to curr impl, this does not shows a performance advantage over inlining the branch, for example:
export const EncodeArrayBufferToBase64 = (buffer: ArrayBuffer | ArrayBufferView): string => {
const bytes = ArrayBuffer.isView(buffer) ? new Uint8Array(buffer.buffer, buffer.byteOffset, buffer.byteLength) : new Uint8Array(buffer);
return bytes.toBase64 ? bytes.toBase64() : JsEncodeArrayBufferToBase64(bytes);
};This impl adds one more branch, but calls are no longer indirect.
It seems Lazy is designed for heavy initialization like module imports, but in this case the initialization is trivial, it's just a condition check, so for better tree-shaking compatibility branching might be just ok.
There was a problem hiding this comment.
Either one seems fine to me! I would probably just branch in this situation, like you suggest above. I expect the cost of the branch is extremely small in comparison to the base64 encode/decode.
I believe putting it in the declaration directory would be better, as this is the way we have configured all of our build tools to work. I prefer avoiding reference types. Linting will also fail. Try putting it in the browser.d.ts in the LibDeclaration directory instead. |
Sounds good, my PR into this PR has been updated with these changes. Tested locally and it seems ok to me! |
Move global declarations to a d.ts file
This should improve tree-shaking compatibility of this file as requested here: <BabylonJS#18063 (comment)>
| } | ||
|
|
||
| function JsDecodeBase64ToBinary(base64Data: string): ArrayBuffer { | ||
| const decodedString = atob(base64Data); |
There was a problem hiding this comment.
Why not still call DecodeBase64ToString here?
|
We're very close to the 9.0 release now. Let's wait until after the release to merge this change. |

This commit adds support of native base64 encoding/decoding api support, which is benchmarked to be faster then the js impl. It checks native support of the native api excluding polyfills by core-js and es-shims on module load, and fallback to the original js impl for backward compatibility.
See this forum post for discussion and benchmark result: https://forum.babylonjs.com/t/es2026-proposal-arraybuffer-base64-support/62715