[wasm] Emit webcil payloadSize/tableSize into boot config for streaming instantiation - #131658
[wasm] Emit webcil payloadSize/tableSize into boot config for streaming instantiation#131658pavelsavara wants to merge 1 commit into
Conversation
…ng instantiation The WASM loader previously instantiated each webcil-in-wasm assembly by buffering its bytes and then calling the module's getWebcilSize/getWebcilPayload exports to discover the payload size at runtime. This records the payload size (and, for ReadyToRun images, the indirect-call table size) in the boot config so the loader can stream-instantiate directly. - Boot config: GeneralAsset gains optional payloadSize (emitted for every webcil-in-wasm assembly) and tableSize (R2R images only). Both use EmitDefaultValue=false so non-R2R assets stay unchanged apart from payloadSize. - ConvertDllsToWebcil computes the sizes from the produced webcil's wasm data segment and exposes them via a new WebcilSizes output. GenerateWasmBootJson consumes that output, with a self-contained fallback that re-reads the sizes straight from the produced webcil when the incremental convert task was skipped (e.g. a boot-config property changed but no assembly did). - Loader/host: instantiateWebcilModule now takes payloadSize/tableSize and uses WebAssembly.instantiateStreaming when possible, without buffering the bytes or parsing the wasm data section. For R2R images it wires the host ABI handshake (stack pointer, exception tag, indirect-call table + base, image base) and grows the table before instantiation.
|
Azure Pipelines: Successfully started running 3 pipeline(s). 13 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
|
Tagging subscribers to 'arch-wasm': @lewing, @pavelsavara |
There was a problem hiding this comment.
Pull request overview
This PR threads Webcil-in-wasm payload sizing (and future R2R table sizing) from the build pipeline into the generated boot config, then updates the JS loader/host to use those sizes to instantiate WebAssembly modules without buffering and data-section parsing.
Changes:
- Extend boot config assets to optionally include
payloadSize(all webcil-in-wasm) andtableSize(R2R-only), and plumb these values through boot config generation. - Teach the
ConvertDllsToWebciltask to compute sizes by parsing the produced wasm and emit them as an MSBuild output; add a GenerateWasmBootJson fallback that re-reads sizes from disk when the convert task is skipped incrementally. - Update the JS host/loader to pass sizes into
instantiateWebcilModuleand prefer streaming instantiation when the response MIME type allows it.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/tasks/Microsoft.NET.Sdk.WebAssembly.Pack.Tasks/GenerateWasmBootJson.cs | Collects Webcil sizes (task output or fallback re-read) and passes them into asset emission. |
| src/tasks/Microsoft.NET.Sdk.WebAssembly.Pack.Tasks/ConvertDllsToWebCil.cs | Computes payloadSize/tableSize by parsing produced wasm and exposes them via a new MSBuild output. |
| src/tasks/Microsoft.NET.Sdk.WebAssembly.Pack.Tasks/BootJsonData.cs | Adds optional payloadSize / tableSize fields to the emitted asset shape. |
| src/tasks/Microsoft.NET.Sdk.WebAssembly.Pack.Tasks/BootJsonBuilderHelper.cs | Maps computed sizes onto emitted GeneralAsset entries (including culture-qualified satellites). |
| src/native/libs/System.Native.Browser/libSystem.Native.Browser.Utils.footer.js | Preserves additional wasm exports needed for the R2R host ABI handshake. |
| src/native/libs/Common/JavaScript/types/public-api.ts | Adds a public WebcilAsset type documenting payloadSize/tableSize. |
| src/native/libs/Common/JavaScript/types/internal.ts | Extends internal asset entries to carry payloadSize/tableSize through loader pipelines. |
| src/native/libs/Common/JavaScript/types/ems-ambient.ts | Extends ambient emscripten typings for additional preserved wasm exports. |
| src/native/libs/Common/JavaScript/loader/assets.ts | Passes tableSize/payloadSize from assets into the host instantiation call. |
| src/native/libs/Common/JavaScript/host/assets.ts | Implements size-aware (streaming when possible) Webcil module instantiation and R2R import wiring. |
| src/mono/sample/wasm/Directory.Build.targets | Includes PublishReadyToRun in nested sample build propagation. |
| src/mono/sample/wasm/Directory.Build.props | Flows nested PublishReadyToRun into the outer build property when unset. |
| src/mono/nuget/Microsoft.NET.Sdk.WebAssembly.Pack/build/Microsoft.NET.Sdk.WebAssembly.Browser.targets | Wires the new WebcilSizes task output into boot config generation for build and publish. |
| const res = await checkWebcilResponse(webcilPromise, virtualPath); | ||
| const payloadPtr = allocWebcilPayload(payloadSize); | ||
| const imports: WebAssembly.Imports = { webcil: buildWebcilImports(memory, payloadPtr, tableEntries) }; | ||
|
|
||
| const { instance } = await instantiateWasm(webcilPromise, imports); | ||
| const webcilVersion = (instance.exports.webcilVersion as WebAssembly.Global).value; | ||
| if (webcilVersion !== 0) { | ||
| throw new Error(`Unsupported Webcil version: ${webcilVersion}`); | ||
| let instance: WebAssembly.Instance; |
| // Reads payloadSize and tableSize from data segment 0 of a Webcil-in-wasm image without | ||
| // instantiating it. tableSize > 0 indicates a ReadyToRun image. The data section is the last | ||
| // wasm section, so for R2R images (large code sections) it can start well beyond the first few | ||
| // KB; this streams through the section headers, seeking past each body, instead of reading a | ||
| // fixed prefix. All multi-byte integers in the wasm binary format are little-endian and are read | ||
| // as such regardless of host endianness. See docs/design/mono/webcil.md. |
| internal static bool TryReadWebcilSizes(string path, out int payloadSize, out int tableSize, out string failureReason) | ||
| { | ||
| payloadSize = 0; | ||
| tableSize = 0; | ||
| failureReason = null; |
| causing WasmApp.Common.props (Mono) to be imported even for CoreCLR builds. --> | ||
| <PropertyGroup> | ||
| <RuntimeFlavor Condition="'$(Nested_RuntimeFlavor)' != ''">$(Nested_RuntimeFlavor)</RuntimeFlavor> | ||
| <PublishReadyToRun Condition="'$(PublishReadyToRun)' == '' and '$(Nested_PublishReadyToRun)' != ''">$(Nested_PublishReadyToRun)</PublishReadyToRun> |
There was a problem hiding this comment.
Just to be sure, only properties that needs to be available very very early are required here.
Is this the case?
| /// this; it is omitted for plain (non-R2R) webcil. | ||
| /// </summary> | ||
| [DataMember(EmitDefaultValue = false)] | ||
| public int? tableSize { get; set; } |
There was a problem hiding this comment.
Please, create a WebcilAsset and put these specific props there.
| public ITaskItem[] WebcilCandidates { get; set; } | ||
|
|
||
| [Output] | ||
| public ITaskItem[] WebcilSizes { get; set; } |
There was a problem hiding this comment.
Can we store the size on the candidate as metadata?
| // A prebuilt R2R webcil-in-wasm image from the runtime pack replaces conversion of the .dll: | ||
| // stage (copy) it into the webcil output so it flows through the same downstream metadata as a | ||
| // converted assembly, but carries native code. The .dll is kept only as the metadata source. | ||
| string r2rWebcilPath = candidate.GetMetadata("R2RWebcilPath"); |
| // KB; this streams through the section headers, seeking past each body, instead of reading a | ||
| // fixed prefix. All multi-byte integers in the wasm binary format are little-endian and are read | ||
| // as such regardless of host endianness. See docs/design/mono/webcil.md. | ||
| internal static bool TryReadWebcilSizes(string path, out int payloadSize, out int tableSize, out string failureReason) |
There was a problem hiding this comment.
Shouldn't this belong to the WebcilReader.cs?
| // emitted for every webcil; tableSize only for R2R. Identify them by the produced | ||
| // ".wasm" extension, excluding native wasm (dotnet.native.wasm) which is handled | ||
| // separately and is not a webcil module. | ||
| bool isWebcilInWasmAssembly = IsTargeting100OrLater() |
|
|
||
| if (isWebcilInWasmAssembly) | ||
| { | ||
| // Fast path: ConvertDllsToWebcil already computed the sizes. Fall back to |
There was a problem hiding this comment.
I would expect we make the computation an incremental step as well, rather than doing a fallback here
Split from #129634
Summary
The WASM loader previously instantiated each webcil-in-wasm assembly by buffering its bytes and then calling the module's
getWebcilSize/getWebcilPayloadexports to discover the payload size at runtime. This records the payload size — and, for ReadyToRun images, the indirect-call table size — in the boot config so the loader can stream-instantiate the module directly.Changes
BootJsonData.GeneralAsset): adds optionalpayloadSize(emitted for every webcil-in-wasm assembly) andtableSize(R2R images only). Both useEmitDefaultValue = false, so non-R2R assets are unchanged apart frompayloadSize.ConvertDllsToWebcilcomputes the sizes from the produced webcil's wasm data segment and exposes them through a newWebcilSizesoutput.GenerateWasmBootJsonconsumes that output, with a self-contained fallback that re-reads the sizes straight from the produced webcil when the incremental convert task was skipped (e.g. a boot-config property changed but no assembly did).BootJsonBuilderHelpermaps the sizes onto the emitted assets, culture-qualified for satellites.host/assets.ts,loader/assets.ts):instantiateWebcilModuletakespayloadSize/tableSizeand usesWebAssembly.instantiateStreamingwhen possible, with no byte buffering and no data-section parsing. For R2R images it wires the host ABI handshake (stack pointer, exception tag, indirect-call table + base, image base) and grows the table before instantiation.Notes
payloadSizeis emitted for all webcil-in-wasm assemblies;tableSizeonly appears once ReadyToRun webcil-in-wasm images are produced (separate follow-up work).WebcilAssettype (payloadSize/tableSize) inpublic-api.ts.Note
This pull request description was generated with the assistance of GitHub Copilot.