Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Intent to use BigInt/numeric union in WebNN #1388

Open
fdwr opened this issue Feb 17, 2024 · 6 comments
Open

Intent to use BigInt/numeric union in WebNN #1388

fdwr opened this issue Feb 17, 2024 · 6 comments

Comments

@fdwr
Copy link

fdwr commented Feb 17, 2024

What is the issue with the Web IDL Standard?

Opening an issue per the wording here: https://webidl.spec.whatwg.org/#limit-bigint-numeric-unions

It would not be appropriate to accept such a union, only to then convert values of the numeric type to a bigint for further processing, as this runs the risk of introducing precision errors. Please file an issue before using this feature.

We have 3 use cases in WebNN where we need to be able to accept full precision from the API and retain it passed all the way down to the bottom, because float64 cannot express the full precision of int64 (and conversely, an int64 could not express the full precision of a float64).

Evidently it's murky territory, as Bin Miao hit a bug in the Chromium implementation "We met the problem of using the keyword or in IDL to create a union type containing double and bigint. In the generated file 'v8_union_bigint_double.cc', it was only judged that the incoming data was of JS number type and there is no judgment that it is BigInt, which causes an error to be reported when BigInt data is passed in when calling this API using JS code.", and Joshua notes that "Use of bigint is extremely limited in Chromium at the moment, so I'm not surprised if the binding code generation can't handle a union with it yet ... this may just be the first spec".

FYI @inexorabletash, @annevk, @Honry, @miaobin

@annevk annevk changed the title Intent to use BigInt/numeric union Intent to use BigInt/numeric union in WebNN Feb 28, 2024
@annevk
Copy link
Member

annevk commented Feb 28, 2024

Reading those three issues it seems like upgrading to double or long long would get you most of the way? With JS you should have ~int53 or some such, which is way more than what you are clamping at currently. It doesn't give you precision beyond that, but it's not immediately clear from the issues whether that's needed.

@fdwr
Copy link
Author

fdwr commented Feb 29, 2024

Howdy Anne. We're looking at a union of (double and BigInt), where double (float64) is large enough to handle any floating point constant (float16, float32, float64), and BigInt is large enough to handle any integer constant (int8, uint8, int32, uint32, int64, uint64). Are you suggesting a union of (double or long long)?

Using only double gets us most of the way, yes, but it's the endpoints that concern me. We want JS to be able to accurately map the full range of int64/uint64 tensors (e.g. numpy.uint64, tensorflow.uint64, ONNX.TensorProto.DataType.UINT64, BNNSDataTypeUInt64, MPSDataType.uInt64, mx.uint64, ov::element::Type_t::u64...), and yeah, most values past 2^52 likely don't need exact precision, but for cases like sentinel values and masks (e.g. 0xFFFFFFFFFFFFFFFF), we'd have a challenge representing them via a double. You would think that passing a really large double like 1.8446744073709552e+19 might map cleanly to 0xFFFFFFFFFFFFFFFF, but you get varying results depending on the FPU logic. Even with C++ on my x64 machine, if you attempt to static_cast a double like 1.8446744073709552e+19 to uint64_t, you get back a surprising answer of 0x8000000000000000 instead of 0xFFFFFFFFFFFFFFFF, and I don't know what happens on ARM/M1.

For long long (https://webidl.spec.whatwg.org/#idl-long-long), is BigInt the way to express int64/uint64 via JS? Thanks.

@annevk
Copy link
Member

annevk commented Feb 29, 2024

No, neither double nor long long would give you int64 precision. It seems really strange to overload a double with an integer though. Seems like it should be (long long or BigInt) then.

@fdwr
Copy link
Author

fdwr commented Feb 29, 2024

Seems like it should be (long long or BigInt) then.

The WebNN user calls a function to apply an operation on a multidimensional tensor (e.g. fill a constant, fill a sequence, pad with a constant value) or to create a multidimensional tensor, where those tensors can be of various data types (including float64/int64/uint64/...), and the caller passes a scalar parameter to that function call which should be able to represent any of the target tensor's potential values. Using BigInt or long long would just introduce the opposite problem, as integer types cannot express float64 point precision (3.4 would be truncated to 3).

The current definition @miaobin has prototyped for a constant sequence fill is:

MLOperand constant(MLOperandDescriptor desc, (double or bigint) start, (double or bigint) step);

@annevk
Copy link
Member

annevk commented Feb 29, 2024

Sorry for being slow, that seems reasonable.

@fdwr
Copy link
Author

fdwr commented Feb 29, 2024

FYI @inexorabletash, Anne says it seems reasonable :).

chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this issue Mar 19, 2024
According to the WebNN spec[1], this CL defines mojo and extends the IDL
definition of constant MLOperand which can be created with the specified
data type and shape that contains the data as specified by the range.
Some related WPT tests and services unit tests are also added.

Since "bigint" currently has extremely limited use in Chromium, binding
code generation cannot yet handle unions with it[2]. We currently
support all data types supported by WebNN except int64 and uint64.

[1]: https://www.w3.org/TR/webnn/#api-mlgraphbuilder-constant-range
[2]: whatwg/webidl#1388

Bug: 40206287, 328109810
Change-Id: Iae0212071fc3567cebeeb8abb306b1d3b81ca462
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this issue Mar 22, 2024
According to the WebNN spec[1], this CL defines mojo and extends the IDL
definition of constant MLOperand which can be created with the specified
data type and shape that contains the data as specified by the range.
Some related WPT tests and services unit tests are also added.

Since "bigint" currently has extremely limited use in Chromium, binding
code generation cannot yet handle unions with it[2]. We currently
support all data types supported by WebNN except int64 and uint64.

[1]: https://www.w3.org/TR/webnn/#api-mlgraphbuilder-constant-range
[2]: whatwg/webidl#1388

Bug: 40206287, 328109810
Change-Id: Iae0212071fc3567cebeeb8abb306b1d3b81ca462
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this issue Mar 27, 2024
According to the WebNN spec[1], this CL defines mojo and extends the IDL
definition of constant MLOperand which can be created with the specified
data type and shape that contains the data as specified by the range.
Some related WPT tests and services unit tests are also added.

Since "bigint" currently has extremely limited use in Chromium, binding
code generation cannot yet handle unions with it[2]. We currently
support all data types supported by WebNN except int64 and uint64.

[1]: https://www.w3.org/TR/webnn/#api-mlgraphbuilder-constant-range
[2]: whatwg/webidl#1388

Bug: 40206287, 328109810
Change-Id: Iae0212071fc3567cebeeb8abb306b1d3b81ca462
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this issue Mar 27, 2024
According to the WebNN spec[1], this CL defines mojo and extends the IDL
definition of constant MLOperand which can be created with the specified
data type and shape that contains the data as specified by the range.
Some related WPT tests and services unit tests are also added.

Since "bigint" currently has extremely limited use in Chromium, binding
code generation cannot yet handle unions with it[2]. We currently
support all data types supported by WebNN except int64 and uint64.

[1]: https://www.w3.org/TR/webnn/#api-mlgraphbuilder-constant-range
[2]: whatwg/webidl#1388

Bug: 40206287, 328109810
Change-Id: Iae0212071fc3567cebeeb8abb306b1d3b81ca462
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this issue Mar 27, 2024
According to the WebNN spec[1], this CL defines mojo and extends the IDL
definition of constant MLOperand which can be created with the specified
data type and shape that contains the data as specified by the range.
Some related WPT tests and services unit tests are also added.

Since "bigint" currently has extremely limited use in Chromium, binding
code generation cannot yet handle unions with it[2]. We currently
support all data types supported by WebNN except int64 and uint64.

[1]: https://www.w3.org/TR/webnn/#api-mlgraphbuilder-constant-range
[2]: whatwg/webidl#1388

Bug: 40206287, 328109810
Change-Id: Iae0212071fc3567cebeeb8abb306b1d3b81ca462
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this issue Apr 1, 2024
According to the WebNN spec[1], this CL defines mojo and extends the IDL
definition of constant MLOperand which can be created with the specified
data type and shape that contains the data as specified by the range.
Some related WPT tests and services unit tests are also added.

Since "bigint" currently has extremely limited use in Chromium, binding
code generation cannot yet handle unions with it[2]. We currently
support all data types supported by WebNN except int64 and uint64.

[1]: https://www.w3.org/TR/webnn/#api-mlgraphbuilder-constant-range
[2]: whatwg/webidl#1388

Bug: 40206287, 328109810
Change-Id: Iae0212071fc3567cebeeb8abb306b1d3b81ca462
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this issue Apr 2, 2024
According to the WebNN spec[1], this CL defines mojo and extends the IDL
definition of constant MLOperand which can be created with the specified
data type and shape that contains the data as specified by the range.
Some related WPT tests and services unit tests are also added.

Since "bigint" currently has extremely limited use in Chromium, binding
code generation cannot yet handle unions with it[2]. We currently
support all data types supported by WebNN except int64 and uint64.

[1]: https://www.w3.org/TR/webnn/#api-mlgraphbuilder-constant-range
[2]: whatwg/webidl#1388

Bug: 40206287, 328109810
Change-Id: Iae0212071fc3567cebeeb8abb306b1d3b81ca462
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this issue Apr 8, 2024
According to the WebNN spec[1], this CL defines mojo and extends the IDL
definition of constant MLOperand which can be created with the specified
data type and shape that contains the data as specified by the range.
Some related WPT tests and services unit tests are also added.

Since "bigint" currently has extremely limited use in Chromium, binding
code generation cannot yet handle unions with it[2]. We currently
support all data types supported by WebNN except int64 and uint64.

[1]: https://www.w3.org/TR/webnn/#api-mlgraphbuilder-constant-range
[2]: whatwg/webidl#1388

Bug: 40206287, 328109810
Change-Id: Iae0212071fc3567cebeeb8abb306b1d3b81ca462
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this issue Apr 9, 2024
According to the WebNN spec[1], this CL defines mojo and extends the IDL
definition of constant MLOperand which can be created with the specified
data type and shape that contains the data as specified by the range.
Some related WPT tests and services unit tests are also added.

Since "bigint" currently has extremely limited use in Chromium, binding
code generation cannot yet handle unions with it[2]. We currently
support all data types supported by WebNN except int64 and uint64.

[1]: https://www.w3.org/TR/webnn/#api-mlgraphbuilder-constant-range
[2]: whatwg/webidl#1388

Bug: 40206287, 328109810
Change-Id: Iae0212071fc3567cebeeb8abb306b1d3b81ca462
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this issue Apr 10, 2024
According to the WebNN spec[1], this CL defines mojo and extends the IDL
definition of constant MLOperand which can be created with the specified
data type and shape that contains the data as specified by the range.
Some related WPT tests and services unit tests are also added.

Since "bigint" currently has extremely limited use in Chromium, binding
code generation cannot yet handle unions with it[2]. We currently
support all data types supported by WebNN except int64 and uint64.

[1]: https://www.w3.org/TR/webnn/#api-mlgraphbuilder-constant-range
[2]: whatwg/webidl#1388

Bug: 40206287, 328109810
Change-Id: Iae0212071fc3567cebeeb8abb306b1d3b81ca462
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this issue Apr 10, 2024
According to the WebNN spec[1], this CL defines mojo and extends the IDL
definition of constant MLOperand which can be created with the specified
data type and shape that contains the data as specified by the range.
Some related WPT tests and services unit tests are also added.

Since "bigint" currently has extremely limited use in Chromium, binding
code generation cannot yet handle unions with it[2]. We currently
support all data types supported by WebNN except int64 and uint64.

[1]: https://www.w3.org/TR/webnn/#api-mlgraphbuilder-constant-range
[2]: whatwg/webidl#1388

Bug: 40206287, 328109810
Change-Id: Iae0212071fc3567cebeeb8abb306b1d3b81ca462
inexorabletash added a commit to inexorabletash/webnn that referenced this issue Apr 18, 2024
For some MLGraphBuilder methods the type of a numeric input can vary -
e.g. for constant() an explicit MLOperandDataType is provided; for
clamp() and pad() the data type is implied by input operands. In these
cases, specifying the numeric value as either a float/double or int64
type runs into accuracy or range issues - you can't accurately
represent all int64 values as a double, and you can't represent the
full range of floats as int64. (You also can't represent all int64
values as an long long either - over 2^53 things get wierd. But that's
a digression.)

Per discussion in whatwg/webidl#1388 this
change introduces a union between a bigint type and unrestricted
double called MLNumber. Callers can pass a JS number (1234, 1.1234e38)
or a JS bigint (9007199254740993n), and the implementation will treat
it properly based on the explicit or implicit MLOperandDataType. Usage
of this type should be limited to only those cases.

Fixes webmachinelearning#442

Note that webmachinelearning#492 proposes changes to the constant sequential filling
operation; this just adds IDL to match the current spec prose.

Some of the concerns raised in webmachinelearning#325 are addressed (e.g. clamp()'s
options). However, several other options are still specified as
"float", and should maybe be "double" - but MLNumber is likely not
appropriate for those, so they are not updated here.
huningxin added a commit to webmachinelearning/webnn that referenced this issue Jul 5, 2024
For some `MLGraphBuilder` methods the type of a numeric input can vary -
e.g. for `constant()` an explicit `MLOperandDataType` is provided; for
`clamp()` and `pad()` the data type is implied by input operands. In
these cases, specifying the numeric value as either a float/double or
int64 type runs into accuracy or range issues - you can't accurately
represent all int64 values as a double, and you can't represent the full
range of floats as int64. (You also can't represent all int64 values as
an long long either - over 2^53 things get wierd. But that's a
digression.)

- Some methods that took a float or double argument/option but can
  operate on integer data types now take `MLNumber` - a "variant"
  (union) of either a JS Number (equivalent to a double a.k.a. float64)
  or BigInt (for full precision int64/uint64 use)
  - `constant()` (scalar overload) - `MLNumber`
  - `clamp()` (min/max options) - `MLNumber`
  - `pad()` (padding value) - `MLNumber`
  - This fixes #442
- Other methods that took a `float` argument/option now take a `double`.
  These are all ops that only operate on floating point types, so no
  need for `MLNumber` though it would be harmless to use it there to
  allow BigInt inputs. This follows web API best practices, and allows
  full precision conversion to float16 (since float64 to float32 to
  float16 can yield different results than float64 directly to float16)
  - `batchNormalization()`, `instanceNormalization()`,
    `layerNormalization()` - `epsilon` option
  - `elu()`, `gemm()`, `hardSigmoid()`, `leakyRelu()`, `linear()`  -
    `alpha` and `beta` options
  - This fixes #325
- In all of these cases, the input number is cast to the input/output
operand data type when it is used. Casting algorithms are spelled out,
always have "clamp" semantics (i.e. no weird modulus wrapping), and
never fail.
  - For `MLOperand`-vending methods, the conversion can be done eagerly.
  - For `MLActivation`s, the conversion is done at "fusion time"; it's
    notable that the same `MLActivation` instance could be fused with
    multiple ops, and cast differently for each.
  - This fixes #678
- Using a BigInt/numeric union is novel here, but WebIDL experts agree
  it's reasonable (whatwg/webidl/issues/1388). Note that all existing
  WebIDL/JS APIs that are designed to take just a Number or just a
  BigInt will throw if the other thing is passed - there is
  intentionally no silent conversion. So we are in novel territory here
  and should look for implementation and developer feedback.

Fixes #442, #325, #678 

---------

Co-authored-by: Dwayne Robinson <[email protected]>
Co-authored-by: Ningxin Hu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants