Skip to content

[ntuple] Precision loss for big integer values #416

Description

@undefined-panda

When reading RNTuple data, Int64/UInt64 values above Number.MAX_SAFE_INTEGER or below Number.MIN_SAFE_INTEGER lose precision because they're cast to Number in rntuple.mjs:

// FIXME: let process BigInt in the TTree::Draw
obj[this.name] = Number(this.view.getBigInt64(this.o, LITTLE_ENDIAN));

The FIXME comment suggests this is known. Since Number and BigInt are distinct types in JS, would it make sense to return a BigInt only when the value doesn't fit in a safe integer, and a Number otherwise? Something like:

const num = this.view.getBigInt64(this.o, LITTLE_ENDIAN);
const MIN = BigInt(Number.MIN_SAFE_INTEGER);
const MAX = BigInt(Number.MAX_SAFE_INTEGER);
obj[this.name] = (num >= MIN && num <= MAX) ? Number(num) : num;

I could open a PR if this direction makes sense, though I'm not sure how it interacts with TTree::Draw.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions