Skip to content

[Bug]: RowJson rejects integral JSON values for a DOUBLE field when they exceed int range #39743

Description

@PDGGK

What happened?

A DOUBLE field rejects any integral JSON literal outside int range, even when a double holds it exactly:

// RowJsonValueExtractors.java:138, inside doubleValueExtractor()'s validator
|| (jsonNode.isIntegralNumber()
    && jsonNode.canConvertToLong()
    && jsonNode.asLong() == (long) (double) jsonNode.asInt())

asLong() on the left, asInt() on the right. asInt() truncates anything outside int range, so the two sides can never agree for a larger value and it falls through to "out of range".

Epoch millis is the everyday case — this fails today:

{"f_double": 1609459200000}

against a schema with FieldType.DOUBLE, even though 1609459200000.0d is exact.

The guard on the line above is canConvertToLong(), so the intent is clearly "an integral value that fits in a long and survives the trip through double". The asInt() is at odds with its own guard.

Anything under 2^53 is affected: epoch millis, IDs, byte counts. Values that happen to fit in an int work, and a producer that emits 1.6094592e12 works, which is presumably why this has gone unnoticed.

Note for whoever fixes this

Swapping asInt() for asLong() is not the fix. (double) Long.MAX_VALUE rounds up to 2^63, and narrowing 2^63 back to long saturates at Long.MAX_VALUE rather than overflowing, so the round-trip appears to succeed and the extractor stores 9223372036854775808 — trading an over-rejection for silent corruption. RowJsonTest's existing LONG_STRING is 2^63 − 2, which does not exercise that.

Comparing through BigDecimal is exact, and is already what the decimal branch immediately below does.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

  • Component: Java SDK

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions