[BUG]: NaT inside a datetime64 array serializes as the string "NaT" instead of null #5829

Description

At 38374ac, a NaT inside a datetime64 array serializes as the string "NaT" on both engines. The same missing value as a list element or a scalar becomes null, and from_json reads the string back as data.

$ python -I repro.py   # 38374ac, pandas 3.0.6, numpy 2.5.4, orjson 3.13.0
stored : ndarray datetime64[us]
json series -> ['2020-01-01T00:00:00.000000', 'NaT']
json list   -> ['2020-01-01T00:00:00', None]
json scalar -> null
json round  -> ['2020-01-01T00:00:00.000000', 'NaT']
orjson series -> ['2020-01-01T00:00:00.000000', 'NaT']
orjson list   -> ['2020-01-01T00:00:00', None]
orjson scalar -> null
orjson round  -> ['2020-01-01T00:00:00.000000', 'NaT']

plotly.js happens to draw a gap for the string, so a chart looks right. Anything else that reads the figure JSON, such as Dash figure State, gets a string where null belongs.

Screenshots/Video

None: the defect is in the serialized JSON, quoted above.

Steps to reproduce

import json
import pandas as pd
import plotly.graph_objects as go
import plotly.io as pio
from plotly.io._json import to_json_plotly

series = pd.Series(pd.to_datetime(["2020-01-01", None]))
fig = go.Figure(go.Scatter(x=series))
print("stored :", type(fig.data[0].x).__name__, fig.data[0].x.dtype)
fig2 = go.Figure(go.Scatter(x=[pd.Timestamp("2020-01-01"), pd.NaT]))
for e in ("json", "orjson"):
    print(e, "series ->", json.loads(pio.to_json(fig, engine=e))["data"][0]["x"])
    print(e, "list   ->", json.loads(pio.to_json(fig2, engine=e))["data"][0]["x"])
    print(e, "scalar ->", to_json_plotly(pd.NaT, engine=e))
    print(e, "round  ->", list(pio.from_json(pio.to_json(fig, engine=e), engine=e).data[0].x))

Notes

contract: tests/test_optional/test_utils/test_utils.py:276 already asserts null for pd.NaT inside an object-dtype Series, and :291 asserts "[null]" for [pd.NaT]. The datetime64 dtype of that same Series breaks the promise. The scalar case is tests/test_io/test_to_from_plotly_json.py:265.

The json engine (the auto fallback when orjson is absent) goes through encode_as_numpy at _plotly_utils/utils.py:297, and orjson goes through clean_to_json_compatible at plotly/io/_json.py:517. Both call datetime_as_string unfiltered:

-                return numpy.datetime_as_string(obj).tolist()
+                return [
+                    None if s == "NaT" else s
+                    for s in numpy.datetime_as_string(obj).tolist()
+                ]

$ python -I repro.py | grep -E "series|round"   # filter at utils.py:297 only
json series -> ['2020-01-01T00:00:00.000000', None]
json round  -> ['2020-01-01T00:00:00.000000', None]
orjson series -> ['2020-01-01T00:00:00.000000', 'NaT']
orjson round  -> ['2020-01-01T00:00:00.000000', 'NaT']
$ python -I repro.py | grep -E "series|round"   # same filter at _json.py:517 too
json series -> ['2020-01-01T00:00:00.000000', None]
json round  -> ['2020-01-01T00:00:00.000000', None]
orjson series -> ['2020-01-01T00:00:00.000000', None]
orjson round  -> ['2020-01-01T00:00:00.000000', None]
# reverted: all four lines print 'NaT' again

No CODEOWNERS entry covers these paths. @camdecoster, would you take a PR with the filter at both sites and a datetime64 case next to test_pandas_json_encoding?