[core][spark] Add statistics sidecar for NDV sketches

[kerwin-zk]

Purpose

For #8381.

This PR implements a core + Spark statistics sidecar vertical slice for mergeable NDV sketches.

It adds:

• StatisticsBlobMetadata for typed blob references, field ids, optional snapshot / sequence numbers, properties, sidecar file location, offset, and length.
• StatisticsBlob and StatisticsSidecarFile to write multiple statistics blob payloads into one sidecar file and read them back by metadata.
• Statistics.blobMetadata JSON serialization with backward-compatible reads when the field is absent.
• StatsFileHandler sidecar read/write APIs and sidecar cleanup when deleting statistics files.
• orphan file cleanup protection for sidecar files referenced by live statistics files.
• FileStorePathFactory.statsSidecarFileFactory() and the stat-sidecar- prefix, so binary sidecars are distinguishable from JSON statistics files under the statistics directory.
• StatisticsNdvSketch for Paimon-specific paimon-ndv-theta-sketch-v1 blobs. The payload is an Apache DataSketches compact Theta sketch, while the container is Paimon's own statistics sidecar format.
• Spark ANALYZE TABLE ... COMPUTE STATISTICS FOR COLUMNS sidecar production behind spark.paimon.analyze.ndv-sketch.enabled (default: false). When enabled, Spark writes one NDV sketch blob per analyzed column and attaches the sidecar metadata to the committed Statistics file.

Existing scalar column statistics remain unchanged; the sidecar metadata is additional optional statistics metadata.

Tests

CI

[JingsongLi]

Can this PR be fully implemented? We will first determine whether this feature is overall reasonable through a complete implementation.

[kerwin-zk]

Thanks for the suggestion and the review.

I have updated this PR from a metadata-only proposal to a complete core implementation. The PR now includes:

• statistics sidecar read / write support for multiple typed binary blob payloads;
• Statistics.blobMetadata JSON lifecycle with backward compatibility;
• StatsFileHandler sidecar APIs, sidecar cleanup on statistics deletion, and orphan-file cleanup protection for sidecars still referenced by live statistics files;
• mergeable NDV blob support based on Apache DataSketches Theta sketch;
• a Paimon-specific NDV blob type, paimon-ndv-theta-sketch-v1, since this PR uses Paimon's own sidecar container rather than an Iceberg Puffin file;
• a separate stat-sidecar- file prefix via statsSidecarFileFactory(), so binary sidecars do not share the JSON statistics file prefix;
• validation for sidecar writes and reads, including rejecting empty sidecar writes and checking offset + length against the actual sidecar file length before reading;
• documentation that the ndv metadata property is a cached estimate.

I kept Spark ANALYZE / CBO producer wiring out of this PR intentionally, so that we can first review the core format and lifecycle behavior. If this direction looks reasonable, Spark can be wired to
produce and consume the NDV sidecar in a follow-up PR.

[JingsongLi]

@kerwin-zk I think you can merge ALL changes in this single PR. If we reach a consensus, you can then break it down into smaller PRs to continue the progress.