Implement `ANALYZE TABLE` support

[jonathanc-n]

Is your feature request related to a problem or challenge?

ANALYZE TABLE command is used to calculate statistics. Spark's documentation can be seen here

This can allow us to do things like:

• Accurately calculating distinct values (with the choice of just counting everything or using a hyperloglog)
• Allowing the user to recalculate statistics and calculate them upfront instead of lazily if suspected statistics have gotten stale.
• Memtable stat calculations

Describe the solution you'd like

Something similar to spark's implementation makes sense

Syntax:

ANALYZE TABLE table_identifier 
  COMPUTE STATISTICS [ NOSCAN | FOR COLUMNS col [, ...] | FOR ALL COLUMNS ]

COMPUTE STATISTICS (nothing else after it): Only calculate table stats (num_rows + total_byte_size)
NOSCAN: Don't probe/scan any values - just return
FOR COLUMNS col [, ...]: table stats + calculating specified columns stats (min, max, null, etc.)
FOR ALL COLUMNS: All column stats are calculated (min, max, null, etc.)

• when specified for specific columns; when it is under a certain amount of columns + row count. A direct computation of distinct count can be done.

• Maybe histogram support, this will need to be discussed if anything

• Maybe a way to persist statistics

Describe alternatives you've considered

No response

Additional context

No response

[jonathanc-n]

take

[jonathanc-n]

@asolimando wdyt?

[asolimando]

It would be great to have a way to compute statistics programmatically, but the price of computing them generally pays off only if amortized over multiple runs, which requires persisting them.

So unless we have a way to do that, the benefit would be limited.

One possible use-case, not requiring persistence, is to enable query replay with this command activated on some sub-sample of production traffic, note the runtime difference, to easily discover good candidates for statistics pre-computation.

In that space, a more standard technique for discovering candidates is to look at the query catalog/history: for NDV, columns appearing in join conditions, GROUP BY clauses, and DISTINCT operations are natural candidates.

On the persistence side, for Parquet-backed tables, one option is to embed computed statistics directly in the files using the user-defined Parquet indexes technique: https://datafusion.apache.org/blog/2025/07/14/user-defined-parquet-indexes/

Stats travel with the data, no external store needed, and other readers simply ignore the extra bytes. Though this wouldn't cover non-Parquet sources and it would require write permission, but it would be a start.

WDYT?