Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Adds Histogram chart migration logic #28780

Conversation

michael-s-molina
Copy link
Member

@michael-s-molina michael-s-molina commented May 30, 2024

SUMMARY

This PR adds the Histogram chart migration logic (legacy ➡️ ECharts). Users can execute this migration using the CLI command and disable the legacy version with the VIZ_TYPE_DENYLIST configuration.

This PR also improves the Histogram chart to automatically handle NULL values.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Screenshot 2024-06-05 at 10 27 14 Screenshot 2024-06-05 at 10 29 12

TESTING INSTRUCTIONS

1 - Upgrade a Histogram (legacy) chart using the CLI command
2 - Check the new chart
3 - Downgrade a Histogram (legacy) chart using the CLI command
4 - Check the legacy chart

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@github-actions github-actions bot added the risk:db-migration PRs that require a DB migration label May 30, 2024
@michael-s-molina michael-s-molina removed the risk:db-migration PRs that require a DB migration label May 30, 2024
@michael-s-molina michael-s-molina force-pushed the add-echarts-histogram-migration branch from b7eeed8 to 687f159 Compare June 5, 2024 13:30
@pull-request-size pull-request-size bot added size/L and removed size/M labels Jun 5, 2024
@github-actions github-actions bot added risk:db-migration PRs that require a DB migration plugins packages labels Jun 5, 2024
@michael-s-molina michael-s-molina marked this pull request as ready for review June 5, 2024 13:32
@michael-s-molina michael-s-molina requested a review from a team as a code owner June 5, 2024 13:32
@michael-s-molina michael-s-molina removed the risk:db-migration PRs that require a DB migration label Jun 5, 2024
@dosubot dosubot bot added viz:charts:echarts Related to Echarts viz:charts:histogram Related to the Histogram chart labels Jun 5, 2024
@@ -25,8 +25,10 @@ export default function buildQuery(formData: HistogramFormData) {
return buildQueryContext(formData, baseQueryObject => [
{
...baseQueryObject,
extras: { where: `${column} IS NOT NULL` },
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that column can only contain numeric values, we automatically exclude NULL values from the result. This also aligns better with the limit control.

@@ -53,7 +53,7 @@ def histogram(
raise ValueError(f"The column '{column}' must be numeric.")

# calculate the histogram bin edges
bin_edges = np.histogram_bin_edges(df[column], bins=bins)
bin_edges = np.histogram_bin_edges(df[column].dropna(), bins=bins)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the dropna() to deal with NULL values independently of the query results.

@github-actions github-actions bot added the risk:db-migration PRs that require a DB migration label Jun 5, 2024
columns: [...groupby, column],
post_processing: [histogramOperator(formData, baseQueryObject)],
metrics: undefined,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

metrics by default is [] but we need it to be undefined to avoid a GROUP BY.

Copy link

codecov bot commented Jun 5, 2024

Codecov Report

Attention: Patch coverage is 94.44444% with 1 line in your changes missing coverage. Please review.

Project coverage is 70.26%. Comparing base (76d897e) to head (85efa2c).
Report is 275 commits behind head on master.

Files Patch % Lines
superset/cli/viz_migrations.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #28780      +/-   ##
==========================================
+ Coverage   60.48%   70.26%   +9.77%     
==========================================
  Files        1931     1951      +20     
  Lines       76236    77607    +1371     
  Branches     8568     8756     +188     
==========================================
+ Hits        46114    54528    +8414     
+ Misses      28017    20954    -7063     
- Partials     2105     2125      +20     
Flag Coverage Δ
hive ?
javascript 57.84% <100.00%> (+0.13%) ⬆️
mysql 77.21% <35.29%> (?)
postgres 77.33% <35.29%> (?)
presto 53.53% <35.29%> (-0.27%) ⬇️
python 83.55% <94.11%> (+20.07%) ⬆️
sqlite 76.77% <35.29%> (?)
unit 59.03% <94.11%> (+1.40%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@michael-s-molina michael-s-molina removed the risk:db-migration PRs that require a DB migration label Jun 5, 2024
Copy link
Member

@justinpark justinpark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@michael-s-molina michael-s-molina merged commit df0b1cb into apache:master Jun 5, 2024
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
packages plugins size/L viz:charts:echarts Related to Echarts viz:charts:histogram Related to the Histogram chart
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants