Curtain plot (model) with Observations overlaid as scatter & Updated Violin Plot with bigger label fonts and ticks consistent with other plots #254

quaz115 · 2024-03-26T19:32:30Z

… other plots in aircraftplots

rschwant

Can you fix the plot title to be based on text_kwargs too. To make it look nice I also had the tickparams a bit smaller than the title and x/y labels at 0.8*. You can look at make_boxplot subroutine in surfplots.py for how to do this.

quaz115 · 2024-04-09T15:38:44Z

Can you fix the plot title to be based on text_kwargs too. To make it look nice I also had the tickparams a bit smaller than the title and x/y labels at 0.8*. You can look at make_boxplot subroutine in surfplots.py for how to do this.

Made this fix in the make_violin_plot() in aircraftplots.py (see 5f32335)

… each model-obs pair like scatter plots

…ring words such as, Parameters & Returns

quaz115 · 2024-04-16T05:54:43Z

Curtain plot added:

The interpolate_and_clean_model_data function is responsible for preparing the model data for visualization, which involves creating a meshgrid from the time and altitude data, interpolating over any gaps, and cleaning the dataset by replacing any non-numerical values with appropriate limits. The make_curtain_plot function takes this prepared data and produces the actual visualization, plotting the model data as a contour plot (a curtain plot) and overlaying observational data as a scatter plot on top of this. The function uses the matplotlib library for plotting, allowing for a custom (color_map_custom as True) or predefined colormap (color_map_custom as False), and ensures the plot is correctly normalized using the minimum and maximum data values. The appearance of the plot(s) (e.g., labels, titles) are customized using the parameters specified in the YAML configuration.

Currently, it prints for each model-obs pair as done for scatter plots earlier. For example:


Processing curtain plot for model 'wrfchem_v4.2' and observation 'firexaq'...

Saving Curtain plot (Model) with Scatter Overlay (Observation) to /wrk/qrasool/output_testaircraft_PR189_uptoviolin_multiplemodels/plot_grp5.curtain.O3_CL_RYERSON.2019-09-05_12.2019-09-06_00.all.CONUS_firexaq_vs_wrfchem_v4.2.png...

quaz115 · 2024-04-16T13:29:09Z

Also @rschwant the test data used here is for FIREX 2019-09-05, I think model results might be a bit old for this: /wrk/qrasool/firex_mech_qzhu/run_CONUS_fv19_BEIS_1.0xISO_RACM_v4.2.2_racm_berk_vcp_noI_phot_soa/0905/* (on Meiyu)

quaz115 · 2024-06-11T23:27:36Z

@colin-harkins @rschwant For the latest commits to fix curtain plot, i am able to generate model curtain plot (pressure as y-axis and time as x-axis) with model scatter as overlay aligned with obs scatter (in the second subplot). Curtain and scatter profiles all share common colorbar (vmin and vmax can be user-defined or dynamically set based on the data min max) for each variable set: An example here: a) using default Spectral_r cmap:

  b) using user-defined custom_color_map in YAML set to True:

rschwant

Great job, pulling in a very complicated plot! I had a couple questions still. I'll be in the office tomorrow if you want to talk about my comments. If you can just address these this week that works well we can work on the spatial plot another time.

examples/yaml/control_wrfchem_aircraft_Latestfor_develop_aircraft_testmultiplemodelgroups.yaml

melodies_monet/driver.py

melodies_monet/plots/aircraftplots.py

examples/yaml/control_wrfchem_aircraft_Latestfor_develop_aircraft_testmultiplemodelgroups.yaml

melodies_monet/plots/aircraftplots.py

rschwant · 2024-07-15T21:30:41Z

Also I just noticed can you make sure the vmin and vmax for the pressure on the y-axis for both plots are the same so it is easier to compare them?

…aircraftplots

…ir separately as png now

quaz115 · 2024-07-24T22:00:22Z

Also I just noticed can you make sure the vmin and vmax for the pressure on the y-axis for both plots are the same so it is easier to compare them?

Lines 373-375 in aircraftplots.py now addresses this issue: # Set the same y-tick labels for both subplots and exclude the minimum value
y_ticks = axs[0].get_yticks()
y_lim = axs[0].get_ylim()
axs[1].set_yticks(y_ticks)
axs[1].set_ylim(y_lim)

examples/yaml/control_wrfchem_aircraft_Latestfor_develop_aircraft_testmultiplemodelgroups.yaml

melodies_monet/driver.py

…stom is used.

…color_map True statement in aircratplots more explicit, colobar minima not forced to be zero now

…y list in YAML for different variables plotted

…ot types, Also in aircraftplots make_curtain_plot() now fontsize, fontweight and labelsize all linked via YAML

quaz115 · 2024-08-01T03:29:37Z

Latest output example (note how changed y-axis target pressure levels (y-axis) to 500 via YAML now): Pressure MIN:<xarray.DataArray 'pressure_model' ()> Size: 8B
array(5582.80372111), max: <xarray.DataArray 'pressure_model' ()> Size: 8B
array(101160.20015191), interval: 500
Saving curtain plot to /wrk/qrasool/output_testaircraft_PR189_uptoviolin_multiplemodels_racmesrlvcpFTPnew/plot_grp5.curtain.O3_CL_RYERSON.2019-09-05_12.2019-09-06_00.all.CONUS_firexaq_vs_wrfchem_v4.2.png...
time shape: (1901,), ndims: 1
altitude shape: (500,), ndims: 1
model_data_2d shape: (1901, 500), ndims: 2
pairdf shape: (1901, 648), ndims: 2
cmin and cmax set dynamically based on model and observation data to 0 and 100
Saving curtain plot to /wrk/qrasool/output_testaircraft_PR189_uptoviolin_multiplemodels_racmesrlvcpFTPnew/plot_grp5.curtain.O3_CL_RYERSON.2019-09-05_12.2019-09-06_00.all.CONUS_firexaq_vs_wrfchem_v4.2.png...

rschwant

Just had a couple small comments to fix for the vmin, vmax, cmin, and cmax variables and everything else looks great! Thanks for updating everything. Once these things are fixed, I can fully approve and we can merge this in.

melodies_monet/driver.py

rschwant · 2024-09-09T15:27:49Z

examples/yaml/control_wrfchem_aircraft_Latestfor_develop_aircraft_testmultiplemodelgroups.yaml

+        "#343434", "#606060", "#8c8c8c", "#b8b8b8", "#e4e4e4"] #,
+        #"#ffffff"] #['blue','royalblue', 'cyan', 'yellow', 'orange', 'red'] ##['#440154', '#3b528b', '#21908d', '#5dc963', '#fde725'] #['purple', 'blue', 'green', 'yellow'] # Example gradient
+    color_levels: 30 #31 #15 #10  # Define the number of distinct colors in the color bar
+    #color_map: 'Spectral_r' #'jet'  # This can be set instead of color_map_custom, color_levels and color_levels to set a colormap defined in matplotlob.  


Should one of the color_levels here be colors instead in the comment?

rschwant · 2024-09-09T22:02:49Z

melodies_monet/driver.py

+        from .util.tools import resample_stratify
+        from .util.tools import vert_interp
+        import matplotlib.dates as mdates
+        import numpy as np


Do you use all of these? I looks like numpy is already pulled in at the top. mdates is not used and vert_interp is not used.

melodies_monet/driver.py

rschwant · 2024-09-09T22:29:20Z

examples/yaml/control_wrfchem_aircraft_Latestfor_develop_aircraft_testmultiplemodelgroups.yaml

+    #color_map: 'Spectral_r' #'jet'  # This can be set instead of color_map_custom, color_levels and color_levels to set a colormap defined in matplotlob.  
+    vmin: 120000 #Pressure in Pa  #0  #Optional (y-axis limits)
+    vmax: 5000 # Optional #need to be edited as per min and max altitude (i.e., vmin and vmax)
+    interval : 500 #Pressure in Pa; target pressure interpolation steps for model curtain 2d plot


It does not look like these vmin and vmax values are used in the code right now. Do you want to correct this or just take this option out?

rschwant · 2024-09-09T22:38:37Z

melodies_monet/plots/aircraftplots.py

+
+    if vmin is not None and vmax is not None:
+        plot_dict['ylim'] = [vmin, vmax]
+


The vmin and vmax values from your yaml file are not linked to the vmin and vmax values used here and this code is not used later on either. So likely should delete this line. And fix this connection like specified above.

colin-harkins

I think once you make the changes Becky requested this looks good. I'll approve on my end.

colin-harkins · 2024-09-13T21:02:33Z

I thought this would take two approvals to be able to merge so hold off on merging until you address Becky's comments but I'll leave as is with the approval on my end.

… limits, Tested with unit change Pa to hPa, cleaned up codes and yaml

quaz115 · 2024-09-17T23:06:16Z

@rschwant @colin-harkins Tested with fixes for previous comments on y-axes and colorbar labels. Tested with changing units of pressure (pressure_units via yaml, distinguished between 'intervals' as y-axis tick intervals and 'num_levels' as interpolation levels for target pressures as well). Updated single model yaml as well with curtain plot config options. An example result with customized y-axis limits working well:

However getting this CI/Check docs build error, which I don't think is related to my changes.

colin-harkins · 2024-09-18T15:23:35Z

It looks like there are broken links in the supported datasets page and the developers guide page on the readthedocs.

(develop/developers_guide: line  175) broken    https://github.com/NOAA-CSL/MELODIES-MONET/projects/2 - 404 Client Error: Not Found for url: https://github.com/NOAA-CSL/MELODIES-MONET/projects/2
(users_guide/supported_plots: line    8) broken    https://github.com/NOAA-CSL/MELODIES-MONET/projects/3 - 404 Client Error: Not Found for url: https://github.com/NOAA-CSL/MELODIES-MONET/projects/3
(users_guide/supported_datasets: line   58) redirect  https://github.com/NOAA-CSL/MELODIES-MONET/projects/4 - permanently to https://github.com/orgs/NOAA-CSL/projects/7
(users_guide/supported_datasets: line   33) ok        https://aeronet.gsfc.nasa.gov/
(users_guide/supported_datasets: line   34) redirect  http://vista.cira.colostate.edu/Improve/ - permanently to https://vista.cira.colostate.edu/Improve/
(users_guide/supported_datasets: line   64) broken    https://github.com/NOAA-CSL/MELODIES-MONET/projects/6 - 404 Client Error: Not Found for url: https://github.com/NOAA-CSL/MELODIES-MONET/projects/6
(users_guide/supported_datasets: line   44) broken    https://github.com/NOAA-CSL/MELODIES-MONET/projects/7 - 404 Client Error: Not Found for url: https://github.com/NOAA-CSL/MELODIES-MONET/projects/7
(users_guide/supported_datasets: line   23) broken    https://github.com/NOAA-CSL/MELODIES-MONET/projects/8 - 404 Client Error: Not Found for url: https://github.com/NOAA-CSL/MELODIES-MONET/projects/8

I think potentially this is related to some restructuring of the project board link? The project board that exists now seems to be: https://github.com/orgs/NOAA-CSL/projects/6

I'm not sure if this is related to something we changed or github changed about the project board. @rschwant do you know?

Regardless, hopefully it shouldn't be too hard to find new links for these items in the project board.

rschwant · 2024-09-18T15:26:06Z

Yes, I asked Quazi to update these links. Github got rid of their classic project boards, but luckily we had already migrated to one project board, so we just need to update the ReadTheDocs for this.

quaz115 · 2024-09-18T16:58:58Z

@colin-harkins i did try to resolve conflicts but not sure why they are still visible on the PR?

…ject board link with text edit works

…NET project board link with text edit works

…ODIES-MONET project board link with text edit works" This reverts commit 79ca49e.

…ed content

rschwant

Looks great!

Updated Violin Plot with bigger label fonts and ticks consistent with…

Loading
Loading status checks…

c6a452c

… other plots in aircraftplots

quaz115 requested review from rschwant and colin-harkins March 26, 2024 19:32

rschwant reviewed Apr 4, 2024

View reviewed changes

Violin plot title and ticklabel fontsizes fixed

Loading
Loading status checks…

5f32335

quaz115 requested a review from rschwant April 9, 2024 15:43

Curtain pLot of Model with Observations scatter overlay: separate for…

Loading
Loading status checks…

7b87cda

… each model-obs pair like scatter plots

quaz115 changed the title ~~Updated Violin Plot with bigger label fonts and ticks consistent with other plots~~ Curtain plot (model) with Observations overlaid as scatter & Updated Violin Plot with bigger label fonts and ticks consistent with other plots Apr 16, 2024

Resolve Sphinx-Build CI error by adding hyphen(-) equal to the doctst…

Loading
Loading status checks…

684d8a1

…ring words such as, Parameters & Returns

Fix Sphinx-Build related CI error for interpolate function docstring

Loading
Loading status checks…

7edacd8

Subplots: 1) Model Curtain with Model scatter overlay and 2) Obs scatter

Loading
Loading status checks…

0285d6a

rschwant reviewed Jul 15, 2024

View reviewed changes

quaz115 added 2 commits July 22, 2024 18:06

Some Fixes for curtain plot YAML and other minor fixes in driver and …

Loading
Loading status checks…

de7aff9

…aircraftplots

Saving plots for scatter and curtain plots for different model-obs pa…

Loading
Loading status checks…

21b8937

…ir separately as png now

colin-harkins reviewed Jul 25, 2024

View reviewed changes

Comment color_map in yaml out because it's not used when color_map_cu…

Loading
Loading status checks…

7f1814c

…stom is used.

rschwant mentioned this pull request Jul 29, 2024

Expand plotting routine to include more specific aircraft plots #200

Closed

quaz115 added 3 commits July 30, 2024 14:07

Some redundancies related to interpolation removed in driver, custom_…

Loading
Loading status checks…

160c1bb

…color_map True statement in aircratplots more explicit, colobar minima not forced to be zero now

colobar min and max now taken from vmin_plot ad vmax_plot from obs ke…

Loading
Loading status checks…

9681061

…y list in YAML for different variables plotted

Model and Obs Scatter overlay now plotted using pairdf as in other pl…

Loading
Loading status checks…

59cbbfe

…ot types, Also in aircraftplots make_curtain_plot() now fontsize, fontweight and labelsize all linked via YAML

removal of parts not really used such as, model_data and obs_data

Loading
Loading status checks…

a1bd14d

quaz115 requested review from rschwant and colin-harkins August 1, 2024 04:10

rschwant reviewed Sep 9, 2024

View reviewed changes

colin-harkins approved these changes Sep 13, 2024

View reviewed changes

quaz115 added 2 commits September 17, 2024 16:49

Resolved issues remaining with yaxis (altitude variable ie. pressure)…

Loading
Loading status checks…

0ba4156

… limits, Tested with unit change Pa to hPa, cleaned up codes and yaml

Added curtain plot options to the single model vs obs yaml as well

Loading
Loading status checks…

b9ba0ef

quaz115 force-pushed the develop branch from 86cd8ee to b9ba0ef Compare September 18, 2024 18:46

docs/develop/developers_guide.rst Edits for fixing MELODIES-MONET pro…

Loading
Loading status checks…

d530379

…ject board link with text edit works

quaz115 force-pushed the develop branch from 7babc8d to d530379 Compare September 18, 2024 19:06

quaz115 and others added 6 commits September 18, 2024 13:10

TEST docs/background/supported_plots.rst Edits for fixing MELODIES-MO…

79ca49e

…NET project board link with text edit works

Revert "TEST docs/background/supported_plots.rst Edits for fixing MEL…

Loading
Loading status checks…

efbfe90

…ODIES-MONET project board link with text edit works" This reverts commit 79ca49e.

Renamed docs/background to docs/users_guide

0c97781

Merge branch 'develop' into develop

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

GPG key ID: B5690EEEBB952194

Verified
Learn about vigilant mode

Loading
Loading status checks…

cb3f058

Renamed supported_analyses.rst to supported_diagnostics.rst and updat…

51cbf21

…ed content

Resolved merge conflict for supported_diagnostics.rst

Loading
Loading status checks…

ec39a3e

rschwant approved these changes Sep 18, 2024

View reviewed changes

quaz115 merged commit a558c3f into NOAA-CSL:develop Sep 18, 2024
4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Curtain plot (model) with Observations overlaid as scatter & Updated Violin Plot with bigger label fonts and ticks consistent with other plots #254

Curtain plot (model) with Observations overlaid as scatter & Updated Violin Plot with bigger label fonts and ticks consistent with other plots #254

quaz115 commented Mar 26, 2024

rschwant left a comment

quaz115 commented Apr 9, 2024

quaz115 commented Apr 16, 2024 •

edited

Loading

quaz115 commented Apr 16, 2024

quaz115 commented Jun 11, 2024

rschwant left a comment

rschwant commented Jul 15, 2024

quaz115 commented Jul 24, 2024

quaz115 commented Aug 1, 2024

rschwant left a comment

rschwant Sep 9, 2024

rschwant Sep 9, 2024

rschwant Sep 9, 2024

rschwant Sep 9, 2024

colin-harkins left a comment

colin-harkins commented Sep 13, 2024

quaz115 commented Sep 17, 2024

colin-harkins commented Sep 18, 2024

rschwant commented Sep 18, 2024

quaz115 commented Sep 18, 2024

rschwant left a comment


		if vmin is not None and vmax is not None:
		plot_dict['ylim'] = [vmin, vmax]

Curtain plot (model) with Observations overlaid as scatter & Updated Violin Plot with bigger label fonts and ticks consistent with other plots #254

Curtain plot (model) with Observations overlaid as scatter & Updated Violin Plot with bigger label fonts and ticks consistent with other plots #254

Conversation

quaz115 commented Mar 26, 2024

rschwant left a comment

Choose a reason for hiding this comment

quaz115 commented Apr 9, 2024

quaz115 commented Apr 16, 2024 • edited Loading

quaz115 commented Apr 16, 2024

quaz115 commented Jun 11, 2024

rschwant left a comment

Choose a reason for hiding this comment

rschwant commented Jul 15, 2024

quaz115 commented Jul 24, 2024

quaz115 commented Aug 1, 2024

rschwant left a comment

Choose a reason for hiding this comment

rschwant Sep 9, 2024

Choose a reason for hiding this comment

rschwant Sep 9, 2024

Choose a reason for hiding this comment

rschwant Sep 9, 2024

Choose a reason for hiding this comment

rschwant Sep 9, 2024

Choose a reason for hiding this comment

colin-harkins left a comment

Choose a reason for hiding this comment

colin-harkins commented Sep 13, 2024

quaz115 commented Sep 17, 2024

colin-harkins commented Sep 18, 2024

rschwant commented Sep 18, 2024

quaz115 commented Sep 18, 2024

rschwant left a comment

Choose a reason for hiding this comment

quaz115 commented Apr 16, 2024 •

edited

Loading