Arm diags v4

[EmmyChengTao]

• Developed based on the branch "ARM_Diags_V3".

[chengzhuzhang]

@EmmyChengTao Hi Cheng, it looks like neither you and I merged Xiaojian's last Pull Request #29 . So I just merged it right now. Could you add a description summarize the new features in this Pull Request discussion?

[chengzhuzhang]

Are both strings actually variable names in the input netcdf files? are they pre-computed?

[EmmyChengTao]

Both strings refer to the name of the metrics (not the variable names), which will be shown on the corresponding html. The two-legged metrics show the covariance relationships between two variables. Please let me know if you would this to be exactly the variable names. Thank you!

[chengzhuzhang]

get_seasonal_3hr and get_seasonal are repeated in two new component. If they are identical, we can move them under utils, also I'm wondering if the existing climo function can provide the same capabilities. Do both functions to get seasonal mean from 3hr and daily datasets?

[EmmyChengTao]

@chengzhuzhang, I checked climo function. It differs from get_seasonal_3hr and get_seasonal, which outputs the diurnal cycle for each season. I rewrote the function, which merged get_seasonal_3hr and get_seasonal into one function, and moved them under utils. Please check get_diurnal_cycle_seasons under utils.py for details. Thank you!

[chengzhuzhang]

Hey Cheng, thank you for the PR! It looks good, I left a couple of comments, please let me know, i think we can address them together.

[EmmyChengTao]

@chengzhuzhang , please find a summary of the Pull Request #29 below:

• Specify the start/end years for the test models in basicparameter.py
• Add the aerosol activation metrics via /src/aerosol_activation.py
• Edit /src/annual_cycle_aci.py with filenames following the ARM standard
• Edit /src/annual_cycle_zt.py by 1) allowing the detection of temporal resolution of input data automatically; 2) change the contour levels to site-dependent
• Edit /src/create_htmls.py by creating htmls for newly generated metrics on aerosol-cloud interactions

[EmmyChengTao]

@chengzhuzhang, I just updated the arm_diags_v4 branch with your comments addressed. The major changes include:

• Merged "get_seaonal" and "get_seasonal_3hr" into one function named "get_diurnal_cycle_seasons", and moved the function under "utils.py".
• Fixed an issue in the "convection onset" metrics. Previously, the figures cannot be generated. Also, adjusted the setting of y-axis in the figures.

Please let me know if anything. Thank you for your time and review!

[chengzhuzhang]

@EmmyChengTao the changes look good to me. I tested on NERSC and the new features are run successfully. Just to confirm, the plots for PBL annual cycle and Terrestrial Cooupling Legs for models are not created, this is expected because the tested version of model doesn't have needed variables included, correct?

[chengzhuzhang]

Also we should probably upload the v4 technical doc to https://github.com/ARM-DOE/arm-gcm-diagnostics/tree/master/docs, also it would be nice to add Xiaojian's paper as a reference paper in READ.me.

At the same time, I will attempt to re-use the climo function to get diurnal-cycle.

Almost there for a new release!

[EmmyChengTao]

Hello Jill, Thanks for your review. This is correct. The PBL and soil moisture are not available in the models, so the corresponding metrics are not generated. One question, do you use the E3SM environment when testing the ARM-Diags on NERSC? See below: source /global/common/software/e3sm/anaconda_envs/load_latest_e3sm_unified_pm-cpu.sh Best, Cheng From: Jill Chengzhu Zhang ***@***.***> Date: Friday, September 13, 2024 at 3:14 PM To: ARM-DOE/arm-gcm-diagnostics ***@***.***> Cc: Tao, Cheng ***@***.***>, Mention ***@***.***> Subject: Re: [ARM-DOE/arm-gcm-diagnostics] Arm diags v4 (PR #31) @EmmyChengTao<https://urldefense.us/v3/__https:/github.com/EmmyChengTao__;!!G2kpM7uM-TzIFchu!28zsuzXkipZMY2wKxYB4JE5jxKrLhlAb1eR1T_1KJhIJ_B6SD3TyrXT0zuA6Fcy3IpCje2ZppY5AfLUvP98g_pI$> the changes look good to me. I tested on NERSC and the new features are run successfully. Just to confirm, the plots for PBL annual cycle and Terrestrial Cooupling Legs for models are not created, this is expected because the tested version of model doesn't have needed variables included, correct? — Reply to this email directly, view it on GitHub<https://urldefense.us/v3/__https:/github.com/ARM-DOE/arm-gcm-diagnostics/pull/31*issuecomment-2350468436__;Iw!!G2kpM7uM-TzIFchu!28zsuzXkipZMY2wKxYB4JE5jxKrLhlAb1eR1T_1KJhIJ_B6SD3TyrXT0zuA6Fcy3IpCje2ZppY5AfLUvI8FJ4zw$>, or unsubscribe<https://urldefense.us/v3/__https:/github.com/notifications/unsubscribe-auth/AMSQQKOT3HVWH3FTPRUNO6DZWNPS5AVCNFSM6AAAAABMU6N5ACVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNJQGQ3DQNBTGY__;!!G2kpM7uM-TzIFchu!28zsuzXkipZMY2wKxYB4JE5jxKrLhlAb1eR1T_1KJhIJ_B6SD3TyrXT0zuA6Fcy3IpCje2ZppY5AfLUvWwaYKH4$>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[EmmyChengTao]

Thanks for the suggestions, Jill! The technical report for ARM-Diags v4 is currently under review by the ARM publication teams. I will upload it to the GitHub when it is finalized. And, sure, I will add Xiaojian’s paper as a reference paper in READ.me. Thank you for working on the climo function. Let me know if you need anything from me. Have a great weekend! Best, Cheng From: Jill Chengzhu Zhang ***@***.***> Date: Friday, September 13, 2024 at 3:16 PM To: ARM-DOE/arm-gcm-diagnostics ***@***.***> Cc: Tao, Cheng ***@***.***>, Mention ***@***.***> Subject: Re: [ARM-DOE/arm-gcm-diagnostics] Arm diags v4 (PR #31) Also we should probably upload the v4 technical doc to https://github.com/ARM-DOE/arm-gcm-diagnostics/tree/master/docs<https://urldefense.us/v3/__https:/github.com/ARM-DOE/arm-gcm-diagnostics/tree/master/docs__;!!G2kpM7uM-TzIFchu!3KYnpa23xepE9rOHyziNLc7vhMdwrx2QAaIOB2WzWL3G_nKUxiTIR3JZ3FlzUleSaQNkvfBVOUP7Ya-bv1U6Yqc$>, also it would be nice to add Xiaojian's paper as a reference paper in READ.me. At the same time, I will attempt to re-use the climo function to get diurnal-cycle. Almost there for a new release! — Reply to this email directly, view it on GitHub<https://urldefense.us/v3/__https:/github.com/ARM-DOE/arm-gcm-diagnostics/pull/31*issuecomment-2350479792__;Iw!!G2kpM7uM-TzIFchu!3KYnpa23xepE9rOHyziNLc7vhMdwrx2QAaIOB2WzWL3G_nKUxiTIR3JZ3FlzUleSaQNkvfBVOUP7Ya-b8CmGvso$>, or unsubscribe<https://urldefense.us/v3/__https:/github.com/notifications/unsubscribe-auth/AMSQQKN4BUZT3Y7IFL46MW3ZWNP43AVCNFSM6AAAAABMU6N5ACVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNJQGQ3TSNZZGI__;!!G2kpM7uM-TzIFchu!3KYnpa23xepE9rOHyziNLc7vhMdwrx2QAaIOB2WzWL3G_nKUxiTIR3JZ3FlzUleSaQNkvfBVOUP7Ya-bCFtfMmY$>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[chengzhuzhang]

@EmmyChengTao yes, I used the e3sm_unified env on perlmutter to test.

[chengzhuzhang]

@EmmyChengTao I think this PR is ready to merge. Though I don't see a commit that updates the README, could it be that the commit is not pushed to remote?