Is FV3_HRRR_gf only supported for smoke and dust?

[mkavulich]

If HRRR_gf is a fully supported suite, it should be usable outside of a smoke/dust context. But currently the FV3_HRRR_gf field and diag tables are hard-coded for smoke/dust rather than following the same naming conventions as other physics suites, and so regular cases can not use the HRRR_gf suite; workflow generation fails with the following error:

cp: cannot stat '/scratch1/BMC/hmtb/kavulich/UFS/workdir/test_develop/2025-02-22/intel/RRFS_sas/ufs-srweather-app/parm/field_table.FV3_HRRR_gf': No such file or directory
  File "/scratch1/BMC/hmtb/kavulich/UFS/workdir/test_develop/2025-02-22/intel/RRFS_sas/ufs-srweather-app/tests/WE2E/./run_WE2E_tests.py", line 554, in <module>
    run_we2e_tests(homedir,args)
  File "/scratch1/BMC/hmtb/kavulich/UFS/workdir/test_develop/2025-02-22/intel/RRFS_sas/ufs-srweather-app/tests/WE2E/./run_WE2E_tests.py", line 230, in run_we2e_tests
    expt_dir = generate_FV3LAM_wflow(ushdir,"config.yaml",logfile=f"{ushdir}/log.generate_FV3LAM_wflow",
  File "/scratch1/BMC/hmtb/kavulich/UFS/workdir/test_develop/2025-02-22/intel/RRFS_sas/ufs-srweather-app/tests/WE2E/../../ush/generate_FV3LAM_wflow.py", line 266, in generate_FV3LAM_wflow
    cp_vrfy(FIELD_TABLE_TMPL_FP, FIELD_TABLE_FP)
  File "/scratch1/BMC/hmtb/kavulich/UFS/workdir/test_develop/2025-02-22/intel/RRFS_sas/ufs-srweather-app/tests/WE2E/../../ush/python_utils/filesys_cmds_vrfy.py", line 32, in cp_vrfy
    return cmd_vrfy("cp", *args)
  File "/scratch1/BMC/hmtb/kavulich/UFS/workdir/test_develop/2025-02-22/intel/RRFS_sas/ufs-srweather-app/tests/WE2E/../../ush/python_utils/filesys_cmds_vrfy.py", line 20, in cmd_vrfy
    print_err_msg_exit(f"System call '{cmd}' failed.")
  File "/scratch1/BMC/hmtb/kavulich/UFS/workdir/test_develop/2025-02-22/intel/RRFS_sas/ufs-srweather-app/tests/WE2E/../../ush/python_utils/print_msg.py", line 18, in print_err_msg_exit
    traceback.print_stack(file=sys.stderr)
FATAL ERROR: System call 'cp /scratch1/BMC/hmtb/kavulich/UFS/workdir/test_develop/2025-02-22/intel/RRFS_sas/ufs-srweather-app/parm/field_table.FV3_HRRR_gf /scratch1/BMC/hmtb/kavulich/UFS/workdir/test_develop/2025-02-22/intel/RRFS_sas/expt_dirs/suite_namelists/grid_SUBCONUS_Ind_3km_ics_HRRR_lbcs_RAP_suite_HRRR_gf/field_table' failed.

It is possible to point to these files manually, but since these diag and field_tables contain smoke/dust hard-coded fields this will likely fail as well, or at least produce unexpected results.

As far as I'm aware, the only reason to have special diag_ and field_tables specifically for smoke and dust is due to extra fields needed for the smoke/dust tracers and other output. This is the case for UFS_FIRE as well, and this is handled with some simple logic in [ush/create_diag_table_file.py](https://github.com/ufs-community/ufs-srweather-app/blob/develop/ush/create_diag_table_file.py#L62) and [ush/generate_FV3LAM_wflow.py](https://github.com/ufs-community/ufs-srweather-app/blob/develop/ush/generate_FV3LAM_wflow.py#L458) rather than maintaining a separate set of files. Smoke and dust should utilize the same logic rather than keeping separate copies of these files.

All that said, I implemented these changes in a branch (develop...mkavulich:ufs-srweather-app:bugfix/allow_hrrr_gf_suite) and while the workflow generation completes successfully, the model now just segfaults; I assume because there are smoke/dust specific settings hard-coded in the parm/FV3.input.yml entry for FV3_HRRR_gf, but without being familiar with the smoke/dust capability I don't have the expertise to debug further.

All that is to say, is there a reason that the FV3_HRRR_gf suite is hard-coded exclusively for use with smoke/dust? Or is this just a quirk of the smoke/dust implementation?

[benkozi]

@mkavulich - Thank you for the attempted changes in the branch. I will see about reproducing the issue and how the hard-coded table data may be removed.

[benkozi]

@mkavulich - I was able to get a successful forecast with FV3_HRRR_gf using the test_community workflow. The extra entries in diag/field can be left as is (no need to append for smoke/dust only - just rename). Also, these task overrides were required:

task_make_ics:
  LEVP: 66
  VCOORD_FILE: "{{ workflow.FIXam }}/global_hyblev_fcst_rrfsL65.txt"
task_make_lbcs:
  LEVP: 66
  VCOORD_FILE: "{{ workflow.FIXam }}/global_hyblev_fcst_rrfsL65.txt"

config.yaml.txt

I'll see about getting a PR together to address this issue.

[mkavulich]

@benkozi those tables represent tracers that are initialized in the model, and a bunch of tracers can mean a large memory overhead in the model, so I'd really recommend using my suggested solution for only including them when necessary.

[benkozi]

@mkavulich - I was planning to try and incorporate your suggestions but wanted to pass along the minimal set of changes to run a successful forecast.