Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix using the forcing file header error. #624

Merged
merged 2 commits into from
Sep 5, 2023

Conversation

stcui007
Copy link
Contributor

@stcui007 stcui007 commented Aug 31, 2023

The details are described in Issue #504. Briefly, when a forcing file with the following format:
Time,RAINRATE,T2D,Q2D,U2D,V2D,PSFC,SWDOWN,LWDOWN
2015-12-01 00:00:00,0,285.8000183,0.0093,-2.299999952,0.100000001,100310,0,361.3000183
is used, ngen throw a runtime " out of range" error.
The usually used header is as follows.
time,APCP_surface,DLWRF_surface,DSWRF_surface,PRES_surface,SPFH_2maboveground,TMP_2maboveground,UGRD_10maboveground,VGRD_10maboveground,precip_rate
2015-12-01 00:00:00,0.0,360.8000183105469,0.0,100220.0,0.009399999864399433,285.8999938964844,-2.299999952316284,0.10000000149011612,0.0

Additions

Removals

Changes

include/realizations/catchment/Bmi_Multi_Formulation.hpp

Testing

Run ngen test

Screenshots

Notes

It is found the error was caused by setting init_time to 0 in CsvPerFeatureForcingProvider.hpp. This was ultimately traced to get_data_start_time() which call get_variable_time_begin(const std::string &variable_name) in Bmi_Multi_Formulation.hpp. This function uses availableData to extract the start time. If var_name in availableData is not in forcing data, then it returns 0 as start time. The solution makes sure if a var_name chosen from availableData is not a forcing variable name, we use the first forcing variable name as var_name to choose start time, which is always a valid time. We could actually use any of the forcing variable names to choose the start time, here we just uses the first one for convenience.

Also note that the fix works for both CSV and netCDF data.

Todos

Checklist

  • [x ] PR has an informative and human-readable title
  • [x ] Changes are limited to a single goal (no scope creep)
  • [ x] Code can be automatically merged (no conflicts)
  • [ x] Code follows project standards (link if applicable)
  • [x ] Passes all existing automated tests
  • [x ] Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged / future todos are captured in comments
  • Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)
  • [x ] Reviewers requested with the Reviewers tool ➡️

Testing checklist (automated report can be put here)

Target Environment support

  • [x ] Linux

@PhilMiller
Copy link
Contributor

Please fill in a description of the problem and its resolution in the PR notes, and correct the checklist formatting.

@mattw-nws mattw-nws linked an issue Sep 1, 2023 that may be closed by this pull request
@@ -245,6 +245,10 @@ namespace realization {
for(std::map<std::string,std::shared_ptr<data_access::GenericDataProvider>>::iterator iter = availableData.begin(); iter != availableData.end(); ++iter)
{
var_name = iter->first;
auto it = std::find(forcing->get_available_variable_names().begin(), forcing->get_available_variable_names().end(), var_name);
if (it == forcing->get_available_variable_names().end()) {
var_name = forcing->get_available_variable_names()[0];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The whole loop now boils down to this line. Just eliminate the for loop and set var_name to this... even the try block below is probably not useful now, because (I'm pretty sure) if availableData[var_name]->get_data_start_time(); fails on a forcing var_name, something has gone wrong (in the current code, at least).

Actually... I think the whole use of a variable name is redundant in this code path (where var_name == "*" || var_name = ""... I think this whole section can be reduced to:

// Find one that successfully returns...
if(var_name == "*" || var_name == ""){
    return forcing->get_data_start_time();
}
// If not found ...

Can you try that?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually the comment above that if is now not really correct either, probably just make it // If no var_name, use forcing... if the above works.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just tried. It seems to work.

@mattw-nws mattw-nws marked this pull request as ready for review September 5, 2023 14:56
@mattw-nws mattw-nws merged commit 8ae0100 into NOAA-OWP:master Sep 5, 2023
19 checks passed
@mattw-nws mattw-nws mentioned this pull request Sep 5, 2023
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Odd forcing file format failure when using SWDOWN+LWDOWN
3 participants