Complete Examples of Evaluation Declarations

WRES performs an evaluation following instructions provided to it by a user through an evaluation project declaration file, as described in the Declaration language. Provided herein are examples of project declarations; instructions to execute those examples either through the NWC-hosted Central OWP WRES (COWRES) or locally using a release package or built software; a link to the current project declaration schema; and supporting information.

Executing the Examples Through the Central OWP WRES Web Interface

The COWRES is a WRES web service hosted at the National Water Center (NWC). For those with access to that service, any example below can be executed with paths for sources modified following guidance available in the user support wiki for the COWRES. Contact the WRES team if you need more information.

Executing the Examples Locally

Instructions for executing the examples locally, from either an installed release package or locally built software, are provided in the table, below, and in the YAML example declarations following the links. Data will need to be downloaded and the paths for sources modified to point to the downloaded location. See the table, below, for more information.

NOTES:

• The data supporting examples 1, 2, and 3, are currently unavailable to the public while we investigate how best to share that data.
• Example 5 can only be executed on machines on the NWC network.

Examples

To understand how to declare evaluations for the WRES, we have provided the following examples. In each case, the project declaration file that is linked is annotated with comments explaining the different aspects of the declaration.

#	Description	Link
1	Ensemble forecasts evaluated against observations, all in PI-timeseries XML format with the forecasts provided in a gzipped tarball. Included is a baseline of ESP forecasts used to calculate skill scores. This makes use of one year of an HEFS Baseline Validation data set.	WRES Project Declaration Example 1
2	Single-valued, operational stream flow forecasts evaluated against observations provided in an NWS datacard file. It includes a comparison against persistence forecasts generated from the same datacard observations. The evaluation is performed at a 24-hour, mean desired time scale with lead times pooled into windows that are adjacent, non-overlapping, and 24-hours in width. Due to a restriction on the lead times to those in the first 48 hours and a time offset necessary to align the observations and forecasts at a 24-hour time scale, there is only one lead time in the evaluation output.	WRES Project Declaration Example 2
3	Evaluation of operational stage forecasts with corresponding observations through a contingency table created for various stage thresholds and probability classifiers indicating when an ensemble forecast is assumed to have forecast a threshold to be exceeded. It is an example illustrating providing thresholds to support metric computation, as well as temporally rescaling the data to 4-day maximums (peak stages).	WRES Project Declaration Example 3
4	Evaluation of single valued forecasts against observations through a time series analysis including the metrics “time to peak error” and “time to peak relative error”. The data backing this evaluation is available in the repository at https://github.com/NOAA-OWP/wres/tree/master/systests/smalldata. Clone the repository or download the data, and modify the declaration accordingly.	WRES Project Declaration Example 4
5	Example of the use of the data sources USGS NWIS, WRDS-served NWM channel routing NetCDF output files, the WRDS AHPS forecast data services, and the WRDS location service. Execution of this examples requires access to NWC-hosted WRDS services; the WRDS URL must be known and the example modified to point to those services.	WRES Project Declaration Example 5

Declaration Schema

Additionally, the schema of the YAML in which the above examples are provided, including descriptions of various options (more descriptions will be added in the future), can be found here:

https://vlab.noaa.gov/redmine/projects/wres/repository/revisions/master/entry/wres-config/nonsrc/schema.yml

This can be used as guidance for identifying all of the different options available as part of a WRES evaluation project declaration. For further information, see Declaration Language.

Support

If support is needed at any time, please post a GitHub issue ticket, https://github.com/NOAA-OWP/wres/issues/new, and we will address it as soon as possible.