Skip to content

Instructions for Programmatic Interaction with a WRES Web service

Jump to bottom
HankHerr-NOAA edited this page Jul 12, 2024 · 13 revisions

Calls to a WRES web-service instance can be made from the command-line, using curl for example, and by “machine” users, such as scripts, other software, and so on. General instructions are provided below for how to interact with a WRES web-service in order to post an evaluation, monitor for success, and get the outputs afterwards. Examples of such interactions are provided in the following scripts:

When reading the General Instructions below, it is recommended you follow along in an appropriate example, above, in order to understand each step.

Using the fully functional bash and Python scripts, above, is recommended

The scripts noted above are described with examples in the WRES Scripts Usage Guide wiki. It is recommended that you use these scripts when interacting with a WRES web-service programmatically. However, if you need to make the HTTP requests directly, perhaps because you are using another programming language, then the below sections describe how to do so.

What if I want to post data directly to the WRES web-service to support my evaluation?

If your evaluation requires posting data directly to a WRES web-service as input for the evaluation, then the instructions below still apply, but additional actions are required in Steps 1 and 3, and are described in Posting timeseries data directly to COWRES as inputs for a WRES job.

In general, how do I interact programmatically with a WRES web-service?

Regardless of language, the general instructions are as follows (see the examples above for illustrations of each step):

1. Prepare the evaluation project declaration.

2. Obtain the latest TLS/SSL certificate authority .pem for authenticating WRES web-service.

This is required to authenticate the WRES web-service instance and will be referred to as the "web-service CA .pem" file in the instructions below. Obtain the web-service CA .pem file from the web-service administrator.

3. “POST” the evaluation project declaration to the web-service using the web-service CA .pem file and record the server’s response.

By POSTing an evaluation project declaration, we are requesting the creation of an evaluation job on the web-server. The server will create the job and we can “GET” information about the job. Furthermore, the server’s response to the POST must be recorded, because it contains the result of the response and the URL location of the job. Upon posting, an example response (the WRES web-service URL has been omitted, replacing it with "...", in this example):

HTTP/1.1 100 Continue

HTTP/1.1 201 Created
Date: Fri, 05 Jun 2020 02:59:57 GMT
Location: https://.../job/1139901177795399825
Content-Type: text/html
Content-Length: 244
Server: Jetty(9.4.29.v20200521)

<!DOCTYPE html><html><head><title>Evaluation job received.</title></head><body><h1>Evaluation job 1139901177795399825 has been received for processing.</h1><p>See <a href="/job/1139901177795399825">/job/1139901177795399825</a></p></body></html>

4. Check whether the POST was successful by looking for HTTP code 200 or 201. If not, then the POST command failed.

That HTTP code is reported in the example HTTP response, above, by the line, “HTTP/1.1 201 Created”. The code is 201, which denotes a success.

5. Identify the job location URL.

That location is reported in the HTTP response by the Location: output line. In the example response above, the job location is,

https://.../job/113990117779539982

The number at the end is referred to as the jobId, while the entire URL is referred to as the job location URL.

6. Monitor the status of the job.

That status is reported in the “status” subdirectory of the job location URL. Using the example response above, the status can be found at the URL,

https://.../job/113990117779539982/status

While the job is waiting to be processed, the status will be, “IN_QUEUE”. While the job is still running, the status will be, “IN_PROGRESS”. When the job has completed, that status will be either “COMPLETED_REPORTED_SUCCESS”, “COMPLETED_REPORTED_FAILURE”, or “NOT_FOUND” (e.g., corresponding to a job that is no longer available). Evaluation output is only available if the status is “COMPLETED_REPORTED_SUCCESS”.

A diagram of the possible states for an evaluation, including those related to posting data directly to the COWRES, is provided (here)POSTING TIME SERIES DIRECTLY WIKI!!!.

7. Immediately after the evaluation, obtain the evaluation outputs.

Upon “COMPLETED_REPORTED_SUCCESS”, the output will be available underneath of the “output” folder below the job location URL for a short time. Using the example above, the output would be under,

https://.../job/113990117779539982/output

Immediately after the evaluation, download all files within that folder.

8. Clean up after yourself by deleting the output from the COWRES.

All files beneath the output folder must be removed to free up those resources. This is done by posting a "DELETE" request to the output endpoint in the above step.

The WRES Wiki

Home

Clone this wiki locally