Specifying Begin/End for cwms-data/timeseries results in System Error

[andyvan-trabus]

Specifying a valid BEGIN value in a timeseries request results in a System Error

I made a request for timeseries data for JPLT2.Flow-Out.Inst.1Hour.0.Rev-SWF-REGI with no BEGIN/END values, and received a response:

Request
curl -X 'GET'
'https://cwms-data.usace.army.mil/cwms-data/timeseries?name=JPLT2.Flow-Out.Inst.1Hour.0.Rev-SWF-REGI&office=SWF'
-H 'accept: application/json;version=2'

https://cwms-data.usace.army.mil/cwms-data/timeseries?name=JPLT2.Flow-Out.Inst.1Hour.0.Rev-SWF-REGI&office=SWF

Response
{ "begin": "2022-12-05T17:42:00+0000[UTC]", "end": "2022-12-06T17:42:00+0000[UTC]", "interval": "PT0S", "interval-offset": 0, "name": "JPLT2.Flow-Out.Inst.1Hour.0.Rev-SWF-REGI", "office-id": "SWF", "page": "MTY3MDI2MzIwMDAwMHx8MjR8fDUwMA==", "page-size": 500, "time-zone": "US/Central", "total": 24, "units": "cfs", "value-columns": [ { "name": "date-time", "ordinal": 1, "datatype": "java.sql.Timestamp" }, { "name": "value", "ordinal": 2, "datatype": "java.lang.Double" }, { "name": "quality-code", "ordinal": 3, "datatype": "int" } ], "values": [ [ 1670263200000, 16, 0 ],

I then substituted the returned BEGIN and END values into the request, and I received System Error. It even does this if I just substitute in the BEGIN value, and let END default. I tried this both with and without the optional timezone. I also tried the example BEGIN and END values from the Swagger UI page with the same result:

Request
curl -X 'GET'
'https://cwms-data.usace.army.mil/cwms-data/timeseries?name=JPLT2.Flow-Out.Inst.1Hour.0.Rev-SWF-REGI&office=SWF&begin=2022-12-05T17%3A42%3A00%2B0000%5BUTC%5D'
-H 'accept: application/json;version=2'

https://cwms-data.usace.army.mil/cwms-data/timeseries?name=JPLT2.Flow-Out.Inst.1Hour.0.Rev-SWF-REGI&office=SWF&begin=2022-12-05T17%3A42%3A00%2B0000%5BUTC%5D

Response
{
"message": "System Error",
"incidentIdentifier": "4288797622609029867",
"details": {}
}

[MikeNeilson]

Well, somethings up with the time handling. This worked: curl -X 'GET' 'https://cwms-data.usace.army.mil/cwms-data/timeseries?name=JPLT2.Flow-Out.Inst.1Hour.0.Rev-SWF-REGI&office=SWF&begin=2022-12-05T17:00:00.000-00:00' -H 'accept: application/json;version=2'

Additionally you can use the ISO 8601 relative times. something like begin=PT-3D or begin=PT-6H.

But I'll look into the string you provided. https://cwms-data.usace.army.mil/cwms-data/timeseries?name=JPLT2.Flow-`begin=2022-12-05T17:42:00+0000[UTC]` should be valid but we've had some issues.

[andyvan-trabus]

Apparently you need the ":" in the timezone offset:

"-0000" fails
"-00:00" works

[andyvan-trabus]

Also, there's a problem with the timezone stuff.

2021-06-10T13:00:00-07:00: Works
2021-06-10T13:00:00-07:00[PST8PDT] does NOT work

[MikeNeilson]

Oh right, that particular one we knew about and haven't gotten to it yet. All those forms are intended to work but we aren't planning to deploy any additional changes until the 19th. Caused some problems the last time so going a bit slower, but I'll try to make sure the above is fixed in the next release.

[andyvan-trabus]

Not a problem, as long as you update the examples in the Swagger UI (and elsewhere) to be forms that work.

[rma-rripken]

We should verify but I remember there was an issue where the legacy pl/sql used one format and CDA used another. Perhaps we need to document "for application/json;version=1 use datetimes like.... for application/json;version=2 use ...

We probably either close the issue or add integration tests that verify which formats work/break and then document all the end-points with examples.

We could make an html page sort of like the Regex page and then add a link to if for all the controllers that use DateUtils.
Its really just the legacy pl/sql where we need to verify the formats.

[MikeNeilson]

We discussed this again recently and I realized we should change the parameter definition of begin and end to use the OpenApi
"oneOf" system to help identify all the valid possibilities and render them correctly to the user.

This would be especially useful for the situtation of the deprecated formats as we can call it out specifically but the basic support is

• ISO 8601 date-time (theortically, we know there's a few issues)
• ISO 8601 intervals
• ISO 8601 durations

The last two especially aren't obvious as due to the way the swagger ui renders a date.