-
Notifications
You must be signed in to change notification settings - Fork 7
SDMX JSON format prototype issues
SDMX RESTful web API includes the concept of a series key. The key parameter used to identify resources in data queries is a series key and one of the options for the detail parameter is "serieskeysonly". If the JSON format treats all dimensions equally then this could cause problems for the client applications.
Response message could provide additional metadata that allows client applications to uniquely identify time dimensions or any other dimension that is not part of the series key. Example:
"REF_AREA": {
"label": "Reference area",
"seriesKey": true
},
"TIME_PERIOD": {
"label": "Time period",
"seriesKey": false
}
Alternatively time dimension could be added to the key parameter and then time dimension would not need any special treatment in the client application. Example:
GET /data/M.DE+FR.X.000000.X.INX.2011-12+2012-01
However this would be a change to the current SDMX 2.1 RESTful web API.
Solution for the "serieskeysonly" value for the detail parameter could be more difficult. Web API could return "501 Not Implemented" status code if this value is requested together with JSON format.
The RESTful web API treats time dimension differently from other dimensions. Queries for the dimensions in the series key are specified in the key parameter. e.g. BE+DE+FR. Queries for the time dimension are specified in the additional parameters startDate and endDate.
Client applications would need to include quite sophisticated time intelligence in order to perform correct queries (e.g. sorting time periods by the starting date). Also the time dimension cannot be displayed to a user in the same way as other dimensions, e.g. a simple list that allows multiple selections will not work. Support for time intelligence would add to the development effort for client applications.
Response message could provide additional metadata for the codes in the time dimension, e.g. begin and end dates. Client application would then not need to perform similar calculations for the time periods. Response message should also include metadata that allows client applications to uniquely identify time dimensions. Example:
"TIME_PERIOD": {
"label": "Time period",
"role": "time",
"category": {
"label": {
"2012": "2012"
"2012-W01": "2012-W01"
},
"startDate": {
"2012": "2012-01-01T00:00:00",
"2012-W01": "2012-01-02T00:00:00"
}
"endDate": {
"2012": "2012-12-31T23:59:59",
"2012-W01: "2012-01-08T23:59:59"
}
}
}
SDMX Web Services Guidelines specifies that "RESTful Web Services should indicate errors using the proper HTTP status code. In addition, whenever appropriate, the error should also be returned using the error message". For JSON responses the error messages should also be in JSON format. There are no standards for error messages in JSON itself so the error message format should be developed specifically for SDMX. Example:
"error": {
"en": [ "Mandatory parameter flowRef is missing from URL." ]
}
SDMX XML formats include standard headers with message specific metadata. Headers could also be useful also for the JSON format. Are all the standard header fields needed for the data visualization use case and if not should they be included any way for compatibility? Example:
{
"id": "Quarterly BoP reporting",
"test": false,
"prepared": "2010-01-13T16:00:33",
"sender": {
"SDMX": {
"id": "SDMX",
"name": { "en": "SDMX.org" },
"contact": [
{ "name": { "en": "Mr Mark Smith" } },
{ "name": { "en": "Mr John Smith" },
"department": { "en": "IS/BoP" },
"telephone": "0049 69 13440"
}
]
}
},
"structure": {
"ECB_EXR1": {
"id": "ECB_EXR1",
"ref": {
"id": "ECB_EXR",
"agencyID": "ECB",
"version": "1.0"
}
}
}
}
SDMX Web Service Guidelines specifies the SDMX-ML as the default format for the response message. This means that if a client wants to receive JSON it must use HTTP content negotiation. The standard mime type for JSON is application/json. Guideline specifies custom mime types for SDMX-ML responses. Is there a need to have similar mime types for JSON? If so what would they be? Examples:
application/vnd.sdmx.data+json;version=2.1
application/vnd.sdmx.genericdata+json;version=2.1
application/vnd.sdmx.structurespecificdata+json;version=2.1
What are the benefits and drawbacks of different code structures (data content is same):
# Objects with code as key
"code": {
"id": [
"M",
"Q",
"A"
],
"name": {
"M": "Monthly",
"A": "Annual",
"Q": "Quarterly"
},
}
# Objects with field name as key
"code": [
{
"id": "M",
"name": "Monthly"
},
{
"id": "Q",
"name": "Quarterly"
},
{
"id": "A",
"name": "Annual"
}
]
Do we need a role field for dimensions and attributes (maybe also measures?)? Does it have to be separate field or just a reserved name (different languages?)?
"dimension": {
"TIME_PERIOD": {
"name": "Time period",
"role": "time",
"code": {
# codes etc.
}
}
SDMX 2.1 defines four attribute relationship types:
- None
- Dimension / AttachmentGroup
- Group
- PrimaryMeasure
Are "Group" and "AttachmentGroup" absolutely necessary or would it be simpler to just have "Dimension"? Schema documentation indicates that "Group" is used as a convenience. Examples:
"attribute": {
"OBS_STATUS": {
"relationship": "PrimaryMeasure"
},
"UNIT": {
"relationship": [
"REF_AREA",
"ADJUSTMENT",
"ICP_ITEM",
"STS_INSTITUTION",
"ICP_SUFFIX"
]
},
"COPYRIGHT": {
"relationship": null
}
}
SDMX 2.1 DataStructure contains element "MeasureList". It contains single element "PrimaryMeasure": "PrimaryMeasure defines the structure of the primary measure, which is the concept that is the value of the phenomenon to be measured in a data set. Although this may take its semantic from any concept, this is provided a fixed identifier (OBS_VALUE) so that it may be easily distinguished in data messages." How to name the fields in JSON? Examples:
# With measurelist
{
"measureList": {
"id": [
"OBS_VALUE"
]
"OBS_VALUE": {
# Fields for OBS_VALUE
}
}
}
# Without measurelist
{
"primaryMeasure": {
"id": "OBS_VALUE"
# Other fields for OBS_VALUE
}
}