Proposal: Enhance Annotation Referencing in SDMX-JSON Data Messages

[hoehrmann]

Introduction:

Currently, SDMX-JSON data messages allow referencing annotations attached to components (like dimensions) only by their index within the $.data.structures[*].annotations[*] array. While this approach offers space and memory efficiency for frequently used annotations, it hinders readability for component-specific annotations. This proposal advocates for introducing an optional annotation object alongside the existing index-based referencing system.

Problem:

The current design necessitates using tools to link a component's annotation reference back to the actual annotation within the annotations array. This significantly impedes human readability of SDMX-JSON data messages, especially for those containing numerous annotations or when dealing with component-specific annotations.

Proposed Solution:

This proposal recommends introducing an optional annotation object alongside the existing index-based referencing system anywhere under $.data.structures. This object would directly hold the annotation details within the component definition itself.

Benefits:

• Improved Readability: Users can directly access component-specific annotations without relying on external tools, enhancing human comprehension of the data message.
• Flexibility: The proposal offers a choice between the existing efficient referencing and the new approach for better readability, catering to different use cases.
• Backward Compatibility: Existing data messages remain valid as the index-based referencing continues to function.

Additional Rationale:

Consider it from this perspective: $.data.structures is almost a schema language for non-SDMX legacy .csv files. There are existing workflows involving Spreadsheet and/or CSV templates to support conversion into SDMX. With very little effort, it would be possible to provide an alternative to such workflows using the SDMX-JSON $.data.structures format with the actual data held in non-SDMX CSV files. The $.data.structures part of such a workflow would likely be maintained manually, but having to reference annotations indirectly under $.data.structures makes that significantly more difficult.

(In doubt, please handle this as a public review comment on SDMX 3.1 once the comment period begins.)