SDMX-CSV Data

From FMR Knowledge Base
Revision as of 06:20, 28 July 2020 by Mnelson (talk | contribs) (Formatting Using Query Parameters)
Jump to navigation Jump to search


Overview

The SDMX-CSV Data format is an official SDMX format described on the SDMX GitHub Repository.

It can be used as both an import and export format for the Fusion Registry, and an export format for the Fusion Edge Server and Fusion Data Browser.

The format is comma separated format where each row describes a single Observation value. The columns describe each component of the Observation value, reflecting the Components in the Data Structure Definition.

The dataset can contain both Code labels, in addition to the Code IDs. The order of the columns is not important, however each column header will contain the ID of the corresponding Component. The first column must be called Dataflow where each value contains the URN postfix of the Dataflow that dataset is for.

Formatting Using Query Parameters

The following URL parameters can be used in a RESTful query for Fusion-CSV data.

  • format = sdmx-csv
  • labels = id | name | both | (id is default)
  • timeFormat = normalized
  • bom = include | exclude (Include or Exclude the Byte Order Mark (BOM).
    The BOM helps Excel interpret non Latin characters when opening a CSV file)

The output type "all-lang" will output both id and name and will add an element to the start of each row which will be the language. For each series and each language a row will be output where the name in the row is for the specified language. If there is no name value for that language, the name is simply not output. See below for an example output/

Normalized Time If the parameter value is normalized then the TIME_PERIOD values are converted to the most granular ISO 8601 representation taking into account the highest frequency of the data in the message

Example https://demo.metadatatechnology.com/FusionRegistry/ws/public/sdmxapi/rest/data/WB,GCI,1.0/GHA.GCI..?format=csv&labels=both&delimiter=tab

Note: The same formatting can be applied using HTTP Accept Headers as opposed to query parameters.

Example

An example query using the format request parameters, HTTP Accept Headers can also be used to define the same format.
https://demo.metadatatechnology.com/FusionRegistry/ws/public/sdmxapi/rest/data/WB,GCI,1.0/GHA.GCI..?format=sdmx-csv&labels=both

An example dataset with IDs only, spaces have been added to this example to assist readability.

DATAFLOW,    REF_AREA, INDICATOR, SUB_INDICATOR, FREQ, TIME_PERIOD, OBS_VALUE
WB:GCI(1.0), GHA,      GCI,       RANK,          A,    2008,        102
WB:GCI(1.0), GHA,      GCI,       RANK,          A,    2009,        114
WB:GCI(1.0), GHA,      GCI,       RANK,          A,    2010,        114
WB:GCI(1.0), GHA,      GCI,       RANK,          A,    2011,        114
WB:GCI(1.0), GHA,      GCI,       RANK,          A,    2012,        103
WB:GCI(1.0), GHA,      GCI,       RANK,          A,    2013,        114
WB:GCI(1.0), GHA,      GCI,       RANK,          A,    2014,        111

The same dataset in SDMX-CSV with lables included.

DATAFLOW,                                  REF_AREA:Reference Area, INDICATOR:Indicator,               SUB_INDICATOR:Sub Indicator, FREQ:Frequency, TIME_PERIOD:Time period, OBS_VALUE:Observation
WB:GCI(1.0): Global Competitiveness Index, GHA: Ghana,              GCI: Global Competitiveness Index, RANK: Rank,                  A: Annual,      2008,                    102
WB:GCI(1.0): Global Competitiveness Index, GHA: Ghana,              GCI: Global Competitiveness Index, RANK: Rank,                  A: Annual,      2009,                    114
WB:GCI(1.0): Global Competitiveness Index, GHA: Ghana,              GCI: Global Competitiveness Index, RANK: Rank,                  A: Annual,      2010,                    114
WB:GCI(1.0): Global Competitiveness Index, GHA: Ghana,              GCI: Global Competitiveness Index, RANK: Rank,                  A: Annual,      2011,                    114
WB:GCI(1.0): Global Competitiveness Index, GHA: Ghana,              GCI: Global Competitiveness Index, RANK: Rank,                  A: Annual,      2012,                    103
WB:GCI(1.0): Global Competitiveness Index, GHA: Ghana,              GCI: Global Competitiveness Index, RANK: Rank,                  A: Annual,      2013,                    114
WB:GCI(1.0): Global Competitiveness Index, GHA: Ghana,              GCI: Global Competitiveness Index, RANK: Rank,                  A: Annual,      2014,                    111