Difference between revisions of "Codelist V10"

From FMR Knowledge Base
Jump to navigation Jump to search
(Codelists with Simple Hierarchies)
(Structure-Level Validity)
 
(50 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
+
[[Category:SDMX 2.1 Structures]]
[[Category:SDMX Structures]]
 
 
=Overview=
 
=Overview=
 
An SDMX Codelist is a managed list of classification codes.
 
An SDMX Codelist is a managed list of classification codes.
  
 +
Codelists can be referenced directly by [[Data Structure Definition V10|Data Structure Definitions]], or indirectly through [[Concepts V10|Concepts]] to explicitly define the closed set of values for enumerated [[Dimension|Dimensions]], [[Attribute|Attributes]] or [[Measure V10|Measures]].
 +
 +
==Structure Properties==
 
{| class="wikitable"
 
{| class="wikitable"
| Structure Type|| Standard SDMX Structural Metadata Artefact
+
! scope=row style="text-align: left;" | Structure Type  
 +
| Standard SDMX Structural Metadata Artefact
 +
|-
 +
! scope=row style="text-align: left;"  | Maintainable
 +
| [[Maintainable_V10|Yes]]
 
|-
 
|-
| Maintainable|| Yes
+
! scope=row style="text-align: left;"  | Identifiable
 +
| [[Identifiable V10|Yes]]
 
|-
 
|-
| Identifiable|| Yes
+
! scope=row style="text-align: left;"  | Item Scheme
 +
| [[Item_Scheme|Yes]]
 
|-
 
|-
| Item Scheme|| Yes
+
! scope=row style="text-align: left;"  | SDMX Information Model Versions
 +
| 2.0, 2.1
 
|-
 
|-
| SDMX Information Model Versions || 1.0, 2.0, 2.1
+
! scope=row style="text-align: left;"  | URN - Codelist namespace
 +
| <nowiki>urn:sdmx:org.sdmx.infomodel.codelist.Codelist</nowiki>
 
|-
 
|-
| Concept ID || CODELIST
+
! scope=row style="text-align: left;"  | URN - Code namespace
 +
| <nowiki>urn:sdmx:org.sdmx.infomodel.codelist.Code</nowiki>
 
|}
 
|}
==Codelists Context within the SDMX 2.1 Information Model==
+
 
::[[File:SDMX Information Model - Core Artefacts - Codelist.png|600px|frameless]]
+
==Context within the SDMX 2.1 Information Model==
The schematic illustrates the core artefacts of the SDMX 2.1 Information Model.<br>
+
 
Codelists can be referenced directly by Data Structure Definitions, or indirectly through Concepts to describe the list of valid values for enumerated Dimensions or Attributes.
+
[[File:Code1.png|Codelist|600px]]
  
 
==Usage==
 
==Usage==
<p>SDMX Codelists are lists of classification codes used principally for defining the set of allowed values for enumerated [[Components]] in [[Data Structure Definition|Data Structure Definitions (DSDs)]] or [[Metadata Structure Definition|Metadata Structure Definitions]] which describe [[Reference Metadata]].</p>
+
<p>SDMX Codelists are lists of classification codes used principally for defining the set of allowed values for enumerated [[Component|Components]] in [[Data Structure Definition V10|Data Structure Definitions (DSDs)]] or [[Metadata Structure Definition|Metadata Structure Definitions]] which describe [[Reference Metadata]].</p>
 +
 
 +
Each code is a separate Item so must have an ID and a Name, but can also have an optional Description. While Code IDs must be unique within a Codelist, the same Code ID may be safely used in other Codelists. For instance: The code 'A' may be used in a <em>Frequency Codelist</em> to represent 'Annual', but also appear in an <em>Industry Codelist</em> to represent 'Agriculture'.<br>
 +
 
 +
=Conventions=
 +
<p><strong>'CL_' Codelist ID Prefix</strong></p>
 +
<p>
 +
Codelist IDs are given a 'CL_' prefix to distinguish them from other structures. For instance: CL_FREQ, CL_REF_AREA, CL_AGE.
 +
</p>
  
Each code is a separate [[Item]] so must have an ID and a Name, but can also have an optional Description. While Code IDs must be unique within a Codelist, the same Code ID may be safely used in other Codelists. For instance: The code 'A' may be used in a <em>Frequency Codelist</em> to represent 'Annual', but also appear in an <em>Industry Codelist</em> to represent 'Agriculture'.<br>
+
<p><strong>Code ID Conventions</strong></p>
 +
<p>
 +
Codes can take any legal SDMX ID. But there are several conventions that should be taken into account when choosing Code IDs:<br>
 +
 
 +
{| class="wikitable"
 +
|-
 +
| <strong>Uppercase</strong> || By convention, Code ID's are in uppercase such as 'ABC'. Lower case Codes are valid (for example 'abc'), but care should be taken in their use to avoid confusion.
 +
|-
 +
| <strong>'_Z' Code</strong> || The '_Z' code is conventionally used for the Undefined and Unknown classification.
 +
|-
 +
| <strong>'TOTAL' Code</strong> || The 'TOTAL' code represents the total or sum of the dimension. For a 'country' dimension, series with TOTAL would indicate the sum of observation values for all countries.
 +
|}
 +
</p>
 +
[[URN V10|You can seem more examples and information on Identities in this article]].
  
 
=Flat Codelists=
 
=Flat Codelists=
Line 50: Line 82:
 
|-
 
|-
 
| N || Minutely
 
| N || Minutely
 +
|}
 +
 +
<p> Example: CL_ADJUSTMENT Codelist with descriptions</p>
 +
{| class="wikitable"
 +
|-
 +
! Code ID !! Code Name !! Code Description
 +
|-
 +
| K || Calendar component || Synonyms: Calendar effects; calendar factors
 +
|-
 +
| X || Seasonal component || Synonyms: Seasonal effects; seasonal factors
 +
|-
 +
| M || Seasonal and calendar components || Synonyms: Seasonal and calendar effects; seasonal and calendar factors
 +
|-
 +
| I || Irregular component || Synonym: Irregular effects
 +
|-
 +
| N || Neither seasonally adjusted nor calendar adjusted data || Synonyms: Raw data; unadjusted data
 +
|}
 +
 +
<p> Example: CL_ADJUSTMENT Codelist with multi-lingual names and descriptions</p>
 +
{| class="wikitable"
 +
|-
 +
! Code ID !! Locale || Code Name !! Code Description
 +
|-
 +
| K || en || Calendar component || Synonyms: Calendar effects; calendar factors
 +
|-
 +
| K || fr || Composant de calendrier || Synonymes: effets de calendrier; facteurs de calendrier
 +
|-
 +
| X || en || Seasonal component || Synonyms: Seasonal effects; seasonal factors
 +
|-
 +
| X || fr || Composant saisonnier || Synonymes: effets saisonniers; facteurs saisonniers
 +
|-
 +
| M || en || Seasonal and calendar components || Synonyms: Seasonal and calendar effects; seasonal and calendar factors
 +
|-
 +
| M || fr || Composants saisonniers et calendaires || Synonymes: effets saisonniers et calendaires; facteurs saisonniers et calendaires
 +
|-
 +
| I || en || Irregular component || Synonym: Irregular effects
 +
|-
 +
| I || fr || Composant irrégulier || Synonyme: effets irréguliers
 +
|-
 +
| N || en || Neither seasonally adjusted nor calendar adjusted data || Synonyms: Raw data; unadjusted data
 +
|-
 +
| N || fr || Ni données désaisonnalisées ni données désaisonnalisées || Synonymes: données brutes; données non ajustées
 
|}
 
|}
  
Line 80: Line 154:
  
 
<p>Other than acting as a parent, EUR behaves as a normal code allowing series for EUR or any of the individual contries.</p>
 
<p>Other than acting as a parent, EUR behaves as a normal code allowing series for EUR or any of the individual contries.</p>
<p>A Code can only have one parent meaning that it can only appear once in a simple Codelist hierarchy. This may be restrictive where Codes can be logically organised into multiple different groups. In the CL_REF_AREA example, countries could also be grouped into trading blocs in addition to geographical regions. SDMX [[Hierarchical Codelist|Hierarchical Codelists]] can be used for this purpose.</p>
+
<p>A Code can only have one parent meaning that it can only appear once in a simple Codelist hierarchy. This may be restrictive where Codes can be logically organised into multiple different groups. In the CL_REF_AREA example, countries could also be grouped into trading blocs in addition to geographical regions.</p>
 +
 
 +
<p>SDMX [[Hierarchical Codelist|Hierarchical Codelists]] should be used where Codelists have more complex hierarchical structures.</p>
 +
 
 +
=Hierarchical Codelists=
 +
[[Hierarchical Codelist|Hierarchical Codelists]] are a separate structure used for describing complex hierarchies of codes, potentially including codes from multiple different codelists.
 +
 
 +
=Validity=
 +
==Structure-Level Validity==
 +
<p>As a [[Maintainable V10|Maintainable]] structure, the period in time during which a complete Codelist is considered to be valid can be defined.</p>
 +
 
 +
==Item-Level Validity==
 +
<p>Validity periods can also be defined individually for each Code item in a Codelist. A common use case involves country codes that are only valid between particular dates when the country was in ‘existence’. It also supports the case where a code changes meaning over time.</p>
 +
 
 +
<p>Example:<br>
 +
The Gilbert and Ellice Islands had the ISO country code ‘GE’ until 1977 when split to become the independent countries of Kiribati and Tuvalu. When Georgia gained independence from the former Soviet Union in 1991, the ‘GE’ code was reused. Setting the Item Validity on the GE Code allows this change in meaning to be described and stored with the Codelist.</p>
  
=Conventions=
+
<p>Period up to 1977: GE = Gilbert and Ellice Islands<br>
==CL_ Codelist ID Prefix==
+
Period between 1977 and 1991: GE is invalid<br>
Codelist IDs are given a 'CL_' prefix to distinguish them from other structures. For instance: CL_FREQ, CL_REF_AREA, CL_AGE.
+
Period from 1991 to present: GE = Georgia
==Code ID Conventions==
+
</p>
Codes can take any legal SDMX [[SDMX ID|ID]]. But there are several conventions that should be taken into account when choosing Code IDs:<br>
 
  
{|
+
<p>In practical terms, Item Validity allows data with dimension codes that vary in meaning over time to be robustly validated and correctly disseminated.</p>
| <strong>Uppercase Code IDs</strong> || By convention, Code ID's are in uppercase such as 'ABC'. Lower case Codes are valid (for example 'abc'), but care should be taken in their use to avoid confusion.
 
|-
 
| <strong>'_Z' Code</strong> || The '_Z' code is conventionally used for the Undefined and Unknown classification.
 
|-
 
| <strong>'TOTAL' Code</strong> || The 'TOTAL' code represents the total or sum of the dimension. For a 'country' dimension, series with TOTAL would indicate the sum of observation values for all countries.
 
|}
 
  
=Examples=
 
==Simple Flat Codelist==
 
{| class="wikitable"
 
|-
 
! Code ID !! Code Name
 
|-
 
| A || Annual
 
|-
 
| S || Half-yearly, semester
 
|-
 
| Q || Quarterly
 
|-
 
| M || Monthly
 
|-
 
| W || Weekly
 
|-
 
| D || Daily
 
|-
 
| B || Daily - Business Week
 
|-
 
| N || Minutely
 
|}
 
==Flat Codelist with Optional Code Descriptions==
 
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
! Code ID !! Code Name !! Code Description
+
! Use Case !! Example
|-
 
| K || Calendar component || Synonyms: Calendar effects; calendar factors
 
 
|-
 
|-
| X || Seasonal component || Synonyms: Seasonal effects; seasonal factors
+
| Validation || Observation data for country ‘GE’ and the time period ‘1970’ must relate to the Gilbert and Ellice islands. Whereas observations for the same ‘GE’ code but for the period 2005 must relate to Georgia. Observations for country ‘GE’ during the period 1978 and 1990 will be rejected as invalid because the validity rule states that no country with that code existed between those dates.
 
|-
 
|-
| M || Seasonal and calendar components || Synonyms: Seasonal and calendar effects; seasonal and calendar factors
+
| Dissemination|| Visualisations showing observation data for country ‘GE’ and time period ‘1970’ will use the code named ‘Gilbert and Ellice’. When showing similar data for ‘2005’, the Item Validity rules will mean that the code named ‘Georgia’ will be displayed.
|-
 
| I || Irregular component || Synonym: Irregular effects
 
|-
 
| N || Neither seasonally adjusted nor calendar adjusted data || Synonyms: Raw data; unadjusted data
 
 
|}
 
|}

Latest revision as of 00:34, 30 August 2022

Overview

An SDMX Codelist is a managed list of classification codes.

Codelists can be referenced directly by Data Structure Definitions, or indirectly through Concepts to explicitly define the closed set of values for enumerated Dimensions, Attributes or Measures.

Structure Properties

Structure Type Standard SDMX Structural Metadata Artefact
Maintainable Yes
Identifiable Yes
Item Scheme Yes
SDMX Information Model Versions 2.0, 2.1
URN - Codelist namespace urn:sdmx:org.sdmx.infomodel.codelist.Codelist
URN - Code namespace urn:sdmx:org.sdmx.infomodel.codelist.Code

Context within the SDMX 2.1 Information Model

Codelist

Usage

SDMX Codelists are lists of classification codes used principally for defining the set of allowed values for enumerated Components in Data Structure Definitions (DSDs) or Metadata Structure Definitions which describe Reference Metadata.

Each code is a separate Item so must have an ID and a Name, but can also have an optional Description. While Code IDs must be unique within a Codelist, the same Code ID may be safely used in other Codelists. For instance: The code 'A' may be used in a Frequency Codelist to represent 'Annual', but also appear in an Industry Codelist to represent 'Agriculture'.

Conventions

'CL_' Codelist ID Prefix

Codelist IDs are given a 'CL_' prefix to distinguish them from other structures. For instance: CL_FREQ, CL_REF_AREA, CL_AGE.

Code ID Conventions

Codes can take any legal SDMX ID. But there are several conventions that should be taken into account when choosing Code IDs:

Uppercase By convention, Code ID's are in uppercase such as 'ABC'. Lower case Codes are valid (for example 'abc'), but care should be taken in their use to avoid confusion.
'_Z' Code The '_Z' code is conventionally used for the Undefined and Unknown classification.
'TOTAL' Code The 'TOTAL' code represents the total or sum of the dimension. For a 'country' dimension, series with TOTAL would indicate the sum of observation values for all countries.

You can seem more examples and information on Identities in this article.

Flat Codelists

Flat Codelists are simple lists of codes with no explicit or implied relationships or hierarchies.

Example: CL_FREQUENCY Codelist

Code ID Code Name
A Annual
S Half-yearly, semester
Q Quarterly
M Monthly
W Weekly
D Daily
B Daily - Business Week
N Minutely

Example: CL_ADJUSTMENT Codelist with descriptions

Code ID Code Name Code Description
K Calendar component Synonyms: Calendar effects; calendar factors
X Seasonal component Synonyms: Seasonal effects; seasonal factors
M Seasonal and calendar components Synonyms: Seasonal and calendar effects; seasonal and calendar factors
I Irregular component Synonym: Irregular effects
N Neither seasonally adjusted nor calendar adjusted data Synonyms: Raw data; unadjusted data

Example: CL_ADJUSTMENT Codelist with multi-lingual names and descriptions

Code ID Locale Code Name Code Description
K en Calendar component Synonyms: Calendar effects; calendar factors
K fr Composant de calendrier Synonymes: effets de calendrier; facteurs de calendrier
X en Seasonal component Synonyms: Seasonal effects; seasonal factors
X fr Composant saisonnier Synonymes: effets saisonniers; facteurs saisonniers
M en Seasonal and calendar components Synonyms: Seasonal and calendar effects; seasonal and calendar factors
M fr Composants saisonniers et calendaires Synonymes: effets saisonniers et calendaires; facteurs saisonniers et calendaires
I en Irregular component Synonym: Irregular effects
I fr Composant irrégulier Synonyme: effets irréguliers
N en Neither seasonally adjusted nor calendar adjusted data Synonyms: Raw data; unadjusted data
N fr Ni données désaisonnalisées ni données désaisonnalisées Synonymes: données brutes; données non ajustées

Codelists with Simple Hierarchies

SDMX allows simple hierarchies to be defined within flat Codelists by making a code the parent of codes that logically sit under it in the hierarchy.

Imagine a CL_REF_AREA Codelist containing individual codes for each European country, and a code (EUR) for Europe as a whole. A simple hierarchy for Europe can be created by setting EUR as the parent for each of the countries:

Code ID Code Name Parent
EUR Europe (none)
DE Germany EUR
FR France EUR
IT Italy EUR
GR Greece EUR
SE Sweden EUR
AU Austria EUR
PL Poland EUR

Other than acting as a parent, EUR behaves as a normal code allowing series for EUR or any of the individual contries.

A Code can only have one parent meaning that it can only appear once in a simple Codelist hierarchy. This may be restrictive where Codes can be logically organised into multiple different groups. In the CL_REF_AREA example, countries could also be grouped into trading blocs in addition to geographical regions.

SDMX Hierarchical Codelists should be used where Codelists have more complex hierarchical structures.

Hierarchical Codelists

Hierarchical Codelists are a separate structure used for describing complex hierarchies of codes, potentially including codes from multiple different codelists.

Validity

Structure-Level Validity

As a Maintainable structure, the period in time during which a complete Codelist is considered to be valid can be defined.

Item-Level Validity

Validity periods can also be defined individually for each Code item in a Codelist. A common use case involves country codes that are only valid between particular dates when the country was in ‘existence’. It also supports the case where a code changes meaning over time.

Example:
The Gilbert and Ellice Islands had the ISO country code ‘GE’ until 1977 when split to become the independent countries of Kiribati and Tuvalu. When Georgia gained independence from the former Soviet Union in 1991, the ‘GE’ code was reused. Setting the Item Validity on the GE Code allows this change in meaning to be described and stored with the Codelist.

Period up to 1977: GE = Gilbert and Ellice Islands
Period between 1977 and 1991: GE is invalid
Period from 1991 to present: GE = Georgia

In practical terms, Item Validity allows data with dimension codes that vary in meaning over time to be robustly validated and correctly disseminated.

Use Case Example
Validation Observation data for country ‘GE’ and the time period ‘1970’ must relate to the Gilbert and Ellice islands. Whereas observations for the same ‘GE’ code but for the period 2005 must relate to Georgia. Observations for country ‘GE’ during the period 1978 and 1990 will be rejected as invalid because the validity rule states that no country with that code existed between those dates.
Dissemination Visualisations showing observation data for country ‘GE’ and time period ‘1970’ will use the code named ‘Gilbert and Ellice’. When showing similar data for ‘2005’, the Item Validity rules will mean that the code named ‘Georgia’ will be displayed.