FMR Knowledge Base - User contributions [en]

EDI Alternate Half Year

2025-11-03T13:01:31Z

Vmurrell:

[[Category:How_To]]
[[Category:How_To V11]]

Core and Local Representations in a Data Structure

2025-08-06T11:17:39Z

Vmurrell:

[[Category:How_To V11]]

Within a Data Structure a component (such as a Dimension) is attached to an underlying concept. This concept can be coded and so the component "inherits" the representation from the concept constraining the values that may be reported for this component. This is the "core representation". The core representation can be over-ridden by a "local representation" of either a Codelist or a "Text Format". A "Text Format" has a type (e.g. String, Integer, DateTime, etc.) and may include further restrictions such as a minimum or maximum length. The rules of inheritance conform to the SDMX 3 standard and constrain the component as follows:

* If there is a Core Representation, but no Local representation or Text Format, the Core Representation constrains the component.
* If there is a Core Representation, and a Local representation but no Text Format, the Local Representation constrains the component.
* If there is a Core Representation, no Local representation but there is a Text Format, the Text Format constrains the component.
* If there is a Core Representation, Local representation and Text Format, the Local Representation constrains the component and the Text Format is for informational purposes only

The User Interface to define this relationship has changed in version 11.21.0 of the Fusion Metadata Registry. This information is displayed on Step 3 of the DSD wizard and now looks like the following:

::[[File:DSD-Wizard-Step3-Example.png|Fusion Metadata Registry|1000px]]

The example above shows a DSD with 4 dimensions, 1 attribute and 1 measure. The Dimension "FREQ" is constrained by the Codelist WB:CL_FREQ_WDI(1.0) and has no local representation. The dimensions "SERIES" inherits from the codelist WB:CL_SERIES_WDI(1.0), but has a local Text Format of String. This overrides the codelist (shown by it being struck-through) and any content is permitted. Similarly the dimension "REF_AREA" is also over-ridden by a String, but this is further restricted (which is shown by the blue 'Details' label).

Main Page

2024-05-13T15:40:19Z

Vmurrell: /* FusionXL Topics */

{{DISPLAYTITLE:{{FULLPAGENAME}}}}
__NOTOC__
Fusion Metadata Registry (FMR) Knowledge Base 
::[[File:FMR11 screenshot.png|Fusion Metadata Registry|500px]]

Fusion Metadata Registry is a free to use structural metadata registry for SDMX supporting EDI, SDMX 1.0, 2.0, 2.1 and 3.0.

Read more about Fusion Metadata Registry on the [https://www.sdmx.io/tools/fmr/#sdmx-structural-metadata-registry sdmx.io web site]

To download Fusion Metadata Registry [https://www.sdmx.io/resources/download/fmr/ please click here.]

= Tools =
The [https://www.sdmx.io/tools/ecosystem/ sdmx.io ecosystem] includes a collection of open source SDMX software tools cooperating to solve official statistics use cases.

This knowledgebase provides further information on how to install, configure and use FMR and the tools:

* Click here to read more about [https://www.sdmx.io/tools/ecosystem/#fusionxl FusionXL on the sdmx.io site.]
* Click here to read more about [https://www.sdmx.io/tools/ecosystem/#fusion-transformer Fusion Transformer on the sdmx.io site.]
* Click here to read more about [https://www.sdmx.io/tools/ecosystem/#fusion-workbench Workbench on the sdmx.io site.]
* See below to read more about how to use [https://fmrwiki.sdmxcloud.org/Backup_and_Restore FMR-CL, a command line application] which performs functions against the web services of a running FMR instance.

==Fusion Metadata Registry Topics==
SDMX statistical data engine and structural metadata registry. The main statistical engine for data collection, integration, processing, storage, query execution and time-series calculations. It provides an SDMX-compliant data and metadata REST API. The integrated SDMX structural metadata registry acts as the central repository, authoring and maintenance tool for all of the structures including Codelists, Concepts and Data Structure Definitions.

{| class="wikitable"
|-
| [[:Fusion_Metadata_Overview|'''Overview''']]|| What is Fusion Metadata Registry?
|-
| [[:Category:Installation_and_Configuration |'''Installation and Configuration''']]|| How to install and configure the Registry
|-
| [[:Category:Concepts_Reference_V11|'''Concepts Reference''']]|| Explanation of key concepts and terms used in SDMX
|-
| [[:Category:SDMX 3.0 Structures|'''SDMX V3.0 Structures''']]|| Detailed explanations of the main SDMX structural metadata artefacts
|-
| [[:Category:Structural Metadata V11|'''Structure Maintenance''']]|| How to maintain the SDMX Structures
|-
| [[:Category:Reference_Metadata|'''Reference Metadata''']] || Guidance on working with SDMX 3.0 reference metadata in Fusion Registry 11
|-
| [[:Category:How_To V11|'''How to and other Features''']] || Guidance on specific tasks using Fusion Registry
|-
| [[:Category:SDMX 3.0 Mapping Structures|'''Mapping''']] || Mapping using Structure Maps and Representation Maps
|-
| [[:Category:FMR Formats Reference|'''Data Formats''']] || A description of supported Data Formats
|-
| [[:Category:RegistrySecurity|'''Security''']] || Authentication and Authorisation Features
|-
| [[:Category:FMR REST API Reference|'''Web Services''']] || Registry Web Services
|-
| [[Change_Log|'''Change Log''']] || Change Logs for Fusion Metadata Registry
|-
| [[:Category:Version 10 |'''Functionality Retired after Version 10''']] || Functionality no longer available in V11
|}

== FusionXL Topics ==
FusionXL is an addin for Microsoft Excel which connects to a Fusion Registry server for both data and structure retrieval, and maintenance.
{| class="wikitable"
|-
| [[FusionXL_install|'''Installation''']] || How to Install Fusion XL and connect to a Fusion Registry server
|-
| [[FusionXL_Create_Dataset|'''Creating Datasets''']] || Create datasets and load datasets for the purpose of data authoring, validation, and conversion
|-
| [[FusionXL_Author_Helper|'''Author Helper''']] || Provides assistance when creating datasets
|-
| [[Structure_Explorer|'''Structure Explorer''']] || Create, Retrieve and Modify Structures using the Excel Ribbon
|-
| [[:Category:Fusion_XL|'''Other Features''']] || Other features available in FXL
|-
| [[FusionXL_Change_Log|'''Change Logs''']] || Change Logs for FusionXL
|-
|}

== Fusion Transformer ==
[https://fmrwiki.sdmxcloud.org/Fusion_Transformer '''Fusion Transformer'''] is a command line application providing transformations between supported data files.The Fusion Transformer is a command line application providing transformations between supported data files.

== Fusion Workbench ==
The FMR Workbench (FWB) provides a web user interface (UI) for browsing and maintaining SDMX Structures obtained from external SDMX compliant Structure Web Services, including (and not limited to) - any FMR installation, OECD.STAT, public SDMX web services, such as the SDMX Global Registry,BIS,ECB,ILO,UNICEF.

{|
|-
| style="width:300px" | [[FMR_Workbench|FMR Workbench Getting Started]] || Getting started guide for the FMR Workbench
|}

== FMR-CL ==
'''F'''usion '''M'''etadata '''R'''egistry-'''C'''ommand '''L'''ine (FMR-CL) is a command line application which performs functions against the web services of a running FMR instance.

[https://fmrwiki.sdmxcloud.org/FMR-CL Overview]

==== Using FMR-CL ====
{|
|-
| style="width:300px" | [[Backup and Restore|Structure Backup]] || Supporting backup of structures and transaction history for restore in new installations of FMR
|}

Main Page

2024-05-13T15:37:06Z

Vmurrell:

{{DISPLAYTITLE:{{FULLPAGENAME}}}}
__NOTOC__
Fusion Metadata Registry (FMR) Knowledge Base 
::[[File:FMR11 screenshot.png|Fusion Metadata Registry|500px]]

Fusion Metadata Registry is a free to use structural metadata registry for SDMX supporting EDI, SDMX 1.0, 2.0, 2.1 and 3.0.

Read more about Fusion Metadata Registry on the [https://www.sdmx.io/tools/fmr/#sdmx-structural-metadata-registry sdmx.io web site]

To download Fusion Metadata Registry [https://www.sdmx.io/resources/download/fmr/ please click here.]

= Tools =
The [https://www.sdmx.io/tools/ecosystem/ sdmx.io ecosystem] includes a collection of open source SDMX software tools cooperating to solve official statistics use cases.

This knowledgebase provides further information on how to install, configure and use FMR and the tools:

* Click here to read more about [https://www.sdmx.io/tools/ecosystem/#fusionxl FusionXL on the sdmx.io site.]
* Click here to read more about [https://www.sdmx.io/tools/ecosystem/#fusion-transformer Fusion Transformer on the sdmx.io site.]
* Click here to read more about [https://www.sdmx.io/tools/ecosystem/#fusion-workbench Workbench on the sdmx.io site.]
* See below to read more about how to use [https://fmrwiki.sdmxcloud.org/Backup_and_Restore FMR-CL, a command line application] which performs functions against the web services of a running FMR instance.

==Fusion Metadata Registry Topics==
SDMX statistical data engine and structural metadata registry. The main statistical engine for data collection, integration, processing, storage, query execution and time-series calculations. It provides an SDMX-compliant data and metadata REST API. The integrated SDMX structural metadata registry acts as the central repository, authoring and maintenance tool for all of the structures including Codelists, Concepts and Data Structure Definitions.

{| class="wikitable"
|-
| [[:Fusion_Metadata_Overview|'''Overview''']]|| What is Fusion Metadata Registry?
|-
| [[:Category:Installation_and_Configuration |'''Installation and Configuration''']]|| How to install and configure the Registry
|-
| [[:Category:Concepts_Reference_V11|'''Concepts Reference''']]|| Explanation of key concepts and terms used in SDMX
|-
| [[:Category:SDMX 3.0 Structures|'''SDMX V3.0 Structures''']]|| Detailed explanations of the main SDMX structural metadata artefacts
|-
| [[:Category:Structural Metadata V11|'''Structure Maintenance''']]|| How to maintain the SDMX Structures
|-
| [[:Category:Reference_Metadata|'''Reference Metadata''']] || Guidance on working with SDMX 3.0 reference metadata in Fusion Registry 11
|-
| [[:Category:How_To V11|'''How to and other Features''']] || Guidance on specific tasks using Fusion Registry
|-
| [[:Category:SDMX 3.0 Mapping Structures|'''Mapping''']] || Mapping using Structure Maps and Representation Maps
|-
| [[:Category:FMR Formats Reference|'''Data Formats''']] || A description of supported Data Formats
|-
| [[:Category:RegistrySecurity|'''Security''']] || Authentication and Authorisation Features
|-
| [[:Category:FMR REST API Reference|'''Web Services''']] || Registry Web Services
|-
| [[Change_Log|'''Change Log''']] || Change Logs for Fusion Metadata Registry
|-
| [[:Category:Version 10 |'''Functionality Retired after Version 10''']] || Functionality no longer available in V11
|}

== FusionXL Topics ==
FusionXL is an addin for Microsoft Excel which connects to a Fusion Registry server for both data and structure retrieval, and maintenance.
{| class="wikitable"
|-
| [[FusionXL_install|'''Installation''']] || How to Install Fusion XL and connect to a Fusion Registry server
|-
| [[FusionXL_Create_Dataset|'''Creating Datasets''']] || Create datasets and load datasets for the purpose of data authoring, validation, and conversion
|-
| [[FusionXL_Author_Helper|'''Author Helper''']] || Provides assistance when creating datasets
|-
| [[Structure_Explorer|'''Structure Explorer''']] || Create, Retrieve and Modify Structures using the Excel Ribbon
|-
| [[:Category:Fusion_XL|'''Other Features''']] || Other features available in FXL
|-
| [[FusionXL_Change_Log|'''Change Logs''']] || Change Logs for FusionXL
|-
| [[FusionXL_DataExplorer|'''Retired Functionality''']] || Functionality no longer available in V3
|-
|}

== Fusion Transformer ==
[https://fmrwiki.sdmxcloud.org/Fusion_Transformer '''Fusion Transformer'''] is a command line application providing transformations between supported data files.The Fusion Transformer is a command line application providing transformations between supported data files.

== Fusion Workbench ==
The FMR Workbench (FWB) provides a web user interface (UI) for browsing and maintaining SDMX Structures obtained from external SDMX compliant Structure Web Services, including (and not limited to) - any FMR installation, OECD.STAT, public SDMX web services, such as the SDMX Global Registry,BIS,ECB,ILO,UNICEF.

{|
|-
| style="width:300px" | [[FMR_Workbench|FMR Workbench Getting Started]] || Getting started guide for the FMR Workbench
|}

== FMR-CL ==
'''F'''usion '''M'''etadata '''R'''egistry-'''C'''ommand '''L'''ine (FMR-CL) is a command line application which performs functions against the web services of a running FMR instance.

[https://fmrwiki.sdmxcloud.org/FMR-CL Overview]

==== Using FMR-CL ====
{|
|-
| style="width:300px" | [[Backup and Restore|Structure Backup]] || Supporting backup of structures and transaction history for restore in new installations of FMR
|}

FusionXL install

2024-05-13T15:26:22Z

Vmurrell: /* Download */

[[Category:Fusion_XL]]
=Overview=
FusionXL is an Excel add in for Fusion Metadata Registry providing three main functions:
* Browse and view structures
* Create and maintain structures - in particular useful for maintaining large codelists
* Author, validate and convert datasets

[[File:FXL1.PNG|1000px]]

=Download=
The FusionXL add in is a single Excel XLAM file which can be downloaded [https://www.sdmx.io/resources/download/fxl/ from here]

To add FXL to Windows 10 or higher, follow '''one''' of the two steps below.

=Install=

====Using XLSTART folder====
Copy the XLAM file to the Excel sub directory folder '''XLSTART''' as shown in the example path below:

C:\Users\[your user name]\AppData\Roaming\Microsoft\Excel\XLSTART

Files in this folder are automatically loaded when Excel starts.

====Using Excel's add-in maintenance facility====
Copy the XLAM file to the Microsoft '''AddIns''' folder:

C:\Users\[your user name]\AppData\Roaming\Microsoft\AddIns

Register the add-in with Excel by selecting the Options menu, then choosing Add-Ins.

[[File:FXL2.png|600px]]

After clicking '''Go''' an Add-Ins window will be displayed which will show '''FusionXL''' as an available add-in. Select this add-in and click '''OK'''.

[[File:FXL3.png|200px]]

After clicking OK, restart Excel to see the RXL tab as shown in the image above.

You are now ready to [[Connect_FXL_to_a_Registry|connect FXL to a Registry]].

=Troubleshooting=
====The FXL addin disappears or fails to load====
A possible cause is Windows file protection security settings blocking the FXL.xlam file as it has originated from another computer. The following YouTube video explains how to solve the problem by disabling Windows protection on the file: https://www.youtube.com/watch?v=AhnOU-ulqNg

====Macro Disabled====
As above.

FusionXL Load Data Set

2024-05-13T15:22:29Z

Vmurrell: /* Supported Formats */

[[Category:Fusion_XL]]
To load an existing dataset, locate the file using Windows Explorer.

If the system can find details of the Provision Agreement, a pop up window will appear enabling you to select the Data Provider and the Dataflow.

[[File:Supply dataset info.png|600px]]

===Supported Formats===
Supported formats are:

* SDMX-ML
* SDMX-CSV
* SDMX-JSON
* FUSION-JSON

An example of a dataset is shown below.

[[File:Load TRD dataset.png|800px]]

'''Legend'''
The way the labels are displayed differs depending on the type of component and whether it is 'coded' or not. Components in a Data Structure can be linked to a Codelist to ensure that only permitted data is entered. The permitted values and be see in the '''Author Helper''' which is discussed [[FusionXL_Author_Helper |in this article.]]

Three colours are in use along with bold text:

{| class="wikitable"
|-
! Colour !! Used for
|-
| Black|| Coded Attributes which are not mandatory. Time and Measure Components
|-
| Black + Bold|| Coded Dimensions
|-
| Brown|| Uncoded Dimensions and Attributes)
|-
| Brown + Bold|| Mandatory Uncoded Attributes and Uncoded Dimensions
|-
| Red + Bold|| Mandatory Coded Dimensions and Attributes
|-
|}

Once the dataset is loaded, it can be edited, saved and loaded into a Fusion Registry.

FusionXL Download

2024-05-13T15:19:53Z

Vmurrell:

[[Category:Fusion_XL]]
=Overview=
FusionXL is an Excel user interface for Fusion Registry providing structure browsing, structure maintenance and data authoring.

The tool can be downloaded from the FusionXl page [https://www.sdmx.io/resources/download/fxl/ on the SDMDX.io site.]

Draft - new main page

2024-05-13T15:12:59Z

Vmurrell: /* FusionXL Topics */

{{DISPLAYTITLE:{{FULLPAGENAME}}}}
__NOTOC__
Fusion Metadata Registry (FMR) Knowledge Base 
::[[File:FMR11 screenshot.png|Fusion Metadata Registry|500px]]

Fusion Metadata Registry is a free to use structural metadata registry for SDMX supporting EDI, SDMX 1.0, 2.0, 2.1 and 3.0.

Read more about Fusion Metadata Registry on the [https://www.sdmx.io/tools/fmr/#sdmx-structural-metadata-registry sdmx.io web site]

To download Fusion Metadata Registry [https://www.sdmx.io/resources/download/fmr/ please click here.]

= Tools =
The [https://www.sdmx.io/tools/ecosystem/ sdmx.io ecosystem] includes a collection of open source SDMX software tools cooperating to solve official statistics use cases.

This knowledgebase provides further information on how to install, configure and use FMR and the tools:

* Click here to read more about [https://www.sdmx.io/tools/ecosystem/#fusionxl FusionXL on the sdmx.io site.]
* Click here to read more about [https://www.sdmx.io/tools/ecosystem/#fusion-transformer Fusion Transformer on the sdmx.io site.]
* Click here to read more about [https://www.sdmx.io/tools/ecosystem/#fusion-workbench Workbench on the sdmx.io site.]
* See below to read more about how to use [https://fmrwiki.sdmxcloud.org/Backup_and_Restore FMR-CL, a command line application] which performs functions against the web services of a running FMR instance.

==Fusion Metadata Registry Topics==
SDMX statistical data engine and structural metadata registry. The main statistical engine for data collection, integration, processing, storage, query execution and time-series calculations. It provides an SDMX-compliant data and metadata REST API. The integrated SDMX structural metadata registry acts as the central repository, authoring and maintenance tool for all of the structures including Codelists, Concepts and Data Structure Definitions.

{| class="wikitable"
|-
| [[:Fusion_Metadata_Overview|'''Overview''']]|| What is Fusion Metadata Registry?
|-
| [[:Category:Installation_and_Configuration |'''Installation and Configuration''']]|| How to install and configure the Registry
|-
| [[:Category:Concepts_Reference_V11|'''Concepts Reference''']]|| Explanation of key concepts and terms used in SDMX
|-
| [[:Category:SDMX 3.0 Structures|'''SDMX V3.0 Structures''']]|| Detailed explanations of the main SDMX structural metadata artefacts
|-
| [[:Category:Structural Metadata V11|'''Structure Maintenance''']]|| How to maintain the SDMX Structures
|-
| [[:Category:Reference_Metadata|'''Reference Metadata''']] || Guidance on working with SDMX 3.0 reference metadata in Fusion Registry 11
|-
| [[:Category:How_To V11|'''How to and other Features''']] || Guidance on specific tasks using Fusion Registry
|-
| [[:Category:SDMX 3.0 Mapping Structures|'''Mapping''']] || Mapping using Structure Maps and Representation Maps
|-
| [[:Category:FMR Formats Reference|'''Data Formats''']] || A description of supported Data Formats
|-
| [[:Category:RegistrySecurity|'''Security''']] || Authentication and Authorisation Features
|-
| [[:Category:FMR REST API Reference|'''Web Services''']] || Registry Web Services
|-
| [[Change_Log|'''Change Log''']] || Change Logs for Fusion Metadata Registry
|-
| [[:Category:Version 10 |'''Functionality Retired after Version 10''']] || Functionality no longer available in V11
|}

== FusionXL Topics ==
FusionXL is an addin for Microsoft Excel which connects to a Fusion Registry server for both data and structure retrieval, and maintenance.
{| class="wikitable"
|-
| [[FusionXL_install|'''Installation''']] || How to Install Fusion XL and connect to a Fusion Registry server
|-
| [[FusionXL_Create_Dataset|'''Creating Datasets''']] || Create datasets and load datasets for the purpose of data authoring, validation, and conversion
|-
| [[FusionXL_Author_Helper|'''Author Helper''']] || Provides assistance when creating datasets
|-
| [[Structure_Explorer|'''Structure Explorer''']] || Create, Retrieve and Modify Structures using the Excel Ribbon
|-
| [[:Category:Fusion_XL|'''Other Features''']] || Other features available in FXL
|-
| [[FusionXL_Change_Log|'''Change Logs''']] || Change Logs for FusionXL
|-
| [[FusionXL_DataExplorer|'''Retired Functionality''']] || Functionality no longer available in V3
|-
|}

== Fusion Transformer ==
[https://fmrwiki.sdmxcloud.org/Fusion_Transformer '''Fusion Transformer'''] is a command line application providing transformations between supported data files.The Fusion Transformer is a command line application providing transformations between supported data files.

== Fusion Workbench ==
The FMR Workbench (FWB) provides a web user interface (UI) for browsing and maintaining SDMX Structures obtained from external SDMX compliant Structure Web Services, including (and not limited to) - any FMR installation, OECD.STAT, public SDMX web services, such as the SDMX Global Registry,BIS,ECB,ILO,UNICEF.

{|
|-
| style="width:300px" | [[FMR_Workbench|FMR Workbench Getting Started]] || Getting started guide for the FMR Workbench
|}

== FMR-CL ==
'''F'''usion '''M'''etadata '''R'''egistry-'''C'''ommand '''L'''ine (FMR-CL) is a command line application which performs functions against the web services of a running FMR instance.

[https://fmrwiki.sdmxcloud.org/FMR-CL Overview]

==== Using FMR-CL ====
{|
|-
| style="width:300px" | [[Backup and Restore|Structure Backup]] || Supporting backup of structures and transaction history for restore in new installations of FMR
|}

Draft - new main page

2024-05-13T15:11:06Z

Vmurrell: /* FusionXL Topics */

{{DISPLAYTITLE:{{FULLPAGENAME}}}}
__NOTOC__
Fusion Metadata Registry (FMR) Knowledge Base 
::[[File:FMR11 screenshot.png|Fusion Metadata Registry|500px]]

Fusion Metadata Registry is a free to use structural metadata registry for SDMX supporting EDI, SDMX 1.0, 2.0, 2.1 and 3.0.

Read more about Fusion Metadata Registry on the [https://www.sdmx.io/tools/fmr/#sdmx-structural-metadata-registry sdmx.io web site]

To download Fusion Metadata Registry [https://www.sdmx.io/resources/download/fmr/ please click here.]

= Tools =
The [https://www.sdmx.io/tools/ecosystem/ sdmx.io ecosystem] includes a collection of open source SDMX software tools cooperating to solve official statistics use cases.

This knowledgebase provides further information on how to install, configure and use FMR and the tools:

* Click here to read more about [https://www.sdmx.io/tools/ecosystem/#fusionxl FusionXL on the sdmx.io site.]
* Click here to read more about [https://www.sdmx.io/tools/ecosystem/#fusion-transformer Fusion Transformer on the sdmx.io site.]
* Click here to read more about [https://www.sdmx.io/tools/ecosystem/#fusion-workbench Workbench on the sdmx.io site.]
* See below to read more about how to use [https://fmrwiki.sdmxcloud.org/Backup_and_Restore FMR-CL, a command line application] which performs functions against the web services of a running FMR instance.

==Fusion Metadata Registry Topics==
SDMX statistical data engine and structural metadata registry. The main statistical engine for data collection, integration, processing, storage, query execution and time-series calculations. It provides an SDMX-compliant data and metadata REST API. The integrated SDMX structural metadata registry acts as the central repository, authoring and maintenance tool for all of the structures including Codelists, Concepts and Data Structure Definitions.

{| class="wikitable"
|-
| [[:Fusion_Metadata_Overview|'''Overview''']]|| What is Fusion Metadata Registry?
|-
| [[:Category:Installation_and_Configuration |'''Installation and Configuration''']]|| How to install and configure the Registry
|-
| [[:Category:Concepts_Reference_V11|'''Concepts Reference''']]|| Explanation of key concepts and terms used in SDMX
|-
| [[:Category:SDMX 3.0 Structures|'''SDMX V3.0 Structures''']]|| Detailed explanations of the main SDMX structural metadata artefacts
|-
| [[:Category:Structural Metadata V11|'''Structure Maintenance''']]|| How to maintain the SDMX Structures
|-
| [[:Category:Reference_Metadata|'''Reference Metadata''']] || Guidance on working with SDMX 3.0 reference metadata in Fusion Registry 11
|-
| [[:Category:How_To V11|'''How to and other Features''']] || Guidance on specific tasks using Fusion Registry
|-
| [[:Category:SDMX 3.0 Mapping Structures|'''Mapping''']] || Mapping using Structure Maps and Representation Maps
|-
| [[:Category:FMR Formats Reference|'''Data Formats''']] || A description of supported Data Formats
|-
| [[:Category:RegistrySecurity|'''Security''']] || Authentication and Authorisation Features
|-
| [[:Category:FMR REST API Reference|'''Web Services''']] || Registry Web Services
|-
| [[Change_Log|'''Change Log''']] || Change Logs for Fusion Metadata Registry
|-
| [[:Category:Version 10 |'''Functionality Retired after Version 10''']] || Functionality no longer available in V11
|}

== FusionXL Topics ==
FusionXL is an addin for Microsoft Excel which connects to a Fusion Registry server for both data and structure retrieval, and maintenance.
{| class="wikitable"
|-
| [[FusionXL_install|'''Installation''']] || How to Install Fusion XL and connect to a Fusion Registry server
|-
| [[FusionXL_Create_Dataset|'''Creating Datasets''']] || Create datasets and load datasets for the purpose of data authoring, validation, and conversion
|-
| [[FusionXL_Author_Helper|'''Author Helper''']] || Provides assistance when creating datasets
|-
| [[Structure_Explorer|'''Structure Explorer''']] || Create, Retrieve and Modify Structures using the Excel Ribbon
|-
| [https://fmrwiki.sdmxcloud.org/Category:Fusion_XL|'''Other Features'''] || Other features available in FXL
|-
| [[FusionXL_Change_Log|'''Change Logs''']] || Change Logs for FusionXL
|-
| [[FusionXL_DataExplorer|'''Retired Functionality''']] || Functionality no longer available in V3
|-
|}

== Fusion Transformer ==
[https://fmrwiki.sdmxcloud.org/Fusion_Transformer '''Fusion Transformer'''] is a command line application providing transformations between supported data files.The Fusion Transformer is a command line application providing transformations between supported data files.

== Fusion Workbench ==
The FMR Workbench (FWB) provides a web user interface (UI) for browsing and maintaining SDMX Structures obtained from external SDMX compliant Structure Web Services, including (and not limited to) - any FMR installation, OECD.STAT, public SDMX web services, such as the SDMX Global Registry,BIS,ECB,ILO,UNICEF.

{|
|-
| style="width:300px" | [[FMR_Workbench|FMR Workbench Getting Started]] || Getting started guide for the FMR Workbench
|}

== FMR-CL ==
'''F'''usion '''M'''etadata '''R'''egistry-'''C'''ommand '''L'''ine (FMR-CL) is a command line application which performs functions against the web services of a running FMR instance.

[https://fmrwiki.sdmxcloud.org/FMR-CL Overview]

==== Using FMR-CL ====
{|
|-
| style="width:300px" | [[Backup and Restore|Structure Backup]] || Supporting backup of structures and transaction history for restore in new installations of FMR
|}

Draft - new main page

2024-05-13T15:06:06Z

Vmurrell: /* FusionXL Topics */

{{DISPLAYTITLE:{{FULLPAGENAME}}}}
__NOTOC__
Fusion Metadata Registry (FMR) Knowledge Base 
::[[File:FMR11 screenshot.png|Fusion Metadata Registry|500px]]

Fusion Metadata Registry is a free to use structural metadata registry for SDMX supporting EDI, SDMX 1.0, 2.0, 2.1 and 3.0.

Read more about Fusion Metadata Registry on the [https://www.sdmx.io/tools/fmr/#sdmx-structural-metadata-registry sdmx.io web site]

To download Fusion Metadata Registry [https://www.sdmx.io/resources/download/fmr/ please click here.]

= Tools =
The [https://www.sdmx.io/tools/ecosystem/ sdmx.io ecosystem] includes a collection of open source SDMX software tools cooperating to solve official statistics use cases.

This knowledgebase provides further information on how to install, configure and use FMR and the tools:

* Click here to read more about [https://www.sdmx.io/tools/ecosystem/#fusionxl FusionXL on the sdmx.io site.]
* Click here to read more about [https://www.sdmx.io/tools/ecosystem/#fusion-transformer Fusion Transformer on the sdmx.io site.]
* Click here to read more about [https://www.sdmx.io/tools/ecosystem/#fusion-workbench Workbench on the sdmx.io site.]
* See below to read more about how to use [https://fmrwiki.sdmxcloud.org/Backup_and_Restore FMR-CL, a command line application] which performs functions against the web services of a running FMR instance.

==Fusion Metadata Registry Topics==
SDMX statistical data engine and structural metadata registry. The main statistical engine for data collection, integration, processing, storage, query execution and time-series calculations. It provides an SDMX-compliant data and metadata REST API. The integrated SDMX structural metadata registry acts as the central repository, authoring and maintenance tool for all of the structures including Codelists, Concepts and Data Structure Definitions.

{| class="wikitable"
|-
| [[:Fusion_Metadata_Overview|'''Overview''']]|| What is Fusion Metadata Registry?
|-
| [[:Category:Installation_and_Configuration |'''Installation and Configuration''']]|| How to install and configure the Registry
|-
| [[:Category:Concepts_Reference_V11|'''Concepts Reference''']]|| Explanation of key concepts and terms used in SDMX
|-
| [[:Category:SDMX 3.0 Structures|'''SDMX V3.0 Structures''']]|| Detailed explanations of the main SDMX structural metadata artefacts
|-
| [[:Category:Structural Metadata V11|'''Structure Maintenance''']]|| How to maintain the SDMX Structures
|-
| [[:Category:Reference_Metadata|'''Reference Metadata''']] || Guidance on working with SDMX 3.0 reference metadata in Fusion Registry 11
|-
| [[:Category:How_To V11|'''How to and other Features''']] || Guidance on specific tasks using Fusion Registry
|-
| [[:Category:SDMX 3.0 Mapping Structures|'''Mapping''']] || Mapping using Structure Maps and Representation Maps
|-
| [[:Category:FMR Formats Reference|'''Data Formats''']] || A description of supported Data Formats
|-
| [[:Category:RegistrySecurity|'''Security''']] || Authentication and Authorisation Features
|-
| [[:Category:FMR REST API Reference|'''Web Services''']] || Registry Web Services
|-
| [[Change_Log|'''Change Log''']] || Change Logs for Fusion Metadata Registry
|-
| [[:Category:Version 10 |'''Functionality Retired after Version 10''']] || Functionality no longer available in V11
|}

== FusionXL Topics ==
FusionXL is an addin for Microsoft Excel which connects to a Fusion Registry server for both data and structure retrieval, and maintenance.
{| class="wikitable"
|-
| [[FusionXL_install|'''Installation''']] || How to Install Fusion XL and connect to a Fusion Registry server
|-
| [[FusionXL_Create_Dataset|'''Creating Datasets''']] || Create datasets and load datasets for the purpose of data authoring, validation, and conversion
|-
| [[FusionXL_Author_Helper|'''Author Helper''']] || Create, Retrieve and Modify Structures using the Excel Ribbon
|-
| [[Structure_Explorer|'''Structure Explorer''']] || Create, Retrieve and Modify Structures using the Excel Ribbon
|-
| [[Structure_Explorer|'''Other Features''']] || Create, Retrieve and Modify Structures using the Excel Ribbon
|-
| [[FusionXL_Change_Log|'''Change Logs''']] || Change Logs for FusionXL
|-
| [[FusionXL_DataExplorer|'''Retired Functionality''']] || Functionality no longer available in V3
|-
|}

== Fusion Transformer ==
[https://fmrwiki.sdmxcloud.org/Fusion_Transformer '''Fusion Transformer'''] is a command line application providing transformations between supported data files.The Fusion Transformer is a command line application providing transformations between supported data files.

== Fusion Workbench ==
The FMR Workbench (FWB) provides a web user interface (UI) for browsing and maintaining SDMX Structures obtained from external SDMX compliant Structure Web Services, including (and not limited to) - any FMR installation, OECD.STAT, public SDMX web services, such as the SDMX Global Registry,BIS,ECB,ILO,UNICEF.

{|
|-
| style="width:300px" | [[FMR_Workbench|FMR Workbench Getting Started]] || Getting started guide for the FMR Workbench
|}

== FMR-CL ==
'''F'''usion '''M'''etadata '''R'''egistry-'''C'''ommand '''L'''ine (FMR-CL) is a command line application which performs functions against the web services of a running FMR instance.

[https://fmrwiki.sdmxcloud.org/FMR-CL Overview]

==== Using FMR-CL ====
{|
|-
| style="width:300px" | [[Backup and Restore|Structure Backup]] || Supporting backup of structures and transaction history for restore in new installations of FMR
|}

Draft - new main page

2024-05-13T15:01:35Z

Vmurrell: /* FusionXL Topics */

{{DISPLAYTITLE:{{FULLPAGENAME}}}}
__NOTOC__
Fusion Metadata Registry (FMR) Knowledge Base 
::[[File:FMR11 screenshot.png|Fusion Metadata Registry|500px]]

Fusion Metadata Registry is a free to use structural metadata registry for SDMX supporting EDI, SDMX 1.0, 2.0, 2.1 and 3.0.

Read more about Fusion Metadata Registry on the [https://www.sdmx.io/tools/fmr/#sdmx-structural-metadata-registry sdmx.io web site]

To download Fusion Metadata Registry [https://www.sdmx.io/resources/download/fmr/ please click here.]

= Tools =
The [https://www.sdmx.io/tools/ecosystem/ sdmx.io ecosystem] includes a collection of open source SDMX software tools cooperating to solve official statistics use cases.

This knowledgebase provides further information on how to install, configure and use FMR and the tools:

* Click here to read more about [https://www.sdmx.io/tools/ecosystem/#fusionxl FusionXL on the sdmx.io site.]
* Click here to read more about [https://www.sdmx.io/tools/ecosystem/#fusion-transformer Fusion Transformer on the sdmx.io site.]
* Click here to read more about [https://www.sdmx.io/tools/ecosystem/#fusion-workbench Workbench on the sdmx.io site.]
* See below to read more about how to use [https://fmrwiki.sdmxcloud.org/Backup_and_Restore FMR-CL, a command line application] which performs functions against the web services of a running FMR instance.

==Fusion Metadata Registry Topics==
SDMX statistical data engine and structural metadata registry. The main statistical engine for data collection, integration, processing, storage, query execution and time-series calculations. It provides an SDMX-compliant data and metadata REST API. The integrated SDMX structural metadata registry acts as the central repository, authoring and maintenance tool for all of the structures including Codelists, Concepts and Data Structure Definitions.

{| class="wikitable"
|-
| [[:Fusion_Metadata_Overview|'''Overview''']]|| What is Fusion Metadata Registry?
|-
| [[:Category:Installation_and_Configuration |'''Installation and Configuration''']]|| How to install and configure the Registry
|-
| [[:Category:Concepts_Reference_V11|'''Concepts Reference''']]|| Explanation of key concepts and terms used in SDMX
|-
| [[:Category:SDMX 3.0 Structures|'''SDMX V3.0 Structures''']]|| Detailed explanations of the main SDMX structural metadata artefacts
|-
| [[:Category:Structural Metadata V11|'''Structure Maintenance''']]|| How to maintain the SDMX Structures
|-
| [[:Category:Reference_Metadata|'''Reference Metadata''']] || Guidance on working with SDMX 3.0 reference metadata in Fusion Registry 11
|-
| [[:Category:How_To V11|'''How to and other Features''']] || Guidance on specific tasks using Fusion Registry
|-
| [[:Category:SDMX 3.0 Mapping Structures|'''Mapping''']] || Mapping using Structure Maps and Representation Maps
|-
| [[:Category:FMR Formats Reference|'''Data Formats''']] || A description of supported Data Formats
|-
| [[:Category:RegistrySecurity|'''Security''']] || Authentication and Authorisation Features
|-
| [[:Category:FMR REST API Reference|'''Web Services''']] || Registry Web Services
|-
| [[Change_Log|'''Change Log''']] || Change Logs for Fusion Metadata Registry
|-
| [[:Category:Version 10 |'''Functionality Retired after Version 10''']] || Functionality no longer available in V11
|}

== FusionXL Topics ==
FusionXL is an addin for Microsoft Excel which connects to a Fusion Registry server for both data and structure retrieval, and maintenance.
{| class="wikitable"
|-
| [[FusionXL_install|'''Installation''']] || How to Install Fusion XL and connect to a Fusion Registry server
|-
| [[FusionXL_Create_Dataset|'''Creating Datasets''']] || Create datasets and load datasets for the purpose of data authoring, validation, and conversion
|-
| [[FusionXL_Author_Helper|'''Author Helper''']] || Create, Retrieve and Modify Structures using the Excel Ribbon
|-
| [[Structure_Explorer|'''Structure Explorer''']] || Create, Retrieve and Modify Structures using the Excel Ribbon
|-

| [[FusionXL_Change_Log|'''Change Logs''']] || Change Logs for FusionXL
|-
| [[FusionXL_DataExplorer|'''Retired Functionality''']] || Functionality no longer available in V3
|-
|}

== Fusion Transformer ==
[https://fmrwiki.sdmxcloud.org/Fusion_Transformer '''Fusion Transformer'''] is a command line application providing transformations between supported data files.The Fusion Transformer is a command line application providing transformations between supported data files.

== Fusion Workbench ==
The FMR Workbench (FWB) provides a web user interface (UI) for browsing and maintaining SDMX Structures obtained from external SDMX compliant Structure Web Services, including (and not limited to) - any FMR installation, OECD.STAT, public SDMX web services, such as the SDMX Global Registry,BIS,ECB,ILO,UNICEF.

{|
|-
| style="width:300px" | [[FMR_Workbench|FMR Workbench Getting Started]] || Getting started guide for the FMR Workbench
|}

== FMR-CL ==
'''F'''usion '''M'''etadata '''R'''egistry-'''C'''ommand '''L'''ine (FMR-CL) is a command line application which performs functions against the web services of a running FMR instance.

[https://fmrwiki.sdmxcloud.org/FMR-CL Overview]

==== Using FMR-CL ====
{|
|-
| style="width:300px" | [[Backup and Restore|Structure Backup]] || Supporting backup of structures and transaction history for restore in new installations of FMR
|}

FusionXL Create Dataset

2024-05-13T14:58:44Z

Vmurrell: /* Overview */

[[Category:Fusion_XL]]
=Overview=
[[File:FXL createdataset overview.JPG|thumb]]
The FusonXL Create Data Set function assists in reporting data to a Registry 10 or 11 statistical data warehouse by creating an Excel spreadsheet form that is tailored to the specific dataflow.

The reporting process is simplified by:
* Defining the reporting period and frequency which automatically generates an Excel form with the correct observation time periods
* Optionally setting fixed values for Dimensions and Series Attributes where the value is the same for all series being reported
* Optionally setting default values for Observation Attributes which apply to all observations unless overridden for specific observations
* Choosing which non-mandatory attributes to report

Required:
* FusionXL 2.6.0 or later
* Fusion Registry 10.8.4 or later
* Fusion Registry 11.0.1 or later

=Get started=
From the FusionXL ribbon bar
# Log in as a user with Data Provider privileges
# Choose ''Data Author'' under the Connection group
# Choose ''Create Data Set''
[[File:FXL createdataset ribbon.JPG|350px]]

=Dataflow=
Choose the Dataflow to report data for. 
Set the frequency and reporting period.

[[File:FXL createdataset dataflow.JPG|700px]]

=Detailed component settings=
Set which components to include in the data set and the values they take.
* Fix values for dimensions
* Fix values for coded and uncoded series attributes
* Set default values for observation attributes - these take effect for all observations for which a specific value has not been set
* Choose which non-mandatory attributes to include
====Fix values for dimensions and series attributes====
Set to <code>(none)</code> or blank to remove the fixed value.

[[File:FXL createdataset fixed.JPG|600px]]

====Choose which non-mandatory attributes to include====
Non-mandatory attributes have a checkbox to indicate whether they should be included in the dataset.

[[File:FXL createdataset nonmandatory.JPG|570px]]

====Set default values for observation attributes====
[[File:FXL createdataset observation.JPG|570px]]

View Activity

2024-05-13T14:45:13Z

Vmurrell: /* User Activity */

[[Category:How_To]]
==Overview==
The activity menu provides the user with the ability to view the structure submissions and deletions in the Registry, and to view how a specific structure has been modified over time. The activity section is split into three parts:

==SDMX Transactions==
The SDMX Transactions provide a record of each transaction over time. A Transaction is the submission of a structure, group of structures or data registration. The submission may have resulted from the structure being modified in the UI or via the web service.

Each Transaction is recorded on the RSS feed, and made available in the UI.

[[File:Activity9.PNG|SDMX Transactions Page|1000px]]

===Options Available===

'''Download Transaction''' 
The transaction can be downloaded in SDMX-XML format and will include the submitted structure(s) alongside the submission action (Replace, Delete).

'''View Details'''
This option displays the transaction's detail and provides the ability to download the log file in CSV format.

'''Roll Back to selected transaction'''
Allows you to roll back the Registry structures to the selected transaction point. Any structural modifications made after that point will be reverted.

==Structure Submissions==
The Structure Submission option provides a breakdown of all the structural metadata content of the Registry (including any content that has been subsequently deleted). Each structure artefact can be viewed to see when it was modified over time, and each revision to the structure can be downloaded in SDMX format.

[[File:Activity6.PNG|Structure Submissions Page|1000px]]

===Options Available===

'''Download Transaction''' 
The transaction can be downloaded in SDMX-XML format and will include the submitted structure(s) alongside the submission action (Replace, Delete).

'''View Details'''
This option displays the transaction's detail and provides the ability to download the log file in CSV format.

'''Compare'''
Allows you to compare two modifications against each other.

=User Activity=

[[File:Activity8.PNG|User Activity Page|1000px]]

===Options Available===

'''View Audit Details'''
This option displays the transaction's detail and provides the ability to download the log file in CSV format.

View Activity

2024-05-13T14:17:43Z

Vmurrell: /* User Activity */

[[Category:How_To]]
==Overview==
The activity menu provides the user with the ability to view the structure submissions and deletions in the Registry, and to view how a specific structure has been modified over time. The activity section is split into three parts:

==SDMX Transactions==
The SDMX Transactions provide a record of each transaction over time. A Transaction is the submission of a structure, group of structures or data registration. The submission may have resulted from the structure being modified in the UI or via the web service.

Each Transaction is recorded on the RSS feed, and made available in the UI.

[[File:Activity9.PNG|SDMX Transactions Page|1000px]]

===Options Available===

'''Download Transaction''' 
The transaction can be downloaded in SDMX-XML format and will include the submitted structure(s) alongside the submission action (Replace, Delete).

'''View Details'''
This option displays the transaction's detail and provides the ability to download the log file in CSV format.

'''Roll Back to selected transaction'''
Allows you to roll back the Registry structures to the selected transaction point. Any structural modifications made after that point will be reverted.

==Structure Submissions==
The Structure Submission option provides a breakdown of all the structural metadata content of the Registry (including any content that has been subsequently deleted). Each structure artefact can be viewed to see when it was modified over time, and each revision to the structure can be downloaded in SDMX format.

[[File:Activity6.PNG|Structure Submissions Page|1000px]]

===Options Available===

'''Download Transaction''' 
The transaction can be downloaded in SDMX-XML format and will include the submitted structure(s) alongside the submission action (Replace, Delete).

'''View Details'''
This option displays the transaction's detail and provides the ability to download the log file in CSV format.

'''Compare'''
Allows you to compare two modifications against each other.

=User Activity=

This option is no longer available.

[[File:Activity8.PNG|User Activity Page|1000px]]

===Options Available===

'''View Audit Details'''
This option displays the transaction's detail and provides the ability to download the log file in CSV format.

Versioning of structural metadata artefacts

2024-05-13T14:08:07Z

Vmurrell: /* Overview */

[[Category:Concepts_Reference_V10]]
[[Category:Concepts_Reference_V11]]
=Overview=
All Maintainable structures have a mandatory version number of the form [major number].[minor number].

Examples: 1.0, 1.1, 2.3

The version number defaults to 1.0.

The combination of the structure type, Agency ID, structure ID and version are used to uniquely identify any Maintainable structure using its [[URN V11]].

Examples of URN illustrating the use of version numbers: 
<code><nowiki>urn:sdmx:org.sdmx.infomodel.codelist.Codelist=SDMX:CL_TIME_PER_COLLECT(1.11)</nowiki></code> 
<code><nowiki>urn:sdmx:org.sdmx.infomodel.conceptscheme.Concept=IMF:ECOFIN_CONCEPTS(2.0).INDICATOR</nowiki></code> 
<code><nowiki>urn:sdmx:org.sdmx.infomodel.datastructure.DataStructure=IMF:ECOFIN_DSD(1.0)</nowiki></code> 
<code><nowiki>urn:sdmx:org.sdmx.infomodel.registry.ProvisionAgreement=WB:WDI_ENVIRONMENT_WB_WB(1.0)</nowiki></code>

=Versioning Guidance=
SDMX does not mandate a versioning policy. Organisations maintaining their own structures are therefore free to choose how and when version numbers change.

There are two generally adopted approaches.
====Option 1 - Keep all structures at version 1.0====
A simple method is to keep all structures at version 1.0.

Changes are made as required without changing any version numbers, but users of structures like DSDs may be surprised to find components have been added or removed unless other metadata governance procedures are in place.

====Option 2 - Change version numbers to indicate significant changes have been made====
For structures like Data Structure Definitions and Dataflows, versioning can be used to indicate to users when changes have been made that impact how they are used.

Data Structure Definition example versioning policy:
{| class="wikitable"
|-
! Change !! Version
|-
| Change the DSD's description || The version number is not modified because the change has no material impact on the way the structure is used.
|-
| Add a new Attribute || Increment the minor version number, for instance moving from 1.0 to 1.1
|-
| A significant redesign including adding and deleting multiple dimensions and attributes|| Increment the major version number, for instance moving from 1.1 to 2.0
|}

For Item Schemes such as Codelists and Concept Schemes, it is generally recommended not to increment the version number when new codes are added. The reason being that Codelists are linked to Data Structure Definitions at a particular version, for example CL_REF_AREA(1.0). If significant changes are made to a Codelist like a complete redesign of a classification scheme, consideration should be given to incrementing the minor or major version. Thus a user may infer that CL_REF_AREA(2.0) is significantly different to CL_REF_AREA(1.0), and not just because it carries some additional codes.

Valuelist

2024-05-13T14:07:05Z

Vmurrell: /* Conventions */

[[Category:SDMX 2.1 Structures]]
=Overview=
A Valuelist is a managed list of values, similar to an [[Codelist_V10|SDMX Codelist]] but unlike a Code the Value Id does not need to conform to the [[Identifiable_V10|SDMX ID]] Type.

'''Note:''' Value List became an official part of the SDMX standard in version 3.0

==Structure Properties==
{| class="wikitable"
! scope=row style="text-align: left;" | Structure Type
| Standard SDMX Structural Metadata Artefact
|-
! scope=row style="text-align: left;" | Maintainable
| [[Maintainable V10|Yes]]
|-
! scope=row style="text-align: left;" | Identifiable
| [[Identifiable V10|Yes]]
|-
! scope=row style="text-align: left;" | Item Scheme
| No
|-
! scope=row style="text-align: left;" | SDMX Information Model Versions
| 3.0
|-
! scope=row style="text-align: left;" | Concept ID
| ValueList
|}

=Example=

The following table is used to demonstrate the types of values that a Valuelist could hold
{| class="wikitable"
|-
! Id !! Name !! Description
|-
| $ || Dollar || -
|-
| £|| Pound || -
|-
| % || Percent || A number or ratio that represents a fraction of 100.
|}

=Formats=
{| class="wikitable"
|-
! Format !! supported !! Example
|-
| SDXM-EDI || No || -
|-
| SDMX-1.0|| No || -
|-
| SDMX-2.0 || No || -
|-
| SDMX-2.1 || No || -
|-
| Fusion-JSON || Yes|| [[ValueList_FusionJSON|Click Here to View Sample]]
|}

==Context within the 2.1 Information Model==
::[[File:Valuelist Model.png|600px]]
The schematic illustrates the how the Valuelist is used by Data Structure Definitions (DSD).
'''Note''' A Valuelist is not an official part of the SDMX Information model at version 2.1, which is currently the latest model.

==Usage==
Valuelists provide an enumerated list of allowable values, for which there is also a mandatory Name and optional description. This list can be referenced for defining the set of allowed values for enumerated Components [[Data_Structure_Definitions|Data Structure Definitions (DSDs)]] or [[Metadata Structure Definition|Metadata Structure Definitions]] which describe [[Reference Metadata]].

=Conventions=
'VL_' Valuelist ID Prefix

Valielist IDs are given a 'VL_' prefix to distinguish them from other structures. For instance: VL_CURRENCY.

[[URN V10|You can seem more examples and information on Identities in this article]].

Valuelist

2024-05-13T14:06:04Z

Vmurrell: /* Usage */

[[Category:SDMX 2.1 Structures]]
=Overview=
A Valuelist is a managed list of values, similar to an [[Codelist_V10|SDMX Codelist]] but unlike a Code the Value Id does not need to conform to the [[Identifiable_V10|SDMX ID]] Type.

'''Note:''' Value List became an official part of the SDMX standard in version 3.0

==Structure Properties==
{| class="wikitable"
! scope=row style="text-align: left;" | Structure Type
| Standard SDMX Structural Metadata Artefact
|-
! scope=row style="text-align: left;" | Maintainable
| [[Maintainable V10|Yes]]
|-
! scope=row style="text-align: left;" | Identifiable
| [[Identifiable V10|Yes]]
|-
! scope=row style="text-align: left;" | Item Scheme
| No
|-
! scope=row style="text-align: left;" | SDMX Information Model Versions
| 3.0
|-
! scope=row style="text-align: left;" | Concept ID
| ValueList
|}

=Example=

The following table is used to demonstrate the types of values that a Valuelist could hold
{| class="wikitable"
|-
! Id !! Name !! Description
|-
| $ || Dollar || -
|-
| £|| Pound || -
|-
| % || Percent || A number or ratio that represents a fraction of 100.
|}

=Formats=
{| class="wikitable"
|-
! Format !! supported !! Example
|-
| SDXM-EDI || No || -
|-
| SDMX-1.0|| No || -
|-
| SDMX-2.0 || No || -
|-
| SDMX-2.1 || No || -
|-
| Fusion-JSON || Yes|| [[ValueList_FusionJSON|Click Here to View Sample]]
|}

==Context within the 2.1 Information Model==
::[[File:Valuelist Model.png|600px]]
The schematic illustrates the how the Valuelist is used by Data Structure Definitions (DSD).
'''Note''' A Valuelist is not an official part of the SDMX Information model at version 2.1, which is currently the latest model.

==Usage==
Valuelists provide an enumerated list of allowable values, for which there is also a mandatory Name and optional description. This list can be referenced for defining the set of allowed values for enumerated Components [[Data_Structure_Definitions|Data Structure Definitions (DSDs)]] or [[Metadata Structure Definition|Metadata Structure Definitions]] which describe [[Reference Metadata]].

=Conventions=
'VL_' Valuelist ID Prefix

Valielist IDs are given a 'VL_' prefix to distinguish them from other structures. For instance: VL_CURRENCY.

[[URN|You can seem more examples and information on Identities in this article]].

Valuelist

2024-05-13T14:04:04Z

Vmurrell: /* Structure Properties */

[[Category:SDMX 2.1 Structures]]
=Overview=
A Valuelist is a managed list of values, similar to an [[Codelist_V10|SDMX Codelist]] but unlike a Code the Value Id does not need to conform to the [[Identifiable_V10|SDMX ID]] Type.

'''Note:''' Value List became an official part of the SDMX standard in version 3.0

==Structure Properties==
{| class="wikitable"
! scope=row style="text-align: left;" | Structure Type
| Standard SDMX Structural Metadata Artefact
|-
! scope=row style="text-align: left;" | Maintainable
| [[Maintainable V10|Yes]]
|-
! scope=row style="text-align: left;" | Identifiable
| [[Identifiable V10|Yes]]
|-
! scope=row style="text-align: left;" | Item Scheme
| No
|-
! scope=row style="text-align: left;" | SDMX Information Model Versions
| 3.0
|-
! scope=row style="text-align: left;" | Concept ID
| ValueList
|}

=Example=

The following table is used to demonstrate the types of values that a Valuelist could hold
{| class="wikitable"
|-
! Id !! Name !! Description
|-
| $ || Dollar || -
|-
| £|| Pound || -
|-
| % || Percent || A number or ratio that represents a fraction of 100.
|}

=Formats=
{| class="wikitable"
|-
! Format !! supported !! Example
|-
| SDXM-EDI || No || -
|-
| SDMX-1.0|| No || -
|-
| SDMX-2.0 || No || -
|-
| SDMX-2.1 || No || -
|-
| Fusion-JSON || Yes|| [[ValueList_FusionJSON|Click Here to View Sample]]
|}

==Context within the 2.1 Information Model==
::[[File:Valuelist Model.png|600px]]
The schematic illustrates the how the Valuelist is used by Data Structure Definitions (DSD).
'''Note''' A Valuelist is not an official part of the SDMX Information model at version 2.1, which is currently the latest model.

==Usage==
Valuelists provide an enumerated list of allowable values, for which there is also a mandatory Name and optional description. This list can be referenced for defining the set of allowed values for enumerated [[Data_Structure_Definition#Data_Structure_Components|Components]] in [[Data Structure Definition|Data Structure Definitions (DSDs)]] or [[Metadata Structure Definition|Metadata Structure Definitions]] which describe [[Reference Metadata]].

=Conventions=
'VL_' Valuelist ID Prefix

Valielist IDs are given a 'VL_' prefix to distinguish them from other structures. For instance: VL_CURRENCY.

[[URN|You can seem more examples and information on Identities in this article]].

Valuelist

2024-05-13T14:03:38Z

Vmurrell: /* Structure Properties */

[[Category:SDMX 2.1 Structures]]
=Overview=
A Valuelist is a managed list of values, similar to an [[Codelist_V10|SDMX Codelist]] but unlike a Code the Value Id does not need to conform to the [[Identifiable_V10|SDMX ID]] Type.

'''Note:''' Value List became an official part of the SDMX standard in version 3.0

==Structure Properties==
{| class="wikitable"
! scope=row style="text-align: left;" | Structure Type
| Standard SDMX Structural Metadata Artefact
|-
! scope=row style="text-align: left;" | Maintainable
| [[Maintainable V10|Yes]]
|-
! scope=row style="text-align: left;" | Identifiable
| [[Identifiable V10|Yes]]
|-
! scope=row style="text-align: left;" | Item Scheme
| No
|-
! scope=row style="text-align: left;" | SDMX Information Model Versions
| 3.0
|-
! scope=row style="text-align: left;" | Concept ID
| ValueList
|}

=Example=

The following table is used to demonstrate the types of values that a Valuelist could hold
{| class="wikitable"
|-
! Id !! Name !! Description
|-
| $ || Dollar || -
|-
| £|| Pound || -
|-
| % || Percent || A number or ratio that represents a fraction of 100.
|}

=Formats=
{| class="wikitable"
|-
! Format !! supported !! Example
|-
| SDXM-EDI || No || -
|-
| SDMX-1.0|| No || -
|-
| SDMX-2.0 || No || -
|-
| SDMX-2.1 || No || -
|-
| Fusion-JSON || Yes|| [[ValueList_FusionJSON|Click Here to View Sample]]
|}

==Structure Properties==
{| class="wikitable"
! scope=row style="text-align: left;" | Structure Type
| Standard SDMX Structural Metadata Artefact
|-
! scope=row style="text-align: left;" | Maintainable
| [[Maintainable|Yes]]
|-
! scope=row style="text-align: left;" | Identifiable
| [[Identifiable|Yes]]
|-
! scope=row style="text-align: left;" | Item Scheme
| [[Item_Scheme|No]]
|-
! scope=row style="text-align: left;" | SDMX Information Model Versions
| 3.0
|-
! scope=row style="text-align: left;" | URN
| <nowiki>urn:sdmx:org.sdmx.infomodel.valuelist.Valuelist</nowiki>
|}

==Context within the 2.1 Information Model==
::[[File:Valuelist Model.png|600px]]
The schematic illustrates the how the Valuelist is used by Data Structure Definitions (DSD).
'''Note''' A Valuelist is not an official part of the SDMX Information model at version 2.1, which is currently the latest model.

==Usage==
Valuelists provide an enumerated list of allowable values, for which there is also a mandatory Name and optional description. This list can be referenced for defining the set of allowed values for enumerated [[Data_Structure_Definition#Data_Structure_Components|Components]] in [[Data Structure Definition|Data Structure Definitions (DSDs)]] or [[Metadata Structure Definition|Metadata Structure Definitions]] which describe [[Reference Metadata]].

=Conventions=
'VL_' Valuelist ID Prefix

Valielist IDs are given a 'VL_' prefix to distinguish them from other structures. For instance: VL_CURRENCY.

[[URN|You can seem more examples and information on Identities in this article]].

Valuelist

2024-05-13T14:02:46Z

Vmurrell:

[[Category:SDMX 2.1 Structures]]
=Overview=
A Valuelist is a managed list of values, similar to an [[Codelist_V10|SDMX Codelist]] but unlike a Code the Value Id does not need to conform to the [[Identifiable_V10|SDMX ID]] Type.

'''Note:''' Value List became an official part of the SDMX standard in version 3.0

==Structure Properties==
{| class="wikitable"
! scope=row style="text-align: left;" | Structure Type
| Standard SDMX Structural Metadata Artefact
|-
! scope=row style="text-align: left;" | Maintainable
| [[Maintainable|Yes]]
|-
! scope=row style="text-align: left;" | Identifiable
| [[Identifiable|Yes]]
|-
! scope=row style="text-align: left;" | Item Scheme
| No
|-
! scope=row style="text-align: left;" | SDMX Information Model Versions
| 3.0
|-
! scope=row style="text-align: left;" | Concept ID
| ValueList
|}

=Example=

The following table is used to demonstrate the types of values that a Valuelist could hold
{| class="wikitable"
|-
! Id !! Name !! Description
|-
| $ || Dollar || -
|-
| £|| Pound || -
|-
| % || Percent || A number or ratio that represents a fraction of 100.
|}

=Formats=
{| class="wikitable"
|-
! Format !! supported !! Example
|-
| SDXM-EDI || No || -
|-
| SDMX-1.0|| No || -
|-
| SDMX-2.0 || No || -
|-
| SDMX-2.1 || No || -
|-
| Fusion-JSON || Yes|| [[ValueList_FusionJSON|Click Here to View Sample]]
|}

==Structure Properties==
{| class="wikitable"
! scope=row style="text-align: left;" | Structure Type
| Standard SDMX Structural Metadata Artefact
|-
! scope=row style="text-align: left;" | Maintainable
| [[Maintainable|Yes]]
|-
! scope=row style="text-align: left;" | Identifiable
| [[Identifiable|Yes]]
|-
! scope=row style="text-align: left;" | Item Scheme
| [[Item_Scheme|No]]
|-
! scope=row style="text-align: left;" | SDMX Information Model Versions
| 3.0
|-
! scope=row style="text-align: left;" | URN
| <nowiki>urn:sdmx:org.sdmx.infomodel.valuelist.Valuelist</nowiki>
|}

==Context within the 2.1 Information Model==
::[[File:Valuelist Model.png|600px]]
The schematic illustrates the how the Valuelist is used by Data Structure Definitions (DSD).
'''Note''' A Valuelist is not an official part of the SDMX Information model at version 2.1, which is currently the latest model.

==Usage==
Valuelists provide an enumerated list of allowable values, for which there is also a mandatory Name and optional description. This list can be referenced for defining the set of allowed values for enumerated [[Data_Structure_Definition#Data_Structure_Components|Components]] in [[Data Structure Definition|Data Structure Definitions (DSDs)]] or [[Metadata Structure Definition|Metadata Structure Definitions]] which describe [[Reference Metadata]].

=Conventions=
'VL_' Valuelist ID Prefix

Valielist IDs are given a 'VL_' prefix to distinguish them from other structures. For instance: VL_CURRENCY.

[[URN|You can seem more examples and information on Identities in this article]].

Valuelist

2024-05-13T14:01:59Z

Vmurrell: /* Overview */

[[Category:SDMX 2.1 Structures]]
=Overview=
A Valuelist is a managed list of values, similar to an [[Codelist_V10|SDMX Codelist]] but unlike a Code the Value Id does not need to conform to the [[Identifiable|SDMX ID]] Type.

'''Note:''' Value List became an official part of the SDMX standard in version 3.0

==Structure Properties==
{| class="wikitable"
! scope=row style="text-align: left;" | Structure Type
| Standard SDMX Structural Metadata Artefact
|-
! scope=row style="text-align: left;" | Maintainable
| [[Maintainable|Yes]]
|-
! scope=row style="text-align: left;" | Identifiable
| [[Identifiable|Yes]]
|-
! scope=row style="text-align: left;" | Item Scheme
| No
|-
! scope=row style="text-align: left;" | SDMX Information Model Versions
| 3.0
|-
! scope=row style="text-align: left;" | Concept ID
| ValueList
|}

=Example=

The following table is used to demonstrate the types of values that a Valuelist could hold
{| class="wikitable"
|-
! Id !! Name !! Description
|-
| $ || Dollar || -
|-
| £|| Pound || -
|-
| % || Percent || A number or ratio that represents a fraction of 100.
|}

=Formats=
{| class="wikitable"
|-
! Format !! supported !! Example
|-
| SDXM-EDI || No || -
|-
| SDMX-1.0|| No || -
|-
| SDMX-2.0 || No || -
|-
| SDMX-2.1 || No || -
|-
| Fusion-JSON || Yes|| [[ValueList_FusionJSON|Click Here to View Sample]]
|}

==Structure Properties==
{| class="wikitable"
! scope=row style="text-align: left;" | Structure Type
| Standard SDMX Structural Metadata Artefact
|-
! scope=row style="text-align: left;" | Maintainable
| [[Maintainable|Yes]]
|-
! scope=row style="text-align: left;" | Identifiable
| [[Identifiable|Yes]]
|-
! scope=row style="text-align: left;" | Item Scheme
| [[Item_Scheme|No]]
|-
! scope=row style="text-align: left;" | SDMX Information Model Versions
| 3.0
|-
! scope=row style="text-align: left;" | URN
| <nowiki>urn:sdmx:org.sdmx.infomodel.valuelist.Valuelist</nowiki>
|}

==Context within the 2.1 Information Model==
::[[File:Valuelist Model.png|600px]]
The schematic illustrates the how the Valuelist is used by Data Structure Definitions (DSD).
'''Note''' A Valuelist is not an official part of the SDMX Information model at version 2.1, which is currently the latest model.

==Usage==
Valuelists provide an enumerated list of allowable values, for which there is also a mandatory Name and optional description. This list can be referenced for defining the set of allowed values for enumerated [[Data_Structure_Definition#Data_Structure_Components|Components]] in [[Data Structure Definition|Data Structure Definitions (DSDs)]] or [[Metadata Structure Definition|Metadata Structure Definitions]] which describe [[Reference Metadata]].

=Conventions=
'VL_' Valuelist ID Prefix

Valielist IDs are given a 'VL_' prefix to distinguish them from other structures. For instance: VL_CURRENCY.

[[URN|You can seem more examples and information on Identities in this article]].

Value List

2024-05-13T13:59:51Z

Vmurrell:

__NOTOC__
[[Category:SDMX 3.0 Structures]]
==Overview==
An SDMX Value List is a managed list of classification codes which are represented by values (example £ or $).

Value Lists can be referenced directly by [[Data_or_Metadata_Structure_Definition|Data Structure Definitions]], or indirectly through [[Concepts V11|Concepts]] to explicitly define the closed set of values for enumerated [[Dimension|Dimensions]], [[Attribute|Attributes]] or [[Measure V11|Measures]].

==Structure Properties==
{| class="wikitable"
! scope=row style="text-align: left;" | Structure Type
| Standard SDMX Structural Metadata Artefact
|-
! scope=row style="text-align: left;" | Maintainable
| [[Maintainable V11|Yes]]
|-
! scope=row style="text-align: left;" | Identifiable
| [[Identifiable V11|Yes]]
|-
! scope=row style="text-align: left;" | Item Scheme
| [[Item_Scheme V11|Yes]]
|-
! scope=row style="text-align: left;" | SDMX Information Model Versions
| 3.0
|-
! scope=row style="text-align: left;" | URN - ValueList namespace
| <nowiki>urn:sdmx:org.sdmx.infomodel.valuelist.ValueList</nowiki>
|-
! scope=row style="text-align: left;" | URN - Code namespace
| <nowiki>urn:sdmx:org.sdmx.infomodel.valuelist.Code</nowiki>
|}

==Context within the SDMX 3.0 Information Model==

[[File:SDMX3 ValueList.png|Value List|600px]]

==Usage==
SDMX Value Lists are lists of classification codes used principally for defining the set of allowed values for enumerated [[Component|Components]] in [[Data_or_Metadata_Structure_Definition|Data Structure Definitions (DSDs)]] or [[Metadata Structure Definition|Metadata Structure Definitions]] which describe [[Reference Metadata]].

Each code is a separate Item so must have an ID and a Name, but can also have an optional Description. While Code IDs must be unique within a Codelist, the same Code ID may be safely used in other Codelists. For instance: The code '£' may be used in a CURRENCY Value list to represent 'Pound Sterling', but also appear in an CURRECNY_DENOMINATION Value List to represent 'GB Pound'. 

==Conventions==
''To be written.''

Validation behaviour using item validity

2024-05-13T11:35:33Z

Vmurrell:

__NOTOC__
[[Category:How_To]]
[[Category:How_To V11]]
== Overview ==
Fusion Registry's [[Data_Validation|Structural Data Validation]] function checks that data submitted conforms to the Dataflow's [[Data Structure Definitions]], and also detects other potential problems like conflicting duplicate observations. For [[Component|enumerated components]], dimension or attribute values submitted must match those in the specified [[Codelists]].
 
When creating a Codelist, the designer has the option of defining the period in time during which the Code is considered valid - the ''validity period'' for the Code item. If no validity period is set, the Code is considered to be valid for all time. Note that a Code may have multiple validity periods.
 
'''Series referencing Codes which have specific validity periods are only valid if all of the observations fall within those periods.'''

==Example==
::[[File:Validation behaviour using item validity.jpg|frameless|900px]]

The above example illustrates a simple dataset using a Data Structure Defintion (DSD) with three dimensions: INDICATOR, REF_AREA and FREQ.

REF_AREA is an enumerated country dimension with the CL_REF_AREA Codelist containing three codes:
* GE - Gilbert and Ellice Islands
* KI - Kiribati
* TV - Tuvulu

CL_REF_AREA has item validity periods defined for each of the three codes as follows:
* GE - valid between 1916 and 1976
* KI - valid from 1977
* TV - valid from 1977

The first three series in the example dataset are valid because the observation values all fall within the validity period for the REF_AREA code 'GE'.

A_IND:GE:A is invalid because the series has observations for 1977 and 1978, years when the REF_AREA code GE is not valid.
 
Similarly A_IND:KI:A and A_IND:TV:A are also invalid beacuse the item validity periods for both the 'KI' and 'TV' REF_AREA codes extend from 1977 to date meaning that the observations
provided for 1970,1971 and 1972 cannot be correct.

Validation behaviour using item validity

2024-05-13T11:34:25Z

Vmurrell:

__NOTOC__
[[Category:How_To]]
[[Category:How_To V11]]
== Overview ==
Fusion Registry's [[Structural data validation|Structural Data Validation]] function checks that data submitted conforms to the Dataflow's [[Data Structure Definitions]], and also detects other potential problems like conflicting duplicate observations. For [[Component|enumerated components]], dimension or attribute values submitted must match those in the specified [[Codelists]].
 
When creating a Codelist, the designer has the option of defining the period in time during which the Code is considered valid - the ''validity period'' for the Code item. If no validity period is set, the Code is considered to be valid for all time. Note that a Code may have multiple validity periods.
 
'''Series referencing Codes which have specific validity periods are only valid if all of the observations fall within those periods.'''

==Example==
::[[File:Validation behaviour using item validity.jpg|frameless|900px]]

The above example illustrates a simple dataset using a Data Structure Defintion (DSD) with three dimensions: INDICATOR, REF_AREA and FREQ.

REF_AREA is an enumerated country dimension with the CL_REF_AREA Codelist containing three codes:
* GE - Gilbert and Ellice Islands
* KI - Kiribati
* TV - Tuvulu

CL_REF_AREA has item validity periods defined for each of the three codes as follows:
* GE - valid between 1916 and 1976
* KI - valid from 1977
* TV - valid from 1977

The first three series in the example dataset are valid because the observation values all fall within the validity period for the REF_AREA code 'GE'.

A_IND:GE:A is invalid because the series has observations for 1977 and 1978, years when the REF_AREA code GE is not valid.
 
Similarly A_IND:KI:A and A_IND:TV:A are also invalid beacuse the item validity periods for both the 'KI' and 'TV' REF_AREA codes extend from 1977 to date meaning that the observations
provided for 1970,1971 and 1972 cannot be correct.

Validation behaviour using item validity

2024-05-13T11:33:50Z

Vmurrell:

__NOTOC__
[[Category:How_To]]
[[Category:How_To V11]]
== Overview ==
Fusion Registry's [[Structural data validation|Structural Data Validation]] function checks that data submitted conforms to the Dataflow's [[Data Structure Definitions]], and also detects other potential problems like conflicting duplicate observations. For [[Component|enumerated components]], dimension or attribute values submitted must match those in the specified [[Codelist]].
 
When creating a Codelist, the designer has the option of defining the period in time during which the Code is considered valid - the ''validity period'' for the Code item. If no validity period is set, the Code is considered to be valid for all time. Note that a Code may have multiple validity periods.
 
'''Series referencing Codes which have specific validity periods are only valid if all of the observations fall within those periods.'''

==Example==
::[[File:Validation behaviour using item validity.jpg|frameless|900px]]

The above example illustrates a simple dataset using a Data Structure Defintion (DSD) with three dimensions: INDICATOR, REF_AREA and FREQ.

REF_AREA is an enumerated country dimension with the CL_REF_AREA Codelist containing three codes:
* GE - Gilbert and Ellice Islands
* KI - Kiribati
* TV - Tuvulu

CL_REF_AREA has item validity periods defined for each of the three codes as follows:
* GE - valid between 1916 and 1976
* KI - valid from 1977
* TV - valid from 1977

The first three series in the example dataset are valid because the observation values all fall within the validity period for the REF_AREA code 'GE'.

A_IND:GE:A is invalid because the series has observations for 1977 and 1978, years when the REF_AREA code GE is not valid.
 
Similarly A_IND:KI:A and A_IND:TV:A are also invalid beacuse the item validity periods for both the 'KI' and 'TV' REF_AREA codes extend from 1977 to date meaning that the observations
provided for 1970,1971 and 1972 cannot be correct.

Validation behaviour using item validity

2024-05-13T11:32:32Z

Vmurrell:

__NOTOC__
[[Category:How_To]]
[[Category:How_To V11]]
== Overview ==
Fusion Registry's [[Structural data validation|Structural Data Validation]] function checks that data submitted conforms to the Dataflow's [[Data Structure Definitions]], and also detects other potential problems like conflicting duplicate observations. For [[Enumerated component|enumerated components]], dimension or attribute values submitted must match those in the specified [[Codelist]].
 
When creating a Codelist, the designer has the option of defining the period in time during which the Code is considered valid - the ''validity period'' for the Code item. If no validity period is set, the Code is considered to be valid for all time. Note that a Code may have multiple validity periods.
 
'''Series referencing Codes which have specific validity periods are only valid if all of the observations fall within those periods.'''

==Example==
::[[File:Validation behaviour using item validity.jpg|frameless|900px]]

The above example illustrates a simple dataset using a Data Structure Defintion (DSD) with three dimensions: INDICATOR, REF_AREA and FREQ.

REF_AREA is an enumerated country dimension with the CL_REF_AREA Codelist containing three codes:
* GE - Gilbert and Ellice Islands
* KI - Kiribati
* TV - Tuvulu

CL_REF_AREA has item validity periods defined for each of the three codes as follows:
* GE - valid between 1916 and 1976
* KI - valid from 1977
* TV - valid from 1977

The first three series in the example dataset are valid because the observation values all fall within the validity period for the REF_AREA code 'GE'.

A_IND:GE:A is invalid because the series has observations for 1977 and 1978, years when the REF_AREA code GE is not valid.
 
Similarly A_IND:KI:A and A_IND:TV:A are also invalid beacuse the item validity periods for both the 'KI' and 'TV' REF_AREA codes extend from 1977 to date meaning that the observations
provided for 1970,1971 and 1972 cannot be correct.

Validate data

2024-05-13T11:27:05Z

Vmurrell:

[[Category:How_To]]
[[Category:How_To V11]]
=Overview=
To Validate data you need to have the following in place.

==Data Provider==
A [[Organisations_-_Data_Providers|Data Provider]] is an Organisation Type. When a [[Provision_Agreements|Provision Agreement]] is created a [[Dataflows|Dataflow]] and a Data Provider must be selected. An example Data Provider is shown below.

[[File:val1.PNG|Data Provider - Step 1|500px]] 

==Provision Agreement==

A Provision Agreement (PA)is the union of a Dataflow with a Data Provider. A PA is a definition that the Data Provider can provide data for the Dataflow. Data is always reported by a Data Provider against the PA. [https://fmrwiki.sdmxcloud.org/Provision_Agreements You can read more about Provision Agreements in this article]. An example Provision Agreement is shown below.

[[File:val2.PNG|Creating a Provision Agreement|850px]] 

==Dataflow==

A Dataflow is a structure on which data is collected and disseminated. A Dataflow references a Data Structure Definition (DSD) which is used as the underlying template to which the data must conform.
[https://wiki.sdmxcloud.org/Dataflow_-_Structural_Metadata_Management You can read more about Dataflows in this article]. An example Dataflow is shown below.

[[File:val3.PNG|Dataflow - Step 1|500px]] 

[[File:val4.PNG|Dataflow - Step 2|850px]] 

=Load Data=

Once all the elements are in place as described above, the next step is to load the data which is done via the Convert option on the Data Menu.

Data can be loaded from a file of via a URL (for example from the SDMX Global Registry site).

To successfully validate, the data must adhere to the SDMX standard in terms of format as well as what has been defined in the Data Structure.

[https://fmrwiki.sdmxcloud.org/Data_Structure_Definitions You can read more about Data Structures in this article].

[https://fmrwiki.sdmxcloud.org/Create_a_simple_DSD You can read more about how to create a simple Data Structures in this article].

'''Supported formats:'''
* SDMX-ML (3.0, 2.1 , 2.0 and 1.0)
* EDI
* SDMX-JSON
* SDMX-CSV
* XLS / XLSX

'''Unsupported formats:'''
* Any other CSV format (Fusion CSV, Time Series CSV)
* RDF, Turtle, etc.

[[File:val5.PNG|Convert Data Page|850px]] 

To see this process in action you can watch this video.

[[File:LOADINGDATA.mp4]] 

==Validate Data==

Click Load Data to start the validation process as explained in the image below.

[[File:val6.PNG|Dataset Details Page|850px]] 

=Validation Scheme=
==What is a Validation Scheme?==
Validation Schemes define one or more validation rules which can be executed against a Dataflow at the data validation stage of a data load. Each validation rule consists of a mathematical expression or a link to an aggregation hierarchy which is used to create an expression. This validation goes beyond syntactical and semantical validation of the dataset and is instead checking that the values supplied in the dataset conform to specific business rules. Examples of this could be that a particular field must have a value less than 100, or that the total value reported must be the same as the total of a set of other observation values.

[[File:Valid1.PNG|Validation Scheme Page|850px]] 

A Validation Scheme must be assigned against a single Dataflow and may consist of one or many validation rules. A single Validation Rule consists of:

* an ID and name
* an optional description
* a type: either a custom expression or a hierarchic expression
* a result type (either numerical or code type) and a value (e.g. 100 or GDP)
* an equality operator (one of the following mathematical operators: =, <>, <, <=, >, >= )
* an expression (e.g. [EUR]+[FR] )

The Validation Scheme rules will be applicable to all datasets submitted against the Dataflow the Validation Scheme is linked to.

==How are Rules Applied==

A validation rule operates on a single dimension, an example of a rule to calculate Total from the inputs Males and Females would look like the following:

[T] = [M] + [F]

'''Note''': the syntax used in a validation scheme puts code Ids into square brackets.

This rule would be applied to every series where all other parts of the series key match, so the following series there would be two matches to this rule, one for employment, and one for unemployment.

[[File:Valid2.PNG|Rule Application 1|850px]] 

For a validation rule to be executed there must be data reported for the output, and at least one of the inputs. If data are missing in the inputs, then they are treated a zero value. In the following example, only 1 rule is matched, and there is only one input (Male).

[[File:Valid3.PNG|Rule Application 2|850px]] 

There are two types of validation rules, ones which use a custom written expression, as described above. The second type references a Hierarchy in the Registry, and the Hierarchy is used as the basis for an Aggregation expression. For example, the following image shows a hierarchy of countries, against theoretical reported values. This is an example of a hierarchy being used to validate a dataset.
A hierarchy can be applied to any dimension that uses the same Codelist as the Hierarchy. When values are read in the data file, the totals at each sub-hierarchy are summed up to ensure they are consistent with the parent value. If any values are missing data, they are treated as having a value of zero.

[[File:Valid4.PNG|Rule Diagram|thumb|850px]] 

'''Note''': the Registry only checks data in the submitted file, and does not cross check against any persisted data when validating. For example, if you have already stored the totals in a Registry database, submitting a Dataset containing the values making up the totals, the Registry will not validate from the file against the totals already stored.

=Validation Scheme Tutorial=
This tutorial describes the manual steps in the process to create a Validation Scheme. It is required that your Registry be populated with structures that support this process (such as Data Structure Definitions and Dataflows).

==Overview==
This section will explain the creation of a simple Validation Scheme that demonstrates when Data is loaded into the Registry that the rules within the scheme will be used for validation purposes.

==Creating the Validation Scheme from the User Interface==
A Validation Scheme is created or maintained by using the authoring Wizard. The cogs icon used to open the wizard, is only available to authenticated Agency or Admin users.

[[File:Create Validation Scheme.PNG|1000px]] 

There are four steps to creating a Validation Scheme and are described in the following sections:

===Step 1: Details===
The first step allows for the specification of the ID and other associated high-level values which make up the definition of the Validation Scheme.

[[File:Valid6.PNG|Validation Scheme - Step 1|600px]] 

Like all other SDMX Structures, the Id, Version, Name and Agency are mandatory, and the other values are optional. Enter these values and click on the '''Next''' button.

===Step 2: Attachment===
This step assigns the Validation Scheme to a single Dataflow.

[[File:Valid7.PNG|Validation Scheme - Step 2|600px]] 

Click on the Dataflow field which will show all Dataflows in the Registry. Select the Dataflow to attach this Scheme to and its details will be displayed in the lower half of this page.

Click on the '''Next''' button once the correct Dataflow has been selected.

===Step 3: CSV Import===
This step is optional and allows for the creation of several validation rules via text entry.

[[File:Valid8.PNG|Validation Scheme - Step 3|600px]] 

To create a rule, enter the rule details on its own line in the text area. Elements of the rule must be separated by commas. Each element must conform to the expected value of that field (e.g., the ID element must not contain illegal characters; the equality operator may only be one of the permitted values). The field order is determined by the drop-downs at the top of this step and can be changed by the user.

Syntax of an Expression:

Each expression operates on a single dimension, and is made up of three parts:

# Resultant 
#
# Equality Operator 
#
# Operands and Operators 

The resultant and operands can either be numerical constants (e.g., 100) or the Id of a code (e.g., EUROPE). If a code id is used, the syntax demands it is placed in square brackets [EUROPE].

For example, an expression to validate the reported value for EUROPE may look like the following:

''[EUROPE]=([UK]+[FR]+[DE]) - (([ES]+[IT]+[LUX]) * 2)''

When importing this via CSV the output, equality operator, and expression are all supplied in their own CSV field. So, a full example of a CSV rule would look like the following:

''SERIES_CHECK,Europe Total,REF_AREA,[EUROPE],=,"([UK]+[FR]+[DE]) - (([ES]+[IT]+[LUX]) * 2)"''

This would create the rule with the following characteristics:
* Rule Id: SERIES_CHECK
* Rule Name: Europe Total
* Rule Description: <none> (no column index supplied)
* Dimension Id: REF_AREA
* Mathematical expression: EUROPE = (UK+FR+DE) - ((ES+IT+LUX) * 2)

===Step 4: Expression Builder===
Rather than use the CSV import (step 3) to create a validation rule, it may be easier to create a validation rule through the user interface on step 4.

[[File:Valid9.PNG|Validation Scheme - Step 4|800px]] 

The left-hand side of the page shows all dimensions for this Dataflow followed by a number. The number indicates the number of Validation Rules against that dimension. The right-hand side shows the rules that have been defined for the currently selected Dimension. Selecting a rule shows the expression for that rule (i.e., the logic of the rule itself) or if this is a hierarchical rule then the aggregation hierarchy is displayed.

Editing or creating a rule displays a dialog where the values for the rule can be defined. This is similar to step 3 of the Validation Scheme Wizard.

[[File:Valid10.PNG|Validation Scheme - Create Rule|800px]] 

As stated previously a rule must have an Id and Name and may have an optional description. The control at top-right allows the user to distinguish between a Custom Expression and a Hierarchical Expression (this choice is only available for Dimensions with a Concept that represents a Hierarchic codelist).

The Mathematical Expression can be defined as either a Code evaluated against an expression (e.g., EUROPE=FR+DE) or a value evaluated against an expression (e.g., 55 >= GE). The expression area may be typed into manually and the user will need to do this in order to add mathematical symbols and numbers as appropriate. However the Quick Code Lookup, situated at the bottom of this page, lists all of the codes for this Dimension. Clicking an item in this list will insert it into the expression area.

For Hierarchic Rules, these can only be created when the Codelist referenced by the Dimension is also referenced by a Hierarchy – this permits the Rule Type option of Aggregate Using Hierarchy to be selected. If no hierarchies exist which use the same Codelist as the Dimension, the Rule Type option will be fixed on Custom Rule.

When the rule type is set to Aggregate using Hierarchy the result and expression section of the dialog is replaced with a list of available Hierarchies that can be selected (under the heading “Hierarchy to Use”). A Hierarchy must be selected, and the view of the selected hierarchy is shown to the right of the hierarchical list.

[[File:Valid11.PNG|Validation Rules|800px]] 

Once the rule has been defined, clicking “Add Rule” will dismiss the dialog and add this rule to the selected dimension.

Note that during creation, rules are not validated for logic or contradictory values. So, it is possible to create nonsensical rules such as EUROPE >= EUROPE. It is down to the user to ensure that the Validation Rules have a logical sense.

VTL Mapping Scheme

2024-05-13T11:18:08Z

Vmurrell:

__NOTOC__
[[Category:Transformation_Scheme]]
==Overview==
''To be written''

==Structure Properties==
{| class="wikitable"
! scope=row style="text-align: left;" | Structure Type
| ''to be confirmed''
|-
! scope=row style="text-align: left;" | Maintainable
| [[Maintainable_V11| ''to be confirmed'']]
|-
! scope=row style="text-align: left;" | Identifiable
| [[Identifiable V11| ''to be confirmed'']]
|-
! scope=row style="text-align: left;" | Item Scheme
| [[Item_Scheme| ''to be confirmed'']]
|-
! scope=row style="text-align: left;" | SDMX Information Model Versions
| 3.0
|-
! scope=row style="text-align: left;" | URN - ''to be confirmed''
| ''to be confirmed''
|-

|}

==Context within the SDMX 3.0 Information Model==

[[File:SDMX3 VTL MappingSchemeScheme.png|VTL Mapping Scheme|600px]]

==Usage==
''to be confirmed''

==Conventions==
''to be confirmed''

Category:Transformation Scheme

2024-05-13T11:16:55Z

Vmurrell:

__NOTOC__
[[Category:SDMX 3.0 Structures]]
==Overview==
''To be written''

==Structure Properties==
{| class="wikitable"
! scope=row style="text-align: left;" | Structure Type
| ''to be confirmed''
|-
! scope=row style="text-align: left;" | Maintainable
| [[Maintainable_V11| ''to be confirmed'']]
|-
! scope=row style="text-align: left;" | Identifiable
| [[Identifiable V11| ''to be confirmed'']]
|-
! scope=row style="text-align: left;" | Item Scheme
| [[Item_Scheme| ''to be confirmed'']]
|-
! scope=row style="text-align: left;" | SDMX Information Model Versions
| 3.0
|-
! scope=row style="text-align: left;" | URN - ''to be confirmed''
| ''to be confirmed''
|-

|}

==Context within the SDMX 3.0 Information Model==

[[File:SDMX3 TransformationScheme.png|Transformation Scheme|600px]]

==Usage==
''to be confirmed''

==Conventions==
''to be confirmed''

User Defined Operator Scheme

2024-05-13T11:15:33Z

Vmurrell:

[[Category:Transformation_Scheme]]
__NOTOC__
==Overview==
''To be written''

==Structure Properties==
{| class="wikitable"
! scope=row style="text-align: left;" | Structure Type
| ''to be confirmed''
|-
! scope=row style="text-align: left;" | Maintainable
| [[Maintainable_V11| ''to be confirmed'']]
|-
! scope=row style="text-align: left;" | Identifiable
| [[Identifiable V11| ''to be confirmed'']]
|-
! scope=row style="text-align: left;" | Item Scheme
| [[Item_Scheme| ''to be confirmed'']]
|-
! scope=row style="text-align: left;" | SDMX Information Model Versions
| 3.0
|-
! scope=row style="text-align: left;" | URN - ''to be confirmed''
| ''to be confirmed''
|-

|}

==Context within the SDMX 3.0 Information Model==

[[File:SDMX3 User Defined OperatorScheme.png|User Defined Operator Scheme|600px]]

==Usage==
''to be confirmed''

==Conventions==
''to be confirmed''

Use the Web Services Page to build queries on Structures

2024-05-13T11:13:51Z

Vmurrell:

[[Category:How_To]]
[[Category:How_To V11]]
__NOTOC__
==Overview==
The Web Services page allows you to use the on-screen options to build a Query URL. The query URL can be used for a variety of purposes and more detail can be found in the Web Services User Guide. This article concentrates on using this page to retrieve and submit structures.

As selections are made from the options, you will notice that the URL will change. The URL can also be manipulated manually by adding other path parameters as detailed in the Web Services User Guide however some useful examples are shown later in this article.

[[File:Webservices.PNG|Web Services - Structures Page|1200px]]

==Options==
===Structure Type===
Clicking the dropdown will display a list of all possible structures. The last option on the list is All Structures which should be used with caution depending on the size of the Registry.

===Agency===
Clicking the dropdown will display a list of all the Agencies in the Registry. The first option on the list is All Agencies which should be used with caution depending on the on the size of the Registry and the Structure Type selected above.

===Version===
The default is Latest but it is possible to choose All or to select a specific Version Number too.

===Structure Format===
The Structure Format and Structure Sub-Format dropdown messages display the options available. For structures which are not part of the SDMX 2.1 Standards, [[Fusion-JSON|Fusion-JSON can be used]].

===Pretty Print XML===
The output produced will use an indented style which makes it easier to view.

===Detail===
{| class="wikitable"
|-
! Option !! Description
|-
| Full (Default) || The structures will be output in full (including any referenced structures)
|-
| All Stubs || Instead of full, the output will be embedded as “stubs” that contain the reference from where the structures can be retrieved.
|-
| Reference Stubs || The selected structure(s) will be output if full with and referenced structures as stubs.
|-
| Reference Partial || Outputs the full query result and any referenced Codelists, Concept Schemes, Agency Schemes are returned as partial lists based on the Codes, Concepts, and Agencies used by the referencing Provision Agreements, Dataflows, Data Structures, Hierarchical Codelists. Partial Codelists are derived from Content Constraints used to define allowable content for data reporting.
|}

===References===
{| class="wikitable"
|-
! Option !! Description
|-
| None (Default) || No additional structures will be output
|-
| Parents || Will output structures the reference the structures matching the query
|-
| Parents and Siblings || Same as parents, but also include all the additional structures referenced by the parents
|-
| Children || Will output the structures referenced by the structures in the query result
|-
| Descendants || Will output children and their children (up to any level)
|-
| All || All of the above
|}
==Query URL==
This displays the URL based on the selections that you have made. You can also add parameters, an example of which is shown in the table below.

You can now copy this URL (for example if you wanted to obtain structures from another registry) and use it as you wish (for example to load the structures into your Registry using the Load via URL option).

You can also select Open URK, which will display the result in a new browser tab or downloads the resulting file (for example to load the structures into your Registry using the Load via File option).

==Useful Parameters==

===Multiple Versions===

{| class="wikitable"
|-
! + Operator !! URL
|-
| Use this to obtain a number of versions as shown in the example which is a request for versions 1.0, 1.1 and 1.2 of a specific Codelist || <nowiki>http://localhost:8085/FusionRegistry/ws/public/sdmxapi/rest/codelist/MY_AGENCY/CL_AREAS/1.0+1.1+1.2/?format=sdmx-2.1&detail=full&references=none</nowiki>
|}

The [[REST API Overview]] provides a good general introduction covering support for the standard SDMX REST API and extended REST functions.

Use the Web Services Page to build queries on Structures

2024-05-13T11:13:19Z

Vmurrell:

Authentication Service

2024-05-13T08:25:07Z

Vmurrell:

[[Category:RegistrySecurity]]
[[Category:How_To_V11]]
==Overview==
The administration page to set the Authentication Service of FMR has controls for the different security services. Use the drop-down at the top of the page to select the service you require. The current options are:
* None
* Active Directory
* LDAP
* Fusion Security

===Configuring Active Directory===

When Active Directory has been chosen as the Authentication Mechanism, the following information needs to be provided

[[File:AuthenticationService-ShowingActiveDirectory.png|Authentication Set-up|1000px]]

* The protocol type: either ldap or ldaps
* The IP address or name of the server hosting the Active Directory instance
* The Active Directory Domain Name - this can be obtained from the Active Directory administration tool
* The Base Group Domain Name (optional)
* The User Search Filter (optional)

The 2 values which are sometimes the most difficult to define are the Base Group Domain Name and the User Search Filter. If these are specified incorrectly, then even though FMR is communicating with your Active Directory instance correctly, then you may find users are unable to login.

====Base Group Domain Name====
The value here can be left blank, in which case the root of the Active Directory tree will be used as the root of searching for users. However to be more specific you can specify the Organizational Units (OU) and Domain Components (DC) to search within. It is likely you will need to specify the Domain Components. Please note that the order of the components is very important here.

When specifying OUs they are typically listed in reverse order, with the lowest-level OU specified first which aligns with the hierarchical nature of how these units are defined within AD. The DC elements are listed in a more natural left-to-right order.

By way of example, if you wished to search in the Domain Component "bisorg.external" and under that domain, there was the hierarchy of the Organizational Unit "fusion" with the Organizational Unit "Registry" within that, then the following string, with its elements in the following order, would be used:
<pre>
OU=Registry,OU=fusion,DC=bisorg,DC=external
</pre>

====User Search Filter====
This field determines how the FMR will locate users within the AD tree. The syntax of valid values can be located on the Microsoft support site. FMR will not validate the contents of this field for syntactical correctness and will simply use what is entered.

If this field is left blank, the following value will be used:

<code>
(&(objectClass=user)(userPrincipalName={0}))
</code>

This states that the search filter is looking for objects in Active Directory of class "user" and where the User Principal Name of those objects is the value as specified by {0}. {0} is a placeholder for the value the user types in to FMR in order to become authenticated, however there are some subtleties. In the User Search field there are two placeholder values:
* {0} - is the "bind principal". This is the username that the user types into the login dialog (when attempting to login). However if this value does NOT contain an "@", then the domain name is appended to it. For example, if the user attempts to log in as "Alice", on the domain "bisorg.external", then the value that replaces {0} will be Alice@bisorg.external
* {1} - is the "username" only. This corresponds exactly to the username. If the user attempts to log in as "Alice", then the value that replaces {1} will be simply "Alice".

By way of example, the following specifies a search for objects of class "user" where the User Principal Name in AD will be the "bind principal". If this is not resolved, then the sAMAccountName is checked against the username:

<code>
(&(objectClass=user)(|(userPrincipalName={0})(sAMAccountName={1})))
</code>

===Testing the Active Directory Settings===
Since it can be difficult to set the Active Directory settings correctly, since version 11.17.0 of FMR, there is also the ability from this page for an Admin user of FMR to test how Active Directory would respond with an attempted log-on for a particular user. This feature allows for quicker investigation and resolution of the Active Directory settings that you will require for your environment.

Clicking the "Test Settings" control displays a section of the page allowing the systems admin to enter a userid and password. Note that the other AD settings can be modified during this time allowing for experimentation.

When the blue "Test User" button is clicked , the system attempts to authenticate the user specified in the lower section. Here are some sample responses:

'''Example of incorrect password for the user:'''

[[File:Test-ADUser-01.png|Authentication Set-up|1000px]]
 
 
'''Example of incorrect AD - the user couldn't be located since the Base Group DN was incorrect:'''

[[File:Test-ADUser-02.png|Authentication Set-up|1000px]]
 
 
'''Example of too many users being found. There must be a unique user for the specified userid'''

[[File:Test-ADUser-03.png|Authentication Set-up|1000px]]
 
 
'''Example of correct credentials:'''

[[File:Test-ADUser-04-Success.png|Authentication Set-up]]
 
 

===Other Active Directory Tools===
If you are still having issues determining what your settings should be for Active Directory, then the Windows Powershell commands "'''Get-ADUser'''" and "'''dsquery'''" may be useful to you.

'''dsquery''' is a command-line tool in Windows operating systems used to query the Active Directory directory service. Active Directory is a service developed by Microsoft that provides centralized authentication, authorization, and directory services for Windows-based networks. The dsquery command allows administrators to search and retrieve information from Active Directory. It can be used to find various objects such as users, groups, computers, and organizational units (OU).

If you wish to find a particular user, you can use the command:

<pre>
dsquery user -samid <username>
</pre>

It can also allow you to test your search filter, such as in the following manner, which obtains the Base Group DN for the user 'test1'

<pre>
dsquery * domainroot -filter "(&(objectClass=user)(userPrincipalName=test1*))"

"CN=test1,OU=users,OU=Registry,OU=fusion,DC=bisorg,DC=external"
</pre>

The Windows PowerShell command '''Get-ADUser''' allows the retrieval of a user's information. So to see what the sAMAccountName is for a particular user, you can use:
<pre>
Get-ADUser -Identity "username" | Select-Object SamAccountName
</pre>

===Next Steps===
It is likely that you will wish to create Role Mappings for your users. Please refer to [[Active_Directory_-_Set_up_Role_Mappings|the Role Mappings page]] for further information.

Upgrading to Version 11

2024-04-17T15:56:48Z

Vmurrell:

[[Category:How_To]]
[[Category:How_To V11]]
'''''This is an old page and is superseded by the page [[Upgrading from Version 10 to Version 11]].'''

Upgrading to Version 11

2024-04-17T15:56:39Z

Vmurrell: Replaced content with "Category:How_To Category:How_To V11 '''''This is an old page and is superseded by the page Upgrading from Version 10 to Version 11.''' T"

[[Category:How_To]]
[[Category:How_To V11]]
'''''This is an old page and is superseded by the page [[Upgrading from Version 10 to Version 11]].'''

T

Update Fusion Metadata Registry

2024-04-17T15:50:28Z

Vmurrell: /* Deploying the Fusion Metadata Registry Core WAR file to Tomcat */

[[Category:How_To]]
=Pre-requisites=
In order to deploy a FusionRegistry war file for a new Installation, file you need to have followed these steps:
* Install a Java Runtime Environment [[Install Java Runtime Environment|More]]
* Install and configure an Apache Tomcat web application server [[Install Apache Tomcat|More]]
* Install and configure a MySQL database [[Install MySQL|More]]

=Overview=
The Fusion Metadata Registry Core software is distributed as a ZIP file containing:
* The FusionRegistry.war file - the actual software
* PDF documentation - setup, configuration and operating guides
* Some sample SDMX structural metadata and data

We recommend that you save the Zip file as you need to extract the war file in order to complete the installation and you may need may need the remaining files at a later date.

To install Fusion Metadata Registry, extract the WAR file which needs to be deployed to the Tomcat web application server. That's explained in the next section.

=Deploying the Fusion Metadata Registry Core WAR file to Tomcat=
* Stop Tomcat if it is already running
* Copy the WAR file into the <code>tomcat\webapps</code> directory
* Start Tomcat

The deployment process will take between 1 and 2 minutes.

Point your web browser at <code><nowiki>http://localhost:8080/FusionRegistry</nowiki></code> and you'll see the Fusion Metadata Registry installation page when the deployment process is complete.

[[File:BlankInstallPage.PNG|Installation Page|1000px]]

[[Install_Fusion_Metadata_Registry_MySQL| Click here to see the next step for a New Installation]].

URN V11

2024-04-17T15:48:42Z

Vmurrell: /* URN Identifiers */

[[Category:Concepts_Reference_V11]]
=Overview=
A URN is an abbreviation for [https://en.wikipedia.org/wiki/Uniform_Resource_Name '''U'''niform '''R'''esource '''N'''ame]. The URN, as used by the SDMX specification, is a globally unique identifier to any [[Identifiable V11|Identifiable]] SDMX Structure.

The URN is used as a mechanism to link structures together without having to know the exact location of the structure, this is how the URN differs from a URL, the former provides a long lasting references which is agnostic to location, the latter provides a precise location of where to obtain the resource.

=Structure of a SDMX URN=
A URN is made up of 2 to 3 parts, depending on the type of structure being referenced.
# Namespace
# Reference to the [[Maintainable V11|Maintainable]] Structure
# Reference to the [[Identifiable V11|Identifiable]] Structure (if the URN is referencing a Structure within a Maintainable, i.e a Code within a Codelist)

==URN Namespace==
The URN Namespaces are defined by SDMX, and take the following syntax:

[[File:URN Namespace.png|700px]]

The prefix for all SDMX Structural Metadata is '''<nowiki> urn:sdmx:org.sdmx.infomodel</nowiki>''' 

The Package and Class are used to represent the type of structure being referenced.

==URN Identifiers==
The URN Namespace identifies which type of Structure is being referenced, i.e., a Codelist, Data Structure, Concept, etc.

The subsequent properties in the URN Namespace are used to Identify the structure, starting with the [[Maintainable V11]] identifiers, which are:
# Agency Id
# Identifiable
# Version

[[File:URN Maintainable.png|700px]]

If the structure is not a [[Maintainable V11|Maintainable]] structure, but instead lives inside one, i.e a Code which lives inside a Codelist or a Dimension which lives inside a Data Structure, then the remainder of the URN is used to identify the structure inside the Maintainable parent.

[[File:URN Identifiable.png|700px]]

For structures which live nested inside others, i.e., a Category which is a number of levels deep, the nesting is part of the URN identification.

'''<nowiki> urn:sdmx:org.sdmx.infomodel.categoryscheme.Category=SDMX:STAT_SUBJECT_MATTER(1.0).DEMO_SOCIAL_STAT.HEALTH</nowiki>''' 

Where the '''HEALTH''' Category is a child of the '''DEMO_SOCIAL_STAT''' Category.

== URN Examples ==
{| class="wikitable sortable"
|-
! Structure Type !! Package !! Class !! Parent !! Example
|-
| Agency Scheme|| base || AgencyScheme || - || <nowiki>urn:sdmx:org.sdmx.infomodel.base.AgencyScheme=SDMX:AGENCIES(1.0)</nowiki>
|-
| Agency || base|| Agency || AgencyScheme || <nowiki>urn:sdmx:org.sdmx.infomodel.base.Agency=SDMX</nowiki>
|-
| Data Provider Scheme || base|| DataProviderScheme || - || <nowiki>urn:sdmx:org.sdmx.infomodel.base.DataProviderScheme=FAO:DATA_PROVIDERS(1.0</nowiki>)
|-
| Data Provider || base|| DataProvider || DataProviderScheme || <nowiki>urn:sdmx:org.sdmx.infomodel.base.DataProvider=FAO:DATA_PROVIDERS(1.0).FR2</nowiki>
|-
| Data Consumer Scheme || base|| DataConsumerScheme || - || <nowiki>urn:sdmx:org.sdmx.infomodel.base.DataConsumerScheme=FAO:DATA_CONSUMERS(1.0)</nowiki>
|-
| Data Consumer || base|| DataConsumer || DataConsumerScheme || <nowiki>urn:sdmx:org.sdmx.infomodel.base.DataConsumer=FAO:DATA_CONSUMER(1.0).FR2</nowiki>
|-
| Organisation Unit Scheme || base || OrganisationUnitScheme || - || <nowiki>urn:sdmx:org.sdmx.infomodel.base.OrganisationUnitScheme=FAO:UNITSCHEME_A(1.0)</nowiki>
|-
| Organisation Unit|| base || OrganisationUnit || OrganisationUnitScheme || <nowiki>urn:sdmx:org.sdmx.infomodel.base.OrganisationUnit=FAO:UNITSCHEME_A(1.0).STATS</nowiki>
|-
| Categorisation || categoryscheme|| Categorisation || - || <nowiki>urn:sdmx:org.sdmx.infomodel.categoryscheme.Categorisation=IMF:CAT1(1.0)</nowiki>
|-
| Category Scheme || categoryscheme|| CategoryScheme || - || <nowiki>urn:sdmx:org.sdmx.infomodel.categoryscheme.CategoryScheme=SDMX:STAT_SUBJECT_MATTER(1.0)</nowiki>
|-
| Category || categoryscheme|| Category || CategoryScheme || <nowiki>urn:sdmx:org.sdmx.infomodel.categoryscheme.Category=SDMX:STAT_SUBJECT_MATTER(1.0).DEMO_SOCIAL_STAT</nowiki>
|-
| Reporting Taxonomy || categoryscheme|| ReportingTaxonomy || - || <nowiki>urn:sdmx:org.sdmx.infomodel.categoryscheme.ReportingTaxonomy=SDMX:RT(1.0)</nowiki>
|-
| Reporting Category|| categoryscheme|| ReportingCategory || ReportingTaxonomy || <nowiki>urn:sdmx:org.sdmx.infomodel.categoryscheme.ReportingCategory=SDMX:RT(1.0).CAT1</nowiki>
|-
| Concept Scheme || conceptscheme || ConceptScheme || -|| <nowiki>urn:sdmx:org.sdmx.infomodel.conceptscheme.ConceptScheme=SDMX:CROSS_DOMAIN_CONCEPTS(2.0)</nowiki>
|-
| Concept || conceptscheme || Concept || ConceptScheme || <nowiki>urn:sdmx:org.sdmx.infomodel.conceptscheme.Concept=SDMX:CROSS_DOMAIN_CONCEPTS(2.0).FREQ</nowiki>
|-
| Codelist || codelist || Codelist || - || <nowiki>urn:sdmx:org.sdmx.infomodel.codelist.Codelist=SDMX:CL_FREQ(2.0)</nowiki>
|-
| Code || codelist || Code || Codelist || <nowiki>urn:sdmx:org.sdmx.infomodel.codelist.Code=SDMX:CL_FREQ(2.0).A</nowiki>
|-
| Data Structure || datastructure || DataStructure || - || <nowiki>urn:sdmx:org.sdmx.infomodel.datastructure.DataStructure=ESTAT:CPI(1.1)</nowiki>
|-
| Attribute List || datastructure || AttributeDescriptor || DataStructure || <nowiki>urn:sdmx:org.sdmx.infomodel.datastructure.AttributeDescriptor=ESTAT:CPI(1.1).AttributeDescriptor</nowiki>
|-
| Attribute || datastructure || DataAttribute || AttributeDescriptor || <nowiki>urn:sdmx:org.sdmx.infomodel.datastructure.DataAttribute=ESTAT:CPI(1.0).DISS_ORG</nowiki>
|-
| Dimension List || datastructure || DimensionDescriptor || DataStructure || <nowiki>urn:sdmx:org.sdmx.infomodel.datastructure.DimensionDescriptor=ESTAT:CPI(1.1).DimensionDescriptor</nowiki>
|-
| Dimension || datastructure || Dimension || DimensionDescriptor || <nowiki>urn:sdmx:org.sdmx.infomodel.datastructure.Dimension=ESTAT:CPI(1.0).FREQ</nowiki>
|-
| Group List || datastructure || GroupDimensionDescriptor || DataStructure || <nowiki>urn:sdmx:org.sdmx.infomodel.datastructure.GroupDimensionDescriptor=ESTAT:CPI(1.1).SiblingGroup</nowiki>
|-
| Measure List || datastructure || MeasureDescriptor || DataStructure || <nowiki>urn:sdmx:org.sdmx.infomodel.datastructure.MeasureDescriptor=ESTAT:CPI(1.1).MeasureDescriptor</nowiki>
|-
| Primary Measure || datastructure || PrimaryMeasure || MeasureDescriptor || <nowiki>urn:sdmx:org.sdmx.infomodel.datastructure.PrimaryMeasure=ESTAT:CPI(1.1).OBS_VALUE</nowiki>
|-
| Time Dimension|| datastructure || TimeDimension || DataStructure || <nowiki>urn:sdmx:org.sdmx.infomodel.datastructure.TimeDimension=ESTAT:CPI(1.0).TIME_PERIOD</nowiki>
|-
| Metadata Structure|| metadatastructure || MetadataStructure || - ||
|-
| Report Structure|| metadatastructure || ReportStructure|| MetadataStructure ||
|-
| Metadata Attribute|| metadatastructure|| MetadataAttribute || ReportStructure ||
|-
| Metadata Target || metadatastructure|| MetadataTarget || MetadataStructure ||
|-
| Metadata Target (URN) || metadatastructure || IdentifiableObjectTarget || MetadataTarget ||
|-
| Metadata Target (Dimension Values) || metadatastructure|| DimensionDescriptorValuesTarget || - ||
|-
| Metadataflow || metadatastructure|| Metadataflow || - ||
|-
| Process || process || Process|| - ||
|-
| Process Step|| process || ProcessStep || Process ||
|-
| Transition || process || Transition || ProcessStep ||
|-
| Provision Agreement|| registry || ProvisionAgreement || - ||
|-
| Content Constraint || registry|| ContentConstraint || - ||
|-
| Structure Set || mapping || StructureSet || - ||
|-
| Structure Map || mapping || Structure Map|| StructureSet ||
|-
| Category Scheme Map || mapping || CategorySchemeMap || StructureSet ||
|-
| Concept Scheme Map || mapping || ConceptSchemeMap || StructureSet ||
|-
| Codelist Map || mapping || CodelistMap|| StructureSet ||
|-
| Component Map || mapping || ComponentMap || StructureSet ||
|-
| Organisation Scheme Map|| mapping || OrganisationSchemeMap || StructureSet ||
|}

Transform data into a different structure using Structure Maps

2024-04-17T15:45:18Z

Vmurrell: /* Step 2 - Create a Structure Set */

[[Category:How_To]]
=Important note=
The following information is for version 10 of Fusion Registry and a large amount of it is not relevant for version 11

=Use Case 1 - using Fixed Values=
In this scenario we have a simple [[Data_Structure_Definitions|Data Structure]] with a number of [[Component|Components]]. They are all [[Dimension|Dimensions]] except the Observation Value which is a [[Measure V10|measure]].

The image below shows that 3 Components are '''Enumerated''' and the Codes which are in the linked [[Codelists|Codelists]].

[[File:DSMMAP1.PNG|Viewing the components in a simple DSD|1200px]] 

If a data file is to succeed the Validation process it would have to conform to these codes. However, it this example the source file supplied does not conform instead it contains the codes:

* FR was used instead of FRA for France 

* ANN was used instead of A for Annual 

* ARAB was used instead of ARA for Arable Land 

To successfully validate this data, a Data Structure Map can be created to map the source data to the Target Data Structure. To Achieve this...

==Step 1 - Create a new Data Structure for the Source==

In this example the Source data structure can be exactly the same as the Target however instead of '''Enumerating''' the '''Concepts''' you can allow any format to be used as shown in the example below.

[[File:DSMMAP2.PNG|DSD - Step 3|1200px]] 

==Step 2 - Create a Structure Set==

All maps must belong to a [[Structure_Set|Structure Set]]. A logged-on user with the appropriate privileges can select '''Structure Maps''' from the Main Menu followed by Structure Sets and create a Structure Set.

[[File:DSMMAP3.PNG|Structure Set|600px]] 

==Step 3 - Create a Data Structure Map==

===Complete Step 1 of the Wizard - High Level Details===

From the '''Structure Maps''' Menu, select '''Data Structure Map''' and enter the high-level details as shown below.

[[File:DSMMAP4.PNG|DS Map - Step 1|600px]] 

===Complete Step 2 of the Wizard - Select the Source and Target Data Structures===

[[File:DSMMAP5.PNG|DS Map - Step 2|1200px]] 

===Complete Step 3 of the Wizard - Map the Components===

In this example I am using the Target Fixed Value area to specify that no matter what is in the Source file for Frequency, Series and Country, the Frequency will be mapped to Annual, the Series will be mapped to ARA and the Country will be mapped to France in the Target.

[[File:DSMMAP6B.PNG|DS Map - Step 3|1200px]] 

For the TIME_PERIOD and OBS_VALUE Components, these can be mapped directly as shown above.
After this step, the Structure Map can be '''Finished'''.

==Step 4 - Test Mappings==

Once the Data Structure Map has been created you will see the orange '''Test''' '''Mapping''' button as shown below.

[[File:DSMMAP7.PNG|Test Mapping|1200px]] 

To test the mapping, you will need to select a file containing the data in the format specified above namely:

FR instead of FRA for France
ANN instead of A for Annual
ARAB instead of ARA for Arable Land

Once the file is loaded, you will see the '''Explain''' '''Plan'''.

[[File:DSMMAP8.PNG|Explain Plan|1200px]] 

The Plan will show you exactly what will happen when this Map is used.

==Step 5 - Load the Source Data File==
Using the Convert menu option load the data file, you will see Validation Errors as shown below which is to be expected.

[[File:DSMMAP9.PNG|Loading source data showing errors|1200px]] 

==Step 6 - Convert the Source Data File==

Having Loaded the Data, the next step is to click the Convert button as shown in the image below. Note that I have selected the Data Structure Map. On export the dataset will be converted in terms of its structure, and representation e.g., the data set can be converted to conform to the target Data Structure Definition.

[[File:DSMMAP10.PNG|Convert the source data by using the DS Map|500px]] 

==Step 7 - Load the Converted Data File==

This time the converted data file will achieve a successful validation.

=Using this example=

==1. Load the Example Structures==
Download this file [https://www.dropbox.com/s/6hb0plox1xhc205/Structures.zip?dl=1 Structures Zip File]. Note, you may need to create an Agency with the ID of MY_AGENCY first.

==2. Load the Example Structure Set==
Download this file [https://www.dropbox.com/s/m90bbyyf5glf1y5/All_Structure_Set.zip?dl=1 Structure Set Zip File]. Note, you will need to have loaded the Structures first.

==3. Use the Test Mapping File==
Note that you need to have loaded the Structures and the Structure set first before you can test the mapping.

1. Download this file [https://www.dropbox.com/s/44dk49wbmx0v7sf/data.xml?dl=1 which you can use to test the Mapping].

2. From the Structure Maps menu, select Data Structure Maps and then select MY_AGENCY:SDM1. Once selected, click '''Test Mapping'''.

3. Select the Data file you downloaded click '''Start Test'''. The Explain Plan will be displayed.

You can now follow the steps detailed above [[Transform_data_into_a_different_structure_using_Structure_Maps#Step_5_-_Load_the_Source_Data_File|from Step 5]].

Transform data into a different structure using Structure Maps

2024-04-17T15:44:16Z

Vmurrell: /* Use Case 1 - using Fixed Values */

[[Category:How_To]]
=Important note=
The following information is for version 10 of Fusion Registry and a large amount of it is not relevant for version 11

=Use Case 1 - using Fixed Values=
In this scenario we have a simple [[Data_Structure_Definitions|Data Structure]] with a number of [[Component|Components]]. They are all [[Dimension|Dimensions]] except the Observation Value which is a [[Measure V10|measure]].

The image below shows that 3 Components are '''Enumerated''' and the Codes which are in the linked [[Codelists|Codelists]].

[[File:DSMMAP1.PNG|Viewing the components in a simple DSD|1200px]] 

If a data file is to succeed the Validation process it would have to conform to these codes. However, it this example the source file supplied does not conform instead it contains the codes:

* FR was used instead of FRA for France 

* ANN was used instead of A for Annual 

* ARAB was used instead of ARA for Arable Land 

To successfully validate this data, a Data Structure Map can be created to map the source data to the Target Data Structure. To Achieve this...

==Step 1 - Create a new Data Structure for the Source==

In this example the Source data structure can be exactly the same as the Target however instead of '''Enumerating''' the '''Concepts''' you can allow any format to be used as shown in the example below.

[[File:DSMMAP2.PNG|DSD - Step 3|1200px]] 

==Step 2 - Create a Structure Set==

All maps must belong to a [[Structure_Mapping_Sets|Structure Set]]. A logged-on user with the appropriate privileges can select '''Structure Maps''' from the Main Menu followed by Structure Sets and create a Structure Set.

[[File:DSMMAP3.PNG|Structure Set|600px]] 

==Step 3 - Create a Data Structure Map==

===Complete Step 1 of the Wizard - High Level Details===

From the '''Structure Maps''' Menu, select '''Data Structure Map''' and enter the high-level details as shown below.

[[File:DSMMAP4.PNG|DS Map - Step 1|600px]] 

===Complete Step 2 of the Wizard - Select the Source and Target Data Structures===

[[File:DSMMAP5.PNG|DS Map - Step 2|1200px]] 

===Complete Step 3 of the Wizard - Map the Components===

In this example I am using the Target Fixed Value area to specify that no matter what is in the Source file for Frequency, Series and Country, the Frequency will be mapped to Annual, the Series will be mapped to ARA and the Country will be mapped to France in the Target.

[[File:DSMMAP6B.PNG|DS Map - Step 3|1200px]] 

For the TIME_PERIOD and OBS_VALUE Components, these can be mapped directly as shown above.
After this step, the Structure Map can be '''Finished'''.

==Step 4 - Test Mappings==

Once the Data Structure Map has been created you will see the orange '''Test''' '''Mapping''' button as shown below.

[[File:DSMMAP7.PNG|Test Mapping|1200px]] 

To test the mapping, you will need to select a file containing the data in the format specified above namely:

FR instead of FRA for France
ANN instead of A for Annual
ARAB instead of ARA for Arable Land

Once the file is loaded, you will see the '''Explain''' '''Plan'''.

[[File:DSMMAP8.PNG|Explain Plan|1200px]] 

The Plan will show you exactly what will happen when this Map is used.

==Step 5 - Load the Source Data File==
Using the Convert menu option load the data file, you will see Validation Errors as shown below which is to be expected.

[[File:DSMMAP9.PNG|Loading source data showing errors|1200px]] 

==Step 6 - Convert the Source Data File==

Having Loaded the Data, the next step is to click the Convert button as shown in the image below. Note that I have selected the Data Structure Map. On export the dataset will be converted in terms of its structure, and representation e.g., the data set can be converted to conform to the target Data Structure Definition.

[[File:DSMMAP10.PNG|Convert the source data by using the DS Map|500px]] 

==Step 7 - Load the Converted Data File==

This time the converted data file will achieve a successful validation.

=Using this example=

==1. Load the Example Structures==
Download this file [https://www.dropbox.com/s/6hb0plox1xhc205/Structures.zip?dl=1 Structures Zip File]. Note, you may need to create an Agency with the ID of MY_AGENCY first.

==2. Load the Example Structure Set==
Download this file [https://www.dropbox.com/s/m90bbyyf5glf1y5/All_Structure_Set.zip?dl=1 Structure Set Zip File]. Note, you will need to have loaded the Structures first.

==3. Use the Test Mapping File==
Note that you need to have loaded the Structures and the Structure set first before you can test the mapping.

1. Download this file [https://www.dropbox.com/s/44dk49wbmx0v7sf/data.xml?dl=1 which you can use to test the Mapping].

2. From the Structure Maps menu, select Data Structure Maps and then select MY_AGENCY:SDM1. Once selected, click '''Test Mapping'''.

3. Select the Data file you downloaded click '''Start Test'''. The Explain Plan will be displayed.

You can now follow the steps detailed above [[Transform_data_into_a_different_structure_using_Structure_Maps#Step_5_-_Load_the_Source_Data_File|from Step 5]].

Tomcat setenv file

2024-04-17T15:41:40Z

Vmurrell:

[[Category:Installation_and_Configuration]]

[[Category:FMR_Configuration_Reference]]
Configuration for Fusion Metadata Registry is principally set by creating a ''setenv.bat'' or ''setenv.sh'' script in the Tomcat ''bin'' directory.

Below are examples for Windows and Linux.

==Windows==
tomcat\bin\setenv.bat 
<samp>set "JAVA_OPTS=%JAVA_OPTS% -Xmx6G -server -DRegistryProperties=file:///C:/Tomcats/Environments/fusion_registry.properties" 
set CLASSPATH=c:\oracle\ojdbc8.jar</samp>

==Linux==
tomcat/bin/setenv.sh 
<samp>
<nowiki>#!/bin/sh</nowiki> 
export JAVA_OPTS="-Xmx6G -server -DRegistryProperties=file:\\opt\tomcats\environments\fusion_registry.properties" 
export CLASSPATH="/opt/oracle/ojdbc8.jar"
</samp>

==CLASSPATH Settings for Oracle==
The CLASSPATH environment variable tells the Java Virtual Machine where to look for additional classes.

If you want to use an Oracle database, you must download the JDBC driver directly from Oracle and reference the location of the ''jar'' by setting the CLASSPATH variable as illustrated in the example.

CLASSPATH=/home/oracle/jdbc/ojdbc8.jar

==CLASSPATH Settings for MySQL==
The following adds a specific location for a MySql connector jar file to the java Classpath:

set CLASSPATH=C:\SQL_Connector\mysql-connector-java-8.0.30\mysql-connector-java-8.0.30.jar

==JAVA_OPTS Configuration Parameters==
JAVA_OPTS parameters configure the Java virtual machine, and also allow system parameters to be passed to the Fusion Metadata Registry application to alter its behaviour.
{| class="wikitable"
|-
! style="width:20%" | Parameter !! Purpose !! Example
|-
| -Xmx || Sets the maximum amount of memory that can be allocated to the Java heap. This should be a minimum of 4GB and ideally about 75% of available memory on the machine. Setting too low may result in Java 'out of memory' errors. || <code>-Xmx4GB</code> sets the maximum heap size to 4GB
|-
| -Xms || Sets the initial amount of memory to be allocated to the Java heap. When running in production, there may be a performance benefit to setting this to the same value as -Xmx which avoids the need to grow the heap as memory demand grows. || <code>-Xms128m</code> sets the initial heap to 128MB
|-
| -server || Improves performance by optimising the way the Java Virtual Machine works. ||
|-
| -DRegistryProperties || System Property: Sets a specific path for the Fusion Registry properties file. If not set, the [[Registry_Properties_File|default location]] is used || <code>-DRegistryProperties=file:\\opt\tomcats\environments\fusion_registry.properties</code> In Linux, sets the properties file location to the path specified.
|-
| -Djson.legacy.enabled=true or -Djson.legacy.enabled=false (default) || System Property: Sets whether the Fusion Registry instance uses the legacy Metadata Technology JSON dialect, or the [https://github.com/sdmx-twg/sdmx-json official SDMX-JSON version 1.0 Structure and Data Exchange format]. 
By default, or if the property is explicity set to ''false'', JSON returned will conform to the official SDMX-JSON specification when: 
* JSON is chosen as a format in the web user interface when exporting data or structures;
* The HTTP 'Accept' header is set to 'application/vnd.sdmx.json' when making a REST web service call; or
* The URL format parameter is set to 'sdmx-json' when making a REST web service call. 

If the property to ''true'', JSON returned will conform to Metadata Technology's legacy JSON format for structures and data. Use this for backward compatibility where existing client-side applications rely on the legacy JSON format.
|| <code>-Djson.legacy.enabled=true</code> Force Fusion Registry to produce and accept structures and data in the legacy Metadata Technology dialect
|-
| -DLicenseFile= (Data Essentials '''only''') || Sets the location of the License file and the Public key which is used in Data Essentials. || <code>-DLicenseFile=file:///C:/tomcats/RegistryProperties/8080_LICENSE/fusionLicense.fl<code> sets the location to a specific location on the C drive
|-
|}

Time Period Format Validation

2024-04-17T15:40:23Z

Vmurrell:

[[Category:Installation_and_Configuration]]

[[Category:FMR_Configuration_Reference]]
The Registry will validate observation time periods for consistency with the series frequency value in the FREQ dimension.

This check is not performed if the DSD has no FREQ dimension.

Default frequencies and their corresponding valid time period formats are set out in the following table:
{| class="wikitable"
|-
! Format !! Frequency Code !! Example Usage
|-
| Yearly || A || 2001
|-
| Half-Of-Year || S || 2001-S1
|-
| Third-Of-Year || T || 2001-T3
|-
| Quarter-Of-Year || Q || 2001-Q4
|-
| Month || M || 2001-12
|-
| Week || W || 2001-W52
|-
| Date || D || 2001-12-31
|-
| Hour || H || 2001-12-31T23
|-
| Date Time || I || 2001-12-31T23:59:59
|}

== Using 'H' for half-year (semiannual) ==

At ESCB level SDMX data exchanges, there is an agreement to use ‘H’ as the frequency value for semiannual observations with ‘YYYY-S1’ for TIME_PERIOD format.

The Registry supports this by setting 'alternate half year' mode under which:
* Frequency value H means half-year (semiannual or semester)
* The hourly frequency is not supported
* 'YYYY-S1' and 'YYYY-H1' are valid TIME_PERIOD formats for half-yearly observations, however time periods are standardised to ‘YYYY-S1’ on conversion (e.g. such as when converting data in Excel to SDMX 2.1 structure specific)

Alternate half year mode is controlled at system level on Registry start-up by setting the Java System variable “'''TF_ALTERNATE_HALF_YEAR'''” to “'''true'''”.

If using Apache Tomcat, JAVA_OPTS can be set in the setenv.bat or setenv.sh file:

SET JAVA_OPTS=-DTF_ALTERNATE_HALF_YEAR=true
(For Windows systems)

export JAVA_OPTS=-DTF_ALTERNATE_HALF_YEAR=true
(For Unix systems)

Different procedures may be required for other web application servers like JBoss.

The Fusion Registry system log will report the following message:

2021-12-31 23:59:59.123 INFO main io.sdmx.core.fmr.application.FusionRegistryApplicationInitialiser - Using 'Alternate Half-Year' Setting

Active system properties can also be queried using the REST API [[System_Properties|here]].

== Is Alternate Half-Format set ==

On startup the Fusion Registry system log will report the following message:

2021-12-31 23:59:59.123 INFO main io.sdmx.core.fmr.application.FusionRegistryApplicationInitialiser - Using 'Alternate Half-Year' Setting

Active system properties can also be queried using the REST API which is detailed [[System_Properties|here]].

Time Format Alternate Half Year

2024-04-17T15:39:37Z

Vmurrell:

[[Category:Installation_and_Configuration]]

[[Category:FMR_Configuration_Reference]]
At ESCB level SDMX data exchanges, there is an agreement to use 'H' as the frequency value for semi-annual observations. This is specified via 'YYYY-H1' or 'YYYY-H2'.

The Registry supports this with a system property. When this is enabled the Registry is in 'alternate half year' mode. Under this mode:
* Frequency value 'H' means half-year (semiannual or semester) as does frequency value 'S'. They are interchangeable as input.
* 'YYYY-S1' and 'YYYY-H1' are valid TIME_PERIOD formats for half-yearly observations.
* Output in SDMX-ML 2.1 will always be in the 'S' format (e.g. an input of 'YYYY-H1' will be converted to 'YYYY-S1'). This ensures that the output is correct to the SDMX specification.
* Output in other formats will not be converted from 'H' to 'S' (e.g. SDMX-CSV)
* The hourly frequency is not supported.

'Alternate half year' mode is controlled at system level on Registry start-up by setting the Java System variable "TF_ALTERNATE_HALF_YEAR" to "true".

== Setting System Property ==

If using Apache Tomcat, it is recommended to set the value via JAVA_OPTS and using the ''setenv.bat'' or ''setenv.sh'' file:

SET JAVA_OPTS=-DTF_ALTERNATE_HALF_YEAR=true
(For Windows systems)

export JAVA_OPTS=-DTF_ALTERNATE_HALF_YEAR=true
(For Unix systems)

Different procedures may be required for other web application servers like JBoss.

== Is Alternate Half-Year Format set? ==

On startup the Fusion Registry system log will report the following message:

2021-12-31 23:59:59.123 INFO main io.sdmx.core.fmr.application.FusionRegistryApplicationInitialiser - Using 'Alternate Half-Year' Setting

Active system properties can also be queried using the REST API which is detailed [[System_Properties|here]].

System Properties

2024-04-17T15:38:44Z

Vmurrell:

__NOTOC__
[[Category:How_To]]
[[Category:How_To V11]]
= Overview =

System Properties are generally set when the Java virual machine that is running Fusion Registry is launched. However the Registry provides a way to list all of the System properties and also provides the ability to set their values. Note that setting some of the system properties will have no effect on the running Registry as these must be supplied at startup time . The properties that can be changed and for the change to have an effect are listed below.

= Obtaining System Properties =
A root or an admin user of Fusion Registry can access the System Properites, returned in JSON form, by performing a GET request to the url:

<code>
http://[server]:[port]/FusionRegistry/ws/secure/settings/getSystemProperties
</code>

As this is a secure Web Service the user must be logged on, or supply correct credentials. The result is an array of JSON objects, where each object has a Key and a Value For example:

<pre>
{
Key: "edi.lenient.enabled",
Value: "true"
}
</pre>

= Setting System Properties =

A root or an admin user of Fusion Registry can set a System Properites by performing a POST request to the url:

<code>
http://[server]:[port]/FusionRegistry/ws/secure/settings/setSystemProperty
</code>

Appropriate credential must be supplied. The input format is JSON of the form
<pre>
{
"Key": "edi.lenient.enabled",
"Value": "false"
}
</pre>

If the "value" specified is blank this will have the effect of removing the property. For example the following will remove the property "ENFORCE_STRICT_SDMX_2.1":

<pre>
{
"Key": "ENFORCE_STRICT_SDMX_2.1",
"Value": ""
}
</pre>

= Properties which may be Dynamically changed =

Changing a property whilst the Registry is running may not have any effect. The following properties can be changed and their effect will be immediate:

{| class="wikitable"
|-
! Property !! Default Value !! Effect
|-
| edi.lenient.enabled || false || [[Edi_Lenient_Mode|Sets the EDI Leniency of the Registry]]
|-
| edi.enforce.primarymeasure.enabled || false || [[Edi_Enforce_Primary_Measure|Sets whether a Primary Measure should be enforced in EDI]]
|-
| edi.forceTimeFormatSeriesAttr.enabled || false || [[Edi_Create_Time_Format|Sets whether a series attribute of TIME_FORMAT should be created]]
|-
| edi.skipagencycheck.enabled || false || When writing an EDI data file, does not check that the Agencies of the Dataflow and DSD match. When writing an EDI structure file, does not check that cross-referenced components are of the same aganecy as the DSD. This feature is part of EDI Leniency mode.
|-
| edi.dsi.dataflow.enabled || false || When reading an EDI file, determines whether the DSI segment should be used to refer to a Dataflow
|-
| ENFORCE_STRICT_SDMX_2.1 || false || Puts the Registry into Enforcing Strict values for SDMX 2.1
 Week data is of the format: YYYY-Www
 Monthly data is of the format: YYYY-Mmm
 Half-Yearly data is of the format: YYYY-S[1-2]
|-
|edi.datagroup.identifier || unset || [[Edi_DataGroup_Identifier|Sets how the Registry sets the Group name when reading an EDI data file]]
|-
| TF_ALTERNATE_HALF_YEAR || false || [[Time_Format_Alternate_Half_Year|Sets the Registry into Alternate Half Year mode]]
|-
| ENFORCE_STRICT_SDMX_2.1 || true || [[EDI_Enforce_Strict_SDMX_2.1|Enforces Strict mode]]
|-
| edi.dsi.dataflow.enabled || true || [[EDI_dsi.dataflow_enabled|Switches the DSI from refering to the Datafllow]]
|-
| edi.skipagencycheck.enabled || true || [[EDI_skipagencycheck_enabled|Ignores the Agency check when evaluating the DSI file]]
|}

Structure Set

2024-04-17T15:36:40Z

Vmurrell: /* Structure Properties */

[[Category:SDMX 2.1 Structures]]

A Structure Set is a maintainable collection of Structure Maps that link Components together in a source/target relationship where there is a semantic equivalence between the source and the target Components.

There are two main types of Structure Map: 

Maps between item schemas
* Codelist Map
* Concept Scheme Map

Maps to change the dimensionality of datasets
* Data Structure Map
* Dataflow Map

A typical use of Structure Sets is to provide mappings between an SDMX data structure used in an internal system with an SDMX structure of an external dataset when imported to or exported from the internal system. Where a dataset's internal [[Data Structure Definition]] is too complex for public dissemination, Structure Maps can be used to map to a Data Structure Definition with fewer dimensions and attributes.

==Structure Properties==
{| class="wikitable"
! scope=row style="text-align: left;" | Structure Type
| Standard SDMX Structural Metadata Artefact
|-
! scope=row style="text-align: left;" | Maintainable
| [[Maintainable V10|Yes]]
|-
! scope=row style="text-align: left;" | Identifiable
| [[Identifiable V10|Yes]]
|-
! scope=row style="text-align: left;" | Item Scheme
| [[Item_Scheme V10|No]]
|-
! scope=row style="text-align: left;" | SDMX Information Model Versions
| 2.0, 2.1
|-
! scope=row style="text-align: left;" | URN - StructureSet namespace
| <nowiki>urn:sdmx:org.sdmx.infomodel.mapping.Structureset</nowiki>
|-
|}

==Context within the SDMX 2.1 Information Model==

[[File:L_AllStructureSets.png|Structure Set|600px]]

Structure Set

2024-04-17T15:36:24Z

Vmurrell: /* Structure Properties */

[[Category:SDMX 2.1 Structures]]

A Structure Set is a maintainable collection of Structure Maps that link Components together in a source/target relationship where there is a semantic equivalence between the source and the target Components.

There are two main types of Structure Map: 

Maps between item schemas
* Codelist Map
* Concept Scheme Map

Maps to change the dimensionality of datasets
* Data Structure Map
* Dataflow Map

A typical use of Structure Sets is to provide mappings between an SDMX data structure used in an internal system with an SDMX structure of an external dataset when imported to or exported from the internal system. Where a dataset's internal [[Data Structure Definition]] is too complex for public dissemination, Structure Maps can be used to map to a Data Structure Definition with fewer dimensions and attributes.

==Structure Properties==
{| class="wikitable"
! scope=row style="text-align: left;" | Structure Type
| Standard SDMX Structural Metadata Artefact
|-
! scope=row style="text-align: left;" | Maintainable
| [[Maintainable V10|Yes]]
|-
! scope=row style="text-align: left;" | Identifiable
| [[Identifiable V10|Yes]]
|-
! scope=row style="text-align: left;" | Item Scheme
| [[Item_Scheme|No]]
|-
! scope=row style="text-align: left;" | SDMX Information Model Versions
| 2.0, 2.1
|-
! scope=row style="text-align: left;" | URN - StructureSet namespace
| <nowiki>urn:sdmx:org.sdmx.infomodel.mapping.Structureset</nowiki>
|-
|}

==Context within the SDMX 2.1 Information Model==

[[File:L_AllStructureSets.png|Structure Set|600px]]

Structure Map

2024-04-17T15:34:27Z

Vmurrell: /* Structure Properties */

[[Category:SDMX 3.0 Structures]]
[[Category:SDMX 3.0 Mapping Structures]]

= Overview =
[[File:StructureMapModel.jpg|thumb|Simple example of a structure map]]

==Structure Properties==
{| class="wikitable"
! scope=row style="text-align: left;" | Structure Type
| Standard SDMX Structural Metadata Artefact
|-
! scope=row style="text-align: left;" | Maintainable
| [[Maintainable V11|Yes]]
|-
! scope=row style="text-align: left;" | Identifiable
| [[Identifiable V11|Yes]]
|-
! scope=row style="text-align: left;" | Item Scheme
| [[Item_Scheme V11|No]]
|-
! scope=row style="text-align: left;" | SDMX Information Model Versions
| 3.0
|-
! scope=row style="text-align: left;" | Concept ID
| StructureMap
|}

==Context within the SDMX 3.0 Information Model==

[[File:SDMX3 StructureMap.png|Structure Map|600px]]

==Usage==
SDMX Version 3.0 Structure Mapping provides the ability to define a relationship between datasets conforming to a source DSD, to datasets which conform to a target DSD. This relationship allows for the automatic conversion of data from one structure to another. For example a source dataset may contain 8 Dimensions and use certain coding schemes, which may map to a dataset with only 5 Dimensions using different coding schemes. 
 
Structure Mapping does not create new data, it should '''not''' be thought of as a mechanism to aggregate data, only to re-organise and re-code it. 
 
Two Maintainable Structures are required to defining how two datasets relate to each other, one is the SDMX Structure Map which is used to define how Components from the source DSD relate to Components on the target DSD, and the other is the [[Representation_Map|Representation Map]] which is used to describe how values reported for source Components should be converted to conform to the desired output DSD. 
 
A simple example is a relationship between the source COUNTRY component to the target REF_AREA component. The Structure Map has a source COUNTRY and target REF_AREA, the rules used to define how the values are mapped are maintained in the corresponding [[Representation_Map|Representation Map]], an example rule would be GB maps to GBR, US maps to USA, and UY maps to URY.

= Structure Map - Model =
== Source and Target DSD / Dataflow ==
The Structure Map defines the source and target DSD or if the mapping is Dataflow specific, the source and target Dataflow. The source DSD/Dataflow is the one from which the data will be input into the mapping. Whilst the mapping rules are bi-directional (data mapped one way can be mapped back again) for more complex mappings, which include regular expression matches or substring matches on the source, it is not always possible to map back again. Therefore the source DSD or Dataflow should be selected based on where the data is coming from, and the target is where the data is going to as a result of the mapping.

== Component Maps ==
The Structure Map defines 1 or more Component Maps, each Component Map has 1 or more Components from the source DSD, mapping to 1 or more Components in the target DSD.

=== 1 to 1 Mapping ===
The most simple relationship is 1:1 mapping, an example being REF_AREA maps to COUNTRY.

=== 1 to n Mapping ===
More complex relationships allow for the combination of reported values across multiple source components to influence what is output in the target, for example REF_AREA in combination with CURRENCY maps to CURRENCY_DENOM.

=== n to 1 Mapping ===
Component Maps may define single source mapping to multiple targets, for example REF_AREA maps to CURRENCY and CURRENCY_DEMOM.

=== n to n Mapping ===
Finally multiple sources may map to multiple targets. A full description of how mapping relationships are used to solve use cases is provided in the mapping relationships section.

=== Defining how values map ===
[[File:Two-set Venn diagram example.jpg|thumb|Finding the mapped output using the intersection of each output set]]
Each Component Map can link to a [[Representation_Map|Representation Map]], which is used to describe how the source values map to the target values. The linked Representation Map links to source and target Codelists, Valuelists, or Free text. Like the Component Map, the Representation Map may contain multiple sources and multiple targets. The number and order of sources and targets to a Representation Map must match exactly that of the Component Map. For example if a Component Map has 2 sources REF_AREA and CURRENCY, then the linked Representation map must also have 2 sources, one for the REF_AREA Codelist and the other for the CURRENCY Codelist.

Representation Maps can include complex rules, such as regular expressions on source values, and can even define periods of time for which a mapping relationship is true, for example if a relationship between source country and target currency is defined then one could map France to the French Franc up until 2002 and then map France to the Euro from 2002 onwards.

If a values do not require mapping, for example if source FREQ maps to target FREQ and the values are the same in both the source and target DSD, then the Component Map should not link to a Representation Map. The lack of link will inform the system that the value should be copied across verbatim.

== Time Mapping ==

A Structure Map may also define relationships between a source Component and target Component where the value in the source is a representation of time, which needs to be converted to conform to SDMX Time Formatting.

The output of a time mapping is always a date in SDMX format. The date must be mapped to a specific Frequency formatting, this is either defined as a fixed value, or can be linked to the value in another Dimension. For example output Frequency=value reported in FREQ Dimension (in the target DSD).

Time mapping is split into two separate types, Epoch Mapping and Time Pattern Mapping.

=== Epoch Mapping ===
This is used if the source Component represents time as a number, the number represents the number of epochs since a base period, and each epoch is a fixed interval of time. An example is UNIX time which is the number of milliseconds since 1970. When defining this mapping, the source Component requires the base period (e.g. 1970), the epoch interval (milliseconds) and the output Frequency (fixed value or based on another dimension value).

=== Time Pattern Mapping ===
This is used if the source Component represents time as a string which conforms to a particular pattern, for example mm-dd-yyy (month, day, year). Each pattern can be mapped to a specific frequency, or the general rule of outputting according to a Frequency Dimension can be used.

The following patterns are supported:

{| class="wikitable"
|-
! Letter !! Date or Time Component !! Presentation !! Examples

|-
| G || Era designator || Text || AD
|-
| y || Year || Year || 1996; 96
|-
| Y || Week year || Year || 2009; 09
|-
| M || Month in year || Month || July; Jul; 07
|-
| w || Week in year || Number || 27
|-
| W || Week in month || Number || 2
|-
| D || Day in year || Number || 189
|-
| d || Day in month || Number || 10
|-
| F || Day of week in month || Number || 2
|-
| E || Day name in week || Text || Tuesday; Tue
|-
| u || Day number of week (1 = Monday, ..., 7 = Sunday) || Number || 1
|-
| a || Am/pm marker|| Text || PM
|-
| H || Hour in day (0-23) || Number || 0
|-
| k || Hour in day (1-24) || Number || 24
|-
| K || Hour in am/pm (0-11) || Number || 0
|-
| h || Hour in am/pm (1-12) || Number || 12
|-
| m || Minute in hour || Number || 30
|-
| s || Second in minute || Number || 55
|-
| S || Millisecond || Number || 978
|-
| z || Time zone || General time zone || Pacific Standard Time; PST; GMT-08:00
|-
| Z || Time zone || RFC 822 time zone || -0800
|-
| X || Time zone || ISO 8601 time zone || -08; -0800; -08:00
|}

== Components with Fixed Values ==
Some Components on the output may have a fixed value, for example Frequency is always M regardless of the input data. This is defined at the level of the structure map. As mappings can be bi-directional the input can also have a fixed value, so when mapping the other way (from target to source) the input becomes the output.

= Mapping Rules: Use Cases =
== Many Outputs from a Single Source ==
An example use case is a Dimension with ID '''UNIQUE_KEY''' whose values are used to uniquely define a series, example '''SER1''', '''SER2''', '''SER3'''. The mapped DSD splits this into multiple Dimensions '''FREQ''', '''REF_AREA''', '''INDICATOR'''. The mapping rules split the unique key '''SER1''' into FREQ:M, REF_AREA:UK and INDICATOR:EMPLOYED. Another unique key would map to a different breakdown of values, for example '''SER2''' maps to FREQ:M, REF_AREA:FR and INDICATOR:EMPLOYED. 
 
'''Solution 1'''
 
This type of use case can be solved by creating 3 Component Maps:
 
'''Component Map 1:''' Source=UNIQUE_KEY Target=FREQ 
'''Component Map 2:''' Source=UNIQUE_KEY Target=REF_AREA 
'''Component Map 3:''' Source=UNIQUE_KEY Target=INDICATOR 
 
Each Component Map is backed by a [[Representation_Map|Representation Map]], which maps the value of the Unique Key to the output.
 
'''Representation Map''' = UNIQUE_KEY -> FREQ 
'''Values:''' SER1=M, SER2=M, SER3=A 
 
'''Representation Map''' = UNIQUE_KEY -> REF_AREA 
'''Values:''' SER1=UK, SER2=FR, SER3=UK 
 
'''Representation Map''' = UNIQUE_KEY -> INDICATOR 
'''Values:''' SER1=EMPLOYED, SER2=EMPLOYED, SER3=EMPLOYED 
 
In this example, the unique Key SER1 would be mapped to '''M''', '''UK''', '''EMPLOYED'''. 
 
'''Solution 2''' 
An alternative solution to this mapping is to create a single Component Map which maps the source '''UNIQUE_KEY''' to three outputs '''FREQ''', '''REF_AREA''', '''INDICATOR'''. 
 
A Single Representation Map is required to map each '''UNIQUE KEY''' to the three outputs. 
'''Representation Map''' = UNIQUE_KEY -> FREQ:REF_AREA:INDICATOR 
'''Values:''' SER1=M:UK:EMPLOYED, SER2=M:FR:EMPLOYED 
 
'''Summary'''
 
The choice of whether to split the mapping up into separate components vs a single rule should be based on what will be more maintainable, understandable, and can individual mapping rules be reused by other structure maps.

== Many Sources map to a Single output ==
This is the reverse of the above use case, and has the same 2 solutions, split the rules into individual maps, or describe the relationship in a single map. If the rule is split into individual maps, then it is important to note how Fusion Registry determines the final output.

If the output from Component Map 1 is a set of possible value: FR, DE and the output from Component Map 2 is another set of possible values, DE and UK, then the intersection of both sets are used to find the final output, in this case DE.

== If-Then-Else (default if not specified) ==
The order of rules in a [[Representation_Map|Representation Map]] can be important, specifically when using regular expressions. The regular expressions are tested in the same order that the appear they are defined, this allows for more specific expressions to be tested before a general catch all.

'''Example'''
 
'''Rule 1''' (no regex): A -> B (A maps to B) 
'''Rule 2''' (reg ex): A\dB -> B2 (A followed by a number followed by B maps to B2) 
'''Rule 3''' (reg ex): .* -> _Z (anything maps to _Z) 
 
Source data will first be checked against the exact match rule (A maps to B), followed by each regular expression rule until a match is found. As the last expression matched on anything, this can be considered as 'if nothing matches then output _Z'.

== Pattern match input, use matched text on output ==
A rule can be defined to match a specific pattern, which is then used in the output. For example the rule can state any three characters followed by a number is converted to the same three characters without the number. This can be satisfied by using Regular Expressions to match the input, with a capture group. A capture group is where the regular expression rule is in parentheses which can then be referred to by number (capture group 1, 2, 3, and so on).

'''Example:''' 
'''RegEx Input''': ([A-Z]{3})_([0-9]) 
'''Output Expression:''' \2_\1

This example consists of 2 capture groups:
# '''([A-Z]{3})''' Any A to Z character 3 times
# '''(0-9)''' the number zero to nine
 
The output expression then reverses the order of the information by outputting capture group 2, an underscore, followed by capture group 1.

An example input for the above expression, and corresponding output is as follows: 
'''Input''' = ABC_1 
'''Output''' = 1_ABC

== Pattern match input to used on second input ==
This use case is where there is more then one source Component for a Mapping, example CURRENCY and REF_AREA. The value for one of the source components is based on a pattern, and the value for the second component is based on what matched the first pattern.
 
Like the pattern match on output, this rule makes use of regular expression capture groups to copy matched information from one rule to another. The capture group (everything matched in the parenthesis) is referred to by number, with a leading slash \.

'''Example''' 
'''CURRENCY''' = (.*) 
'''REF_AREA''' = \1_X 

In this example the first input matches on anything, but the REF_AREA rule is using the matched value from CURRENCY, defined by capture group \1, followed by _X. The following shows a match and a miss:
 
'''Match:''' CURRENCY=USD, REF_AREA=USD_X 
'''Miss:''' CURRENCY_USD, REF_AREA=US_X

== Convert Measures to Dimensions ==
This use case is converting source data with multiple Measures (example BIRTHS, DEATHS, MARRIAGES) to a DSD with only one OBS_VALUE. In this case the Measure may want to be converted into a Dimension value, for example INDICATOR.
 
The table below shows an example source dataset with three measures, BIRTHS, DEATHS, MARRIAGES.

{| class="wikitable"
|+ Source Data
|-
! REF_AREA !! TIME !! BIRTHS !! DEATHS !! MARRIAGES
|-
| UK || 2020 || 11 || 12 || 13
|-
| FR || 2020 || 21 || 22 || 23
|}

The dataset should be mapped to convert the BIRTHS, DEATHS and MARRIAGES to the INDICATOR B, D, and M respectively. The observation value is the value of each corresponding measure.
{| class="wikitable"
|+ Desired Output
|-
! REF_AREA !! INDICATOR !! TIME_PERIOD !! OBS_VALUE
|-
| UK || B || 2020 || 11
|-
| UK || D || 2020 || 12
|-
| UK || M || 2020 || 13
|-
| FR || B || 2020 || 21
|-
| FR || D || 2020 || 22
|-
| FR || M || 2020 || 23
|}

This mapping relationship can be defined by mapping each source MEASURE to both the OBS_VALUE component and the INDICATOR component. The rule for the INDICATOR mapping should be a single 'catch all' regular expression, which maps the particular measure to a fixed value.

'''Example''' 
'''Component Map 1''': BIRTHS maps to OBS_VALUE 
'''Component Map 2''': BIRTHS maps to INDICATOR (uses Births [[Representation_Map|Representation Map]]) 
 
'''Births Representation Map''': source=[anything], target=B
 
Where the [anything] rule is simply the regular expression '''.*'''

== Mapping UNIX Time ==
If the source Component has a timestamp in UNIX Time, an Epoch Map should be created which maps from the source Component to target Component, with the base period set to 1970, and the epoch set to milliseconds.

= Testing Structure Map =
Structure Mapping can be a complex task, if multiple sources influence many targets, and rules include regular expressions and substring matches, then it is important to test the mapping to ensure the outputs are as expected.

The Fusion Registry provides a testing feature, to use this, first login to the Fusion Registry, navigate to the Structure Map page, select the Structure Map to test and click on the Test Mapping button. A data file must be loaded which conforms to either the source or the target DSD. The datafile may be in SDMX format, or CSV. If loading CSV, ensure the column headers match the Component Id.

Mapping reports are generated at the level of each row of information, if loading time series data, a row should be thought of as a single observation along with all the series information. The report will describe the output row for each input row.

The Fusion Registry can perform finer grained reports detailing exactly how a specific row was mapped, in the User Interface this is achieved by clicking on the row. The mapping report is broken down into each mapping rule in the structure map, what the input was for the rule, what and what the output were.

= Conversion of Data =
The Fusion Registry Transformation services can use a Structure Map to apply to the conversion of data.
* [[Asynchronous_Data_Validation_and_Transformation_Web_Service|Asynchronous Transformation]]
* [[Data_Transformation_Web_Service|Synchronous Transformation]]

Structure IDs, Names and Descriptions

2024-04-17T15:33:25Z

Vmurrell:

__NOTOC__
[[Category:FMR_Concepts_Reference]]
=Overview=
[[Identifiable V11]] structures are identified by an ID which must be unique within their context. 
[[Maintainable V11]] and Nameable structures are also Identifiable and therefore must have an ID, but in addition have a Name, and an optional Description.

=ID=
An ID is a string that is mandatory for Identifiable structures and serves as a unique reference as part of each structure's [[URN]].

IDs must be unique within their context. For instance the ID of each Concept must be unique within its Concept Scheme. However, Concepts in different Concept Schemes can have the same ID.

{| class="wikitable"
|-
! Allowable Content !! Example
|-
| Alpha Numeric || A-Z a-z 0-9
|-
| Dollar || $
|-
| Hyphen || -
|-
| Underscore || _
|}

By convention IDs are often uppercase, but lowercase characters can also be used. It is also conventional for Codelist IDs to start with 'CL_'.

ID Examples:
{| class="wikitable"
|-
! Structure Type !! ID
|-
| Agency || SDMX
|-
| Agency || IMF
|-
| Concept Scheme || CROSS_DOMAIN_CONCEPTS
|-
| Codelist || CL_REF_AREA
|-
| Codelist || CL_FREQ
|-
| Data Structure Definition || ECOFIN_DSD
|-
| Dataflow || WDI_HEALTH
|-
| Dimension || FREQ
|-
| Dimension || REF_AREA
|-
| Attribute || OBS_STATUS
|}

=Name=
A Name is a string that is mandatory for Maintainable and Nameable structures providing a human-readable way to identify the structure.

Unlike an ID, a structure's name does not have to be unique within its context.

Names can be multilingual.

Name examples:
{| class="wikitable"
|-
! Structure Type !! ID !! Name(en) !! Name(fr)
|-
| Agency || SDMX || SDMX || SDMX
|-
| Agency || OECD || Organisation for Economic Co-operation and Development (OECD) || Organisation de Coopération et de Développement Économiques (OCDE)
|-
| Concept Scheme || CROSS_DOMAIN_CONCEPTS || SDMX Cross Domain Concept Scheme ||
|-
| Codelist || CL_REF_AREA || Reference area code list || Liste des indicatifs régionaux de référence
|-
| Codelist || CL_FREQ || Frequency code list || Liste des codes de fréquence
|-
| Data Structure Definition || ECOFIN_DSD || ECOFIN data structure definition ||
|-
| Dataflow || WDI_HEALTH || World Bank World Development Indicators Health ||
|}

=Description=
Maintainable and Nameable structures may also be given a description. Descriptions are optional and serve to provide an additional human-readable explanation about the purpose of the structure.

Descriptions can be multilingual.

Description Example:
{| class="wikitable"
|-
| Structure Type || Agency
|-
| ID|| ESTAT
|-
| Name (en) || Statistical Office of the European Union (Eurostat)
|-
| Description (en) || Eurostat is the statistical office of the European Union situated in Luxembourg. Its task is to provide the European Union with high quality statistics at European level that enable comparisons between countries and regions. Website: http://ec.europa.eu/eurostat/.
|-
| Description (fr) || Eurostat est l'office statistique de l'Union européenne situé au Luxembourg. Sa mission est de fournir à l'Union européenne des statistiques de haute qualité au niveau européen permettant des comparaisons entre pays et régions. Site Web: http://ec.europa.eu/eurostat/.
|}

Structure IDs, Names and Descriptions

2024-04-17T15:32:35Z

Vmurrell:

__NOTOC__
[[Category:FMR_Concepts_Reference]]
=Overview=
[[Identifiable V11]] structures are identified by an ID which must be unique within their context. 
[[Maintainable V11]] and Nameable structures are also [[Identifiable]] and therefore must have an ID, but in addition have a Name, and an optional Description.

=ID=
An ID is a string that is mandatory for Identifiable structures and serves as a unique reference as part of each structure's [[URN]].

IDs must be unique within their context. For instance the ID of each Concept must be unique within its Concept Scheme. However, Concepts in different Concept Schemes can have the same ID.

{| class="wikitable"
|-
! Allowable Content !! Example
|-
| Alpha Numeric || A-Z a-z 0-9
|-
| Dollar || $
|-
| Hyphen || -
|-
| Underscore || _
|}

By convention IDs are often uppercase, but lowercase characters can also be used. It is also conventional for Codelist IDs to start with 'CL_'.

ID Examples:
{| class="wikitable"
|-
! Structure Type !! ID
|-
| Agency || SDMX
|-
| Agency || IMF
|-
| Concept Scheme || CROSS_DOMAIN_CONCEPTS
|-
| Codelist || CL_REF_AREA
|-
| Codelist || CL_FREQ
|-
| Data Structure Definition || ECOFIN_DSD
|-
| Dataflow || WDI_HEALTH
|-
| Dimension || FREQ
|-
| Dimension || REF_AREA
|-
| Attribute || OBS_STATUS
|}

=Name=
A Name is a string that is mandatory for Maintainable and Nameable structures providing a human-readable way to identify the structure.

Unlike an ID, a structure's name does not have to be unique within its context.

Names can be multilingual.

Name examples:
{| class="wikitable"
|-
! Structure Type !! ID !! Name(en) !! Name(fr)
|-
| Agency || SDMX || SDMX || SDMX
|-
| Agency || OECD || Organisation for Economic Co-operation and Development (OECD) || Organisation de Coopération et de Développement Économiques (OCDE)
|-
| Concept Scheme || CROSS_DOMAIN_CONCEPTS || SDMX Cross Domain Concept Scheme ||
|-
| Codelist || CL_REF_AREA || Reference area code list || Liste des indicatifs régionaux de référence
|-
| Codelist || CL_FREQ || Frequency code list || Liste des codes de fréquence
|-
| Data Structure Definition || ECOFIN_DSD || ECOFIN data structure definition ||
|-
| Dataflow || WDI_HEALTH || World Bank World Development Indicators Health ||
|}

=Description=
Maintainable and Nameable structures may also be given a description. Descriptions are optional and serve to provide an additional human-readable explanation about the purpose of the structure.

Descriptions can be multilingual.

Description Example:
{| class="wikitable"
|-
| Structure Type || Agency
|-
| ID|| ESTAT
|-
| Name (en) || Statistical Office of the European Union (Eurostat)
|-
| Description (en) || Eurostat is the statistical office of the European Union situated in Luxembourg. Its task is to provide the European Union with high quality statistics at European level that enable comparisons between countries and regions. Website: http://ec.europa.eu/eurostat/.
|-
| Description (fr) || Eurostat est l'office statistique de l'Union européenne situé au Luxembourg. Sa mission est de fournir à l'Union européenne des statistiques de haute qualité au niveau européen permettant des comparaisons entre pays et régions. Site Web: http://ec.europa.eu/eurostat/.
|}

Structure IDs, Names and Descriptions

2024-04-17T15:31:12Z

Vmurrell:

__NOTOC__
[[Category:FMR_Concepts_Reference]]
=Overview=
[[Identifiable]] structures are identified by an ID which must be unique within their context. 
[[Maintainable]] and Nameable structures are also [[Identifiable]] and therefore must have an ID, but in addition have a Name, and an optional Description.

=ID=
An ID is a string that is mandatory for Identifiable structures and serves as a unique reference as part of each structure's [[URN]].

IDs must be unique within their context. For instance the ID of each Concept must be unique within its Concept Scheme. However, Concepts in different Concept Schemes can have the same ID.

{| class="wikitable"
|-
! Allowable Content !! Example
|-
| Alpha Numeric || A-Z a-z 0-9
|-
| Dollar || $
|-
| Hyphen || -
|-
| Underscore || _
|}

By convention IDs are often uppercase, but lowercase characters can also be used. It is also conventional for Codelist IDs to start with 'CL_'.

ID Examples:
{| class="wikitable"
|-
! Structure Type !! ID
|-
| Agency || SDMX
|-
| Agency || IMF
|-
| Concept Scheme || CROSS_DOMAIN_CONCEPTS
|-
| Codelist || CL_REF_AREA
|-
| Codelist || CL_FREQ
|-
| Data Structure Definition || ECOFIN_DSD
|-
| Dataflow || WDI_HEALTH
|-
| Dimension || FREQ
|-
| Dimension || REF_AREA
|-
| Attribute || OBS_STATUS
|}

=Name=
A Name is a string that is mandatory for Maintainable and Nameable structures providing a human-readable way to identify the structure.

Unlike an ID, a structure's name does not have to be unique within its context.

Names can be multilingual.

Name examples:
{| class="wikitable"
|-
! Structure Type !! ID !! Name(en) !! Name(fr)
|-
| Agency || SDMX || SDMX || SDMX
|-
| Agency || OECD || Organisation for Economic Co-operation and Development (OECD) || Organisation de Coopération et de Développement Économiques (OCDE)
|-
| Concept Scheme || CROSS_DOMAIN_CONCEPTS || SDMX Cross Domain Concept Scheme ||
|-
| Codelist || CL_REF_AREA || Reference area code list || Liste des indicatifs régionaux de référence
|-
| Codelist || CL_FREQ || Frequency code list || Liste des codes de fréquence
|-
| Data Structure Definition || ECOFIN_DSD || ECOFIN data structure definition ||
|-
| Dataflow || WDI_HEALTH || World Bank World Development Indicators Health ||
|}

=Description=
Maintainable and Nameable structures may also be given a description. Descriptions are optional and serve to provide an additional human-readable explanation about the purpose of the structure.

Descriptions can be multilingual.

Description Example:
{| class="wikitable"
|-
| Structure Type || Agency
|-
| ID|| ESTAT
|-
| Name (en) || Statistical Office of the European Union (Eurostat)
|-
| Description (en) || Eurostat is the statistical office of the European Union situated in Luxembourg. Its task is to provide the European Union with high quality statistics at European level that enable comparisons between countries and regions. Website: http://ec.europa.eu/eurostat/.
|-
| Description (fr) || Eurostat est l'office statistique de l'Union européenne situé au Luxembourg. Sa mission est de fournir à l'Union européenne des statistiques de haute qualité au niveau européen permettant des comparaisons entre pays et régions. Site Web: http://ec.europa.eu/eurostat/.
|}

Security Configuration

2024-04-17T15:29:02Z

Vmurrell:

[[Category:Installation_and_Configuration]]

[[Category:FMR Configuration Reference]]
==Overview==

The Security function is available to logged-in users and found on the Main menu.

Security is split into two distinct functions: User '''Authentication''' and User '''Authorisation'''. Authentication is the process of ensuring the provided user credentials match up against a valid user account. Authorisation is the process of ensuring a user is allowed to perform the action they are trying to perform.

The Fusion Metadata Registry only provides Authentication services for one user; the Fusion Metadata Registry Root user. Authentication for other users are provided by either:

* Fusion Security Web Server
* [https://en.wikipedia.org/wiki/Active_Directory Active Directory] using the [https://en.wikipedia.org/wiki/Lightweight_Directory_Access_Protocol LDAP] protocol
* [https://en.wikipedia.org/wiki/OpenLDAP OpenLDAP]

Once a user is Authenticated, the relevant User Account is loaded into the session, and the Registry uses its security model and rules to '''authorise''' the user is allowed to access the resource. The example below shown as example for Active Directory. You can read more [[Active_Directory_-_Set_up_Role_Mappings| about Active Directory in this article]].

[[File:VMSS0.PNG|Authentication Set-up|800px]]

== Authentication ==
An Authentication Service is required to verify the provided credentials and to supply the Registry with information about the user.

Fusion Metadata Registry's web services support [https://en.wikipedia.org/wiki/Basic_access_authentication HTTP Basic Authentication].

'''Username and Password''' authentication requires an authentication service to be running which can be used to verify the credentials. This external authentication service may be [[Fusion Security]] or [[Active Directory]], the two authentication services are mutually exclusive - the Fusion Metadata Registry can only be configured to use one of these services.

After the Authentication process, the Registry must Authorise the user to access the resources. This is achieved by the Fusion Metadata Registry linking the user's account to one or more [[Organisations]], this link is achieved in different ways depending on the Authentication mechanism.

=== Fusion Security ===
If the Authentication Service is Fusion Security, then the Fusion Security server will verify the user credentials and return the user account details to the Fusion Metadata Registry, including which [[Organisations]] the user belongs to. No additional configuration is required in the Fusion Metadata Registry.

Fusion Security is only applicable for Fusion Metadata Registry 10 and is not an option for Fusion Metadata Registry version 11.

=== Active Directory ===
If Active Directory is used as an Authentication server, then the Common Name (CN) is used to authenticate with the server. The CN is mapped in the Fusion Metadata Registry to one or more [[Organisations]]. To learn more about how to map users to Active Directory, [https://fmrwiki.sdmxcloud.org/Active_Directory_-_Role_Mapping please refer to this article.]

== Authorisation ==
To understand Authorisation, it is important to understand the security model for the Fusion Metadata Registry. Each user account links to zero or more [[Organisations]] maintained in the Fusion Metadata Registry. The Organisation a user account can be linked to falls into one of three categories:
# An Agency
# A Data Provider
# A Data Consumer

A user account may have '''administrative''' privileges, which allows the user unrestricted access to any information in the product, including access to the configuration settings of the product.

A '''Agency''' user is able to create, maintain, and delete structures that belong to the Agency, or any of its sub-agencies.

A '''Data Provider''' user is able to validate and convert datasets the Data Provider has been set up to provide data for via a [[Provision Agreement]].

A '''Data Consumer''' user has no special privileges provided by default, however they are able to access the Registry if the product has been set up to enforce login.

== Root User ==
Fusion Metadata Registry provides a single root user account, where the credentials are stored locally (not in an external authentication service). The Fusion Registry authenticates the root user, and as such the root user is always able to log into the product should the external authentication service become inaccessible.

It is not a requirement to set up an external authentication service - Fusion Metadata Registry can be run in ''single user mode'' using the built-in '''root''' superuser account. Root user has unrestricted access to the product, and as such security rules do not apply to the root user.

==Reporting Template==
===Restrict Access===

Fusion Metadata Registry's default security ensures that generating an Excel workbook from a Report Template is a restricted action. Only users with certain access permission can generate an Excel workbook. The rules are:

{| class="wikitable"
|-
! User Type !! Permission
|-
| Admin || Can download Excel Report Workbooks for any Data Provider
|-
| Agency || Can download Excel Report Workbooks for any Data Provider which is maintained by the Agency
|-
| Data Provider || Can download Excel Report Workbooks for which they can report data
|-
| Data Consumer || Can not download any Excel Report Workbooks
|-
| Anonymous || Can not download any Excel Report Workbooks

|}

To disable theses default settings, you can use the Enable / Disable button but note it is either all ON or all OFF.

===Worksheet Passwords===
When a password is applied, when Fusion Metadata Registry generates an Excel workbook from a Report Template definition, it will ensure non-observation cells are locked and can not be edited unless a password is supplied to unlock the worksheet.

This will help prevent the data reporter editing the workbook in such a way to make it unreadable by the Registry.

Passwords are set per Agency and can be changed at any time. If no password is set, the Excel workbook will not be locked.

It is strongly recommended to ensure each agency has a password set to ensure the workbooks are locked.