Transform data into a different structure using Structure Maps

Important note

The following information is for version 10 of Fusion Registry and a large amount of it is not relevant for version 11

Use Case 1 - using Fixed Values

In this scenario we have a simple Data Structure with a number of Components. They are all Dimensions except the Observation Value which is a measure.

The image below shows that 3 Components are Enumerated and the Codes which are in the linked Codelists.

If a data file is to succeed the Validation process it would have to conform to these codes. However, it this example the source file supplied does not conform instead it contains the codes:

• FR was used instead of FRA for France
  

• ANN was used instead of A for Annual
  

• ARAB was used instead of ARA for Arable Land
  

To successfully validate this data, a Data Structure Map can be created to map the source data to the Target Data Structure. To Achieve this...

Step 1 - Create a new Data Structure for the Source

In this example the Source data structure can be exactly the same as the Target however instead of Enumerating the Concepts you can allow any format to be used as shown in the example below.

Step 2 - Create a Structure Set

All maps must belong to a Structure Set. A logged-on user with the appropriate privileges can select Structure Maps from the Main Menu followed by Structure Sets and create a Structure Set.

Step 3 - Create a Data Structure Map

Complete Step 1 of the Wizard - High Level Details

From the Structure Maps Menu, select Data Structure Map and enter the high-level details as shown below.

Complete Step 2 of the Wizard - Select the Source and Target Data Structures

Complete Step 3 of the Wizard - Map the Components

In this example I am using the Target Fixed Value area to specify that no matter what is in the Source file for Frequency, Series and Country, the Frequency will be mapped to Annual, the Series will be mapped to ARA and the Country will be mapped to France in the Target.

For the TIME_PERIOD and OBS_VALUE Components, these can be mapped directly as shown above. After this step, the Structure Map can be Finished.

Step 4 - Test Mappings

Once the Data Structure Map has been created you will see the orange Test Mapping button as shown below.

To test the mapping, you will need to select a file containing the data in the format specified above namely:

FR instead of FRA for France ANN instead of A for Annual ARAB instead of ARA for Arable Land

Once the file is loaded, you will see the Explain Plan.

The Plan will show you exactly what will happen when this Map is used.

Step 5 - Load the Source Data File

Using the Convert menu option load the data file, you will see Validation Errors as shown below which is to be expected.

Step 6 - Convert the Source Data File

Having Loaded the Data, the next step is to click the Convert button as shown in the image below. Note that I have selected the Data Structure Map. On export the dataset will be converted in terms of its structure, and representation e.g., the data set can be converted to conform to the target Data Structure Definition.

Step 7 - Load the Converted Data File

This time the converted data file will achieve a successful validation.

Using this example

1. Load the Example Structures

Download this file Structures Zip File. Note, you may need to create an Agency with the ID of MY_AGENCY first.

2. Load the Example Structure Set

Download this file Structure Set Zip File. Note, you will need to have loaded the Structures first.

3. Use the Test Mapping File

Note that you need to have loaded the Structures and the Structure set first before you can test the mapping.

1. Download this file which you can use to test the Mapping.

2. From the Structure Maps menu, select Data Structure Maps and then select MY_AGENCY:SDM1. Once selected, click Test Mapping.

3. Select the Data file you downloaded click Start Test. The Explain Plan will be displayed.

You can now follow the steps detailed above from Step 5.