Transform data into a different structure using Structure Maps
- 1 Use Case 1 - using Fixed Values
- 1.1 Step 1 - Create a new Data Structure for the Source
- 1.2 Step 2 - Create a Structure Set
- 1.3 Step 3 - Create a Data Structure Map
- 1.4 Step 4 - Test Mappings
- 1.5 Step 5 - Load the Source Data File
- 1.6 Step 6 - Convert the Source Data File
- 1.7 Step 7 - Load the Converted Data File
- 2 Using this example
Use Case 1 - using Fixed Values
The image below shows that 3 Components are Enumerated and the Codes which are in the linked Codelists.
If a data file is to succeed the Validation process it would have to conform to these codes. However, it this example the source file supplied does not conform instead it contains the codes:
- FR was used instead of FRA for France
- ANN was used instead of A for Annual
- ARAB was used instead of ARA for Arable Land
To successfully validate this data, a Data Structure Map can be created to map the source data to the Target Data Structure. To Achieve this...
Step 1 - Create a new Data Structure for the Source
In this example the Source data structure can be exactly the same as the Target however instead of Enumerating the Concepts you can allow any format to be used as shown in the example below.
Step 2 - Create a Structure Set
All maps must belong to a Structure Set. A logged-on user with the appropriate privileges can select Structure Maps from the Main Menu followed by Structure Sets and create a Structure Set.
Step 3 - Create a Data Structure Map
Complete Step 1 of the Wizard - High Level Details
From the Structure Maps Menu, select Data Structure Map and enter the high-level details as shown below.
Complete Step 2 of the Wizard - Select the Source and Target Data Structures
Complete Step 3 of the Wizard - Map the Components
In this example I am using the Target Fixed Value area to specify that no matter what is in the Source file for Frequency, Series and Country, the Frequency will be mapped to Annual, the Series will be mapped to ARA and the Country will be mapped to France in the Target.
For the TIME_PERIOD and OBS_VALUE Components, these can be mapped directly as shown above. After this step, the Structure Map can be Finished.
Step 4 - Test Mappings
Once the Data Structure Map has been created you will see the orange Test Mapping button as shown below.
To test the mapping, you will need to select a file containing the data in the format specified above namely:
FR instead of FRA for France ANN instead of A for Annual ARAB instead of ARA for Arable Land
Once the file is loaded, you will see the Explain Plan.
The Plan will show you exactly what will happen when this Map is used.
Step 5 - Load the Source Data File
Using the Convert menu option load the data file, you will see Validation Errors as shown below which is to be expected.
Step 6 - Convert the Source Data File
Having Loaded the Data, the next step is to click the Convert button as shown in the image below. Note that I have selected the Data Structure Map. On export the dataset will be converted in terms of its structure, and representation e.g., the data set can be converted to conform to the target Data Structure Definition.
Step 7 - Load the Converted Data File
This time the converted data file will achieve a successful validation.
Using this example
1. Load the Example Structures
Download this file Structures Zip File. Note, you may need to create an Agency with the ID of MY_AGENCY first.
2. Load the Example Structure Set
Download this file Structure Set Zip File. Note, you will need to have loaded the Structures first.
3. Use the Test Mapping File
Note that you need to have loaded the Structures and the Structure set first before you can test the mapping.
1. Download this file which you can use to test the Mapping.
2. From the Structure Maps menu, select Data Structure Maps and then select MY_AGENCY:SDM1. Once selected, click Test Mapping.
3. Select the Data file you downloaded click Start Test. The Explain Plan will be displayed.
You can now follow the steps detailed above from Step 5.