Difference between revisions of "Load Structures"
(→Upload Unsuccessful) |
(→Submitting Invalid Structures) |
||
(18 intermediate revisions by the same user not shown) | |||
Line 2: | Line 2: | ||
[[Category:How_To V11]] | [[Category:How_To V11]] | ||
=Overview= | =Overview= | ||
− | Structures can be loaded into the registry | + | Structures can be loaded into the registry either by [[Submit_Structures_Web_Service|Web Service API]] or by the User Interface, but first you have to have some to load. If you already have structures to load, please skip to the '''Loading Structures''' Section below. |
=Obtaining Structures= | =Obtaining Structures= | ||
Line 28: | Line 28: | ||
==Before you start== | ==Before you start== | ||
* To Load a Structure(s) you do need to be logged in to the Registry. | * To Load a Structure(s) you do need to be logged in to the Registry. | ||
− | * To successfully load a Structure | + | * To successfully load a Structure it may be necessary to ensure that the owning Agency already exists (select '''Organisation''' from the main menu then '''Agencies''') unless the Agency is submitted in the structures file. |
==Loading a Structure== | ==Loading a Structure== | ||
Line 61: | Line 61: | ||
Once the import process has finished, you will see a Submission report as shown in the example below. You can also '''Abort''' the process from here. | Once the import process has finished, you will see a Submission report as shown in the example below. You can also '''Abort''' the process from here. | ||
− | [[File: | + | [[File:SubmissionSuccess.png|Submission Report|500px]]<br> |
− | If you click ''' | + | If you click '''Commit''' the structure will be committed into the FMR. |
− | '''Note''': if you are uploading a large number of Structures it can take a little while before the structures are | + | '''Note''': if you are uploading a large number of Structures it can take a little while before the structures are committed. A good way of checking is to Navigate to the Bulk Actions page. |
==Unsuccessful Submissions== | ==Unsuccessful Submissions== | ||
If the structure file is broken or of an invalid format, a "red box" error message will be displayed, similar to the one shown below. | If the structure file is broken or of an invalid format, a "red box" error message will be displayed, similar to the one shown below. | ||
− | [[File: | + | [[File:SubmissionError.png|Example Error Message|750px]]<br> |
− | == | + | ==Submitting Invalid Structures== |
− | From FMR version 11. | + | From FMR version 11.19.0 onwards, by default the Upload functionality will "skip" structures that are deemed to be invalid. The advantage of this is that if a structure file containing invalid structures is submitted, the valid structures can be submitted to the Registry without the invalid structures preventing this. There are 2 types of invalidity check performed in the FMR: |
+ | * Missing cross-references - where a structure refers to another structure that does not exist in either the Registry, nor the structure file itself. | ||
+ | * Failing business validation - where a structure is invalid against the schema. For example, submitting partial structures, or the submission of a DataConstraint with Role specified as 'Actual' which is not supported. | ||
− | + | Structures that are not covered by this are structures that cannot be constructed from the submitted file, for example due to invalid XML. | |
− | + | ||
− | + | When a structure file is submitted with invalid structures, the submission report is shown as usual, along with a warning message and an extra button is also displayed ('View Invalid Structures'). Clicking this button will display a modal stating the structures that have failed validation and their reason for failure. | |
− | + | ||
+ | Please remember that the Registry enforces structural integrity and if a valid structure is invalidated by one of its descendants, it can not be loaded and will be deemed invalid. For example: a Dataflow that refers to a Data Structure that refers to a Concept Scheme which has an invalid reference (e.g. a Concept refers to a Codelist that does not exist), the Registry will prevent any of those structures being loaded. All failing structures would be listed in the validation report. This is illustrated in the example below, where a structure file was loaded which contains 2 Codelists, 3 Concept Schemes, 1 DataStructure and 1 Dataflow. One of the Concept Schemes references an invalid Codelist which is referenced by the DataStructure and Dataflow, thus these structures are not considered valid and can not be loaded until a change is made to the Concept Scheme with the invalid reference. | ||
+ | |||
+ | [[File:SubmissionWithWarnings.png|Submission with structures failing validation|400px]] | ||
+ | [[File:FailingValidation.png|List of structures that have failed validation|652px]]<br> |
Latest revision as of 03:29, 5 November 2024
Overview
Structures can be loaded into the registry either by Web Service API or by the User Interface, but first you have to have some to load. If you already have structures to load, please skip to the Loading Structures Section below.
Obtaining Structures
All structures must be compliant with the SDMX Standard. See this article for a full list of Maintainable Structures. You can learn more about the Standards on the SDMX web sites.
Whilst you can easily create Structure from scratch, there are a number of other options available.
Using Export Structures options in another Registry
Please see this article which explains how to do this and where you can find the SDMX Global Registry site.
Using Web Services option in another Registry
Structures can be obtained using the Web Service option from another Registry. The image below shows a Registry with Web Services and Structure selected.
You can use the selection boxes to define what you want and then you can either download the structures OR copy the URL to use when uploading.
Using Environmental Sync
See this article for more information on how to do this.
Loading Structures
Before you start
- To Load a Structure(s) you do need to be logged in to the Registry.
- To successfully load a Structure it may be necessary to ensure that the owning Agency already exists (select Organisation from the main menu then Agencies) unless the Agency is submitted in the structures file.
Loading a Structure
On most pages of the Registry, at the top right of the page, you will see the Load Structures button (provided you are logged in).
Once this button is clicked the Upload option will open as shown in the image below.
Either search for and select a file (individual files or Zip files can be used) OR select the URL button which will display an additional field for you to enter a URL location of the structure. See the Web Services option above for more details on how to obtain an appropriate URL as shown in the example below.
Import Actions
Before the structures are imported you have further options available:
Append
A structure file loaded with action 'Append' may only add new structures and may not overwrite any existing structures.
Replace
A structure file loaded with action 'Replace' may add new structures to the Registry, and can also replace existing structures with new ones. Replace is the default selection.
Merge
A structure file loaded with action 'Merge' may add new structures and replace existing structures. However for Item Schemes (Codelists, Concept Schemes, Agency Schemes, Data Provider Schemes) the items submitted will be added to the existing scheme. For example if a Codelist exists with Codes A, B, and C, and the same Codelist is submitted with Codes B and X, then the resulting Codelist will have Codes A, B, C, X, The Merge submission has replaced Code B and added code X to the Codelist.
Full Replace
A structure file loaded with action 'Full Replace' will remove all existing structures from the Registry and replace them with the contents of the loaded file. This can be a useful feature when copying structures from one environment to another.
Upload Successful
Once the import process has finished, you will see a Submission report as shown in the example below. You can also Abort the process from here.
If you click Commit the structure will be committed into the FMR.
Note: if you are uploading a large number of Structures it can take a little while before the structures are committed. A good way of checking is to Navigate to the Bulk Actions page.
Unsuccessful Submissions
If the structure file is broken or of an invalid format, a "red box" error message will be displayed, similar to the one shown below.
Submitting Invalid Structures
From FMR version 11.19.0 onwards, by default the Upload functionality will "skip" structures that are deemed to be invalid. The advantage of this is that if a structure file containing invalid structures is submitted, the valid structures can be submitted to the Registry without the invalid structures preventing this. There are 2 types of invalidity check performed in the FMR:
- Missing cross-references - where a structure refers to another structure that does not exist in either the Registry, nor the structure file itself.
- Failing business validation - where a structure is invalid against the schema. For example, submitting partial structures, or the submission of a DataConstraint with Role specified as 'Actual' which is not supported.
Structures that are not covered by this are structures that cannot be constructed from the submitted file, for example due to invalid XML.
When a structure file is submitted with invalid structures, the submission report is shown as usual, along with a warning message and an extra button is also displayed ('View Invalid Structures'). Clicking this button will display a modal stating the structures that have failed validation and their reason for failure.
Please remember that the Registry enforces structural integrity and if a valid structure is invalidated by one of its descendants, it can not be loaded and will be deemed invalid. For example: a Dataflow that refers to a Data Structure that refers to a Concept Scheme which has an invalid reference (e.g. a Concept refers to a Codelist that does not exist), the Registry will prevent any of those structures being loaded. All failing structures would be listed in the validation report. This is illustrated in the example below, where a structure file was loaded which contains 2 Codelists, 3 Concept Schemes, 1 DataStructure and 1 Dataflow. One of the Concept Schemes references an invalid Codelist which is referenced by the DataStructure and Dataflow, thus these structures are not considered valid and can not be loaded until a change is made to the Concept Scheme with the invalid reference.