Difference between revisions of "Item Validity"
m (Vmurrell moved page Item Validity - Structural Metadata Management to Item Validity without leaving a redirect: looks belter) |
(→Codelist Wizard – Stating the Codelist Type) |
||
| (16 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
[[Category:Structural Metadata]] | [[Category:Structural Metadata]] | ||
| − | |||
==Overview== | ==Overview== | ||
| − | Item Validity allows for codes and concepts to be specified as being valid for specific periods of time. This is useful for when codes in a Codelist | + | Item Validity allows for codes and concepts to be specified as being valid for specific periods of time. This is useful for when codes in a Codelist need to be re-assigned, or discontinued. Without Item Validity the way to achieve this would be to create a new Codelist which has the changes and then migrate the users of the existing Codelist to use the new Codelist. |
Item Validity can be applied to Concepts in a Concept Scheme or Codes in a Codelist, a Hierarchical Codelist or a Codelist Map. | Item Validity can be applied to Concepts in a Concept Scheme or Codes in a Codelist, a Hierarchical Codelist or a Codelist Map. | ||
| − | [[File:IV1.PNG|500px]]<br> | + | [[File:IV1.PNG|Diagram 1|500px]]<br> |
The Currency Codelist (version 1.0) contains the codes for all currencies. With the introduction of the Euro and the discontinuation of existing currencies (DEM and FRF), a new Codelist must be created (version 2.0) which contains only the codes that are to be used. Each version of the Codelist should have a date that states when it should be used. For Codelist version 1.0, this is up to Jan 1, 1999 and for Codelist version 2.0 this is from Jan 1, 1999. Any codes that don’t change are reflected in both Codelists. | The Currency Codelist (version 1.0) contains the codes for all currencies. With the introduction of the Euro and the discontinuation of existing currencies (DEM and FRF), a new Codelist must be created (version 2.0) which contains only the codes that are to be used. Each version of the Codelist should have a date that states when it should be used. For Codelist version 1.0, this is up to Jan 1, 1999 and for Codelist version 2.0 this is from Jan 1, 1999. Any codes that don’t change are reflected in both Codelists. | ||
| Line 15: | Line 14: | ||
| − | [[File:IV2.PNG|500px]]<br> | + | [[File:IV2.PNG|Diagram 2|500px]]<br> |
For the new Euro code, this can be added to the Codelist with a date specifying when this code is Valid From. So now the Codelist will have different codes for different periods of time. | For the new Euro code, this can be added to the Codelist with a date specifying when this code is Valid From. So now the Codelist will have different codes for different periods of time. | ||
| − | [[File:IV3.PNG|500px]]<br> | + | [[File:IV3.PNG|Diagram 3|500px]]<br> |
| − | |||
==Reassigning a Code== | ==Reassigning a Code== | ||
| Line 27: | Line 25: | ||
| − | [[File:IV4.PNG|500px]]<br> | + | [[File:IV4.PNG|Diagram 4|500px]]<br> |
With the dissolution of the Gilbert and Ellice Islands, the code “GE” is re-assigned to Georgia and Georgia’s existing code (“XX”) is discontinued. | With the dissolution of the Gilbert and Ellice Islands, the code “GE” is re-assigned to Georgia and Georgia’s existing code (“XX”) is discontinued. | ||
| − | With the use of Item Validity a single Codelist can discontinue the use of the code “GE” to mean the Gilbert and Ellice Islands and re-assign it to Georgia. The use of Valid From and Valid To dates means that codes can be terminated and re-assigned as desired. | + | With the use of Item Validity, a single Codelist can discontinue the use of the code “GE” to mean the Gilbert and Ellice Islands and re-assign it to Georgia. The use of Valid From and Valid To dates means that codes can be terminated and re-assigned as desired. |
| − | [[File:IV5.PNG|500px]]<br> | + | [[File:IV5.PNG|Diagram 5|500px]]<br> |
| − | It is important to note that in the example above the Reference Area Codelist has two entries for the code “GE”. This is not permitted in a “standard” Codelist. However since each of the GE entries has a Validity Period and these Validity Periods do not overlap, this is permissible. At no point in time are both codes valid simultaneously. If a further re-assignment to the code “GE” it is possible to have more entries for “GE” provided that the Validity Periods do not overlap. | + | It is important to note that in the example above the Reference Area Codelist has two entries for the code “GE”. This is not permitted in a “standard” Codelist. However, since each of the GE entries has a Validity Period and these Validity Periods do not overlap, this is permissible. At no point in time are both codes valid simultaneously. If a further re-assignment to the code “GE” it is possible to have more entries for “GE” provided that the Validity Periods do not overlap. |
==Assigning Item Validity== | ==Assigning Item Validity== | ||
| − | ===Codelist Wizard – | + | ===Codelist Wizard – Specifying the Codelist Type=== |
The Codelist Wizard in Fusion Registry allows the user to create and edit Codelists. In Step 1 of the wizard aside from the standard elements that make up a structure, the Codelist wizard adds a further choice control which allows a Codelist to be defined in one of three forms: | The Codelist Wizard in Fusion Registry allows the user to create and edit Codelists. In Step 1 of the wizard aside from the standard elements that make up a structure, the Codelist wizard adds a further choice control which allows a Codelist to be defined in one of three forms: | ||
| Line 51: | Line 49: | ||
| − | [[File:IV6.PNG|800px]]<br> | + | [[File:IV6.PNG|800px|Codelist type]]<br> |
===Codelist Wizard – Assigning a Validity Period to an Item=== | ===Codelist Wizard – Assigning a Validity Period to an Item=== | ||
| − | [[File:IV7.PNG|800px]]<br> | + | [[File:IV7.PNG|Assigning a Validity Period to an Item|800px]]<br> |
The third step of the wizard allows the user to manually add, edit, and delete Codes in the Codelist. For editing and deleting codes, if the Codelist is of type “Item Validity” then the edit window will have entries for “Valid From” and “Valid To”. If it is a “standard” Codelist these fields will not be present. | The third step of the wizard allows the user to manually add, edit, and delete Codes in the Codelist. For editing and deleting codes, if the Codelist is of type “Item Validity” then the edit window will have entries for “Valid From” and “Valid To”. If it is a “standard” Codelist these fields will not be present. | ||
| − | [[File:IV8.PNG|400px]]<br> | + | [[File:IV8.PNG|Manual Edit|400px]]<br> |
Simply editing the “Valid From” and “Valid To” fields is enough to specify Item Validity to the currently edited item. The input takes SDMX Datetimes, so such values as 2001-Q1 and 2002-W50 are also permitted. | Simply editing the “Valid From” and “Valid To” fields is enough to specify Item Validity to the currently edited item. The input takes SDMX Datetimes, so such values as 2001-Q1 and 2002-W50 are also permitted. | ||
| Line 70: | Line 68: | ||
| − | [[File:IV9.PNG|800px]]<br> | + | [[File:IV9.PNG|Add Validity Period|800px]]<br> |
===Concept Scheme Wizard=== | ===Concept Scheme Wizard=== | ||
| Line 81: | Line 79: | ||
| − | [[File:IV10.PNG|800px]]<br> | + | [[File:IV10.PNG|Add Validity Period|800px]]<br> |
It is permissible to add multiple Validity Periods to a particular item if desired. | It is permissible to add multiple Validity Periods to a particular item if desired. | ||
| Line 92: | Line 90: | ||
| − | [[File:IV11.PNG|800px]]<br> | + | [[File:IV11.PNG|Adding a Validity Period to a Codelist Map|800px]]<br> |
| − | If a mapping has a validity period, it is shown with a calendar icon to state that this mapping is date | + | If a mapping has a validity period, it is shown with a calendar icon to state that this mapping is date dependent. To assign, remove or edit a Validity Period for a mapping, activate the button labelled “Modify Validity Period” and the following modal is displayed: |
| − | [[File:IV12.PNG|500px]]<br> | + | [[File:IV12.PNG|Modify Validity Period|500px]]<br> |
The input for each is an SDMX datetime. Clearing the values and clicking “Set Validity Period” will remove the mapping. | The input for each is an SDMX datetime. Clearing the values and clicking “Set Validity Period” will remove the mapping. | ||
| − | ==Viewing Item Validity | + | ==Viewing Item Validity== |
===Codelist=== | ===Codelist=== | ||
| Line 109: | Line 107: | ||
| − | [[File:IV13.PNG|800px]]<br> | + | [[File:IV13.PNG|Codelist|800px]]<br> |
When the user clicks “View Codelist”, if the Codelist does not have codes with validity at the item level, then the modal dialog shows a table of codes with the columns: Position, Id, Name and Description. If the Codelist has codes with validity at the item level then the Description column is not shown in the table and the Validity Period is shown instead. | When the user clicks “View Codelist”, if the Codelist does not have codes with validity at the item level, then the modal dialog shows a table of codes with the columns: Position, Id, Name and Description. If the Codelist has codes with validity at the item level then the Description column is not shown in the table and the Validity Period is shown instead. | ||
| − | [[File:IV14.PNG|800px]]<br> | + | [[File:IV14.PNG|View Codelist|800px]]<br> |
Note that there may be duplicate rows for the same ID and that for these rows, the position is the same. Any rows which are considered invalid for today will be rendered in grey text. | Note that there may be duplicate rows for the same ID and that for these rows, the position is the same. Any rows which are considered invalid for today will be rendered in grey text. | ||
In the middle of the screen are controls for the time period and display of Invalid Codes. | In the middle of the screen are controls for the time period and display of Invalid Codes. | ||
| − | |||
===View Time=== | ===View Time=== | ||
| Line 124: | Line 121: | ||
The “View Time” is the time that the user is viewing this Codelist. By default, this will be blank. A blank value infers the current time. | The “View Time” is the time that the user is viewing this Codelist. By default, this will be blank. A blank value infers the current time. | ||
| − | The user may select a datetime by using the Time Period input field. This field allows the user to type in any valid SDMX Datetime Value (e.g. 2001-Q4) and when return is pressed, the table updates making any of the codes which are not valid for that time turn grey. This allows the user to see the state of a Codelist for any point in time. | + | The user may select a datetime by using the Time Period input field. This field allows the user to type in any valid SDMX Datetime Value (e.g., 2001-Q4) and when return is pressed, the table updates making any of the codes which are not valid for that time turn grey. This allows the user to see the state of a Codelist for any point in time. |
There is a slide control for the hiding or showing of “invalid codes” (i.e. codes which are not valid for the current time period). By default, this is set to show invalid codes – rendered in grey. If this is set to “hide” then only currently valid codes are shown (rows which would have been rendered in grey are now not shown). | There is a slide control for the hiding or showing of “invalid codes” (i.e. codes which are not valid for the current time period). By default, this is set to show invalid codes – rendered in grey. If this is set to “hide” then only currently valid codes are shown (rows which would have been rendered in grey are now not shown). | ||
| Line 151: | Line 148: | ||
If the template is modified and then re-submitted, the existing generated templates are removed from the Registry and replaced by the new Template submission. | If the template is modified and then re-submitted, the existing generated templates are removed from the Registry and replaced by the new Template submission. | ||
| − | Deleting a Codelist Template from the Registry, will delete all Codelists that were generated from that Template. Templates may not be referenced by other structures and it is also not possible to convert a Template into another type of Codelist (e.g. Item Validity). | + | Deleting a Codelist Template from the Registry, will delete all Codelists that were generated from that Template. Templates may not be referenced by other structures and it is also not possible to convert a Template into another type of Codelist (e.g., Item Validity). |
Latest revision as of 07:14, 1 April 2024
Contents
Overview
Item Validity allows for codes and concepts to be specified as being valid for specific periods of time. This is useful for when codes in a Codelist need to be re-assigned, or discontinued. Without Item Validity the way to achieve this would be to create a new Codelist which has the changes and then migrate the users of the existing Codelist to use the new Codelist.
Item Validity can be applied to Concepts in a Concept Scheme or Codes in a Codelist, a Hierarchical Codelist or a Codelist Map.
The Currency Codelist (version 1.0) contains the codes for all currencies. With the introduction of the Euro and the discontinuation of existing currencies (DEM and FRF), a new Codelist must be created (version 2.0) which contains only the codes that are to be used. Each version of the Codelist should have a date that states when it should be used. For Codelist version 1.0, this is up to Jan 1, 1999 and for Codelist version 2.0 this is from Jan 1, 1999. Any codes that don’t change are reflected in both Codelists.
Item Validity can overcome the overhead of creating a new Codelist. In the existing Codelist, items can be discontinued by adding a date stating when it is Valid To. Since DEM and FRF need to be discontinued, these are given a “Valid To” date.
For the new Euro code, this can be added to the Codelist with a date specifying when this code is Valid From. So now the Codelist will have different codes for different periods of time.
Reassigning a Code
If a code needs to be re-assigned because its use has changed, without the use of Item Validity a new Codelist must be created. Consider the example of the Gilbert and Ellice Islands which was a territory that existed until 1976. The following illustrates an example Codelist of Reference Area:
With the dissolution of the Gilbert and Ellice Islands, the code “GE” is re-assigned to Georgia and Georgia’s existing code (“XX”) is discontinued.
With the use of Item Validity, a single Codelist can discontinue the use of the code “GE” to mean the Gilbert and Ellice Islands and re-assign it to Georgia. The use of Valid From and Valid To dates means that codes can be terminated and re-assigned as desired.
It is important to note that in the example above the Reference Area Codelist has two entries for the code “GE”. This is not permitted in a “standard” Codelist. However, since each of the GE entries has a Validity Period and these Validity Periods do not overlap, this is permissible. At no point in time are both codes valid simultaneously. If a further re-assignment to the code “GE” it is possible to have more entries for “GE” provided that the Validity Periods do not overlap.
Assigning Item Validity
Codelist Wizard – Specifying the Codelist Type
The Codelist Wizard in Fusion Registry allows the user to create and edit Codelists. In Step 1 of the wizard aside from the standard elements that make up a structure, the Codelist wizard adds a further choice control which allows a Codelist to be defined in one of three forms:
- A Standard Codelist – this allows a Validity Period to be specified on the Codelist which specifies the Codelist’s validity to a single date range, although this may be open-ended. All other structural metadata in the Registry has this concept.
- An Item-Validity Codelist – this is where the codes in the Codelist may be assigned individual validity periods. This means that over time, individual codes in the Codelist may become valid or invalid as specified by their code definition. This is particularly useful when re-assigning codes for a different semantic meaning.
- A Codelist Template – this is used to generate other Codelists from a single template. Like an Item-Validity Codelist, the Codelist Template permits validity at the item level. Please see below for an explanation of a Codelist Template.
It is not possible to assign Validity Periods to both the Codelist and individual codes.
Codelist Wizard – Assigning a Validity Period to an Item
The third step of the wizard allows the user to manually add, edit, and delete Codes in the Codelist. For editing and deleting codes, if the Codelist is of type “Item Validity” then the edit window will have entries for “Valid From” and “Valid To”. If it is a “standard” Codelist these fields will not be present.
Simply editing the “Valid From” and “Valid To” fields is enough to specify Item Validity to the currently edited item. The input takes SDMX Datetimes, so such values as 2001-Q1 and 2002-W50 are also permitted.
Note that unless a full date and time is specified, the Registry will infer that start dates always happen at the start of a period and that end dates will always be the end of a period. For example, typing 2001 into the “Valid From” field will result in a value of 2001-01-01T00:00:00 whereas entering 2001 into the “Valid To” field will result in a value of 2001-12-31T23:59:59. To avoid ambiguity, the Registry will always expand these values to a full date and time.
If a code has Item Validity, clicking the button “Add Validity Period” will bring up the edit window but allows for a second item with the same Code Id to be created, provided that the Validity Periods of the items with the same code, do not overlap.
Concept Scheme Wizard
To add Validity Periods to items in a Concept Scheme, the Concept Scheme wizard follows the same approach as the Codelist Wizard, except that on Step 3, it is not permissible to assign multiple validity periods to the same code.
Hierarchical Codelist Wizard
The Hierarchical Codelist does not permit the setting of an item type in Step 1, like the Codelist and Concept Scheme wizard. Instead on step 3 Validity Periods are assigned to an individual item via the control “Add Validity Period”.
It is permissible to add multiple Validity Periods to a particular item if desired.
Also note that a child code cannot have a validity period “wider” than its parent. If a parent code of “EUR” exists which has a validity of the year 1999 onwards, any children of EUR can only have validity periods from 1999 onward.
Codelist Map Wizard
Step 4 of the Codelist Map Wizard allows for the definition of a Code from a source to a target. On this step it is also possible to add a Validity Period to a single mapping
If a mapping has a validity period, it is shown with a calendar icon to state that this mapping is date dependent. To assign, remove or edit a Validity Period for a mapping, activate the button labelled “Modify Validity Period” and the following modal is displayed:
The input for each is an SDMX datetime. Clearing the values and clicking “Set Validity Period” will remove the mapping.
Viewing Item Validity
Codelist
When viewing the list of Codelists in the system, the Validity Type (as defined on Stage 1 of the Codelist Wizard) is displayed in the lower half of the screen. This value will either be “Standard”, “Item Validity”, “Template” or “Generated”.
When the user clicks “View Codelist”, if the Codelist does not have codes with validity at the item level, then the modal dialog shows a table of codes with the columns: Position, Id, Name and Description. If the Codelist has codes with validity at the item level then the Description column is not shown in the table and the Validity Period is shown instead.
Note that there may be duplicate rows for the same ID and that for these rows, the position is the same. Any rows which are considered invalid for today will be rendered in grey text. In the middle of the screen are controls for the time period and display of Invalid Codes.
View Time
The “View Time” is the time that the user is viewing this Codelist. By default, this will be blank. A blank value infers the current time.
The user may select a datetime by using the Time Period input field. This field allows the user to type in any valid SDMX Datetime Value (e.g., 2001-Q4) and when return is pressed, the table updates making any of the codes which are not valid for that time turn grey. This allows the user to see the state of a Codelist for any point in time.
There is a slide control for the hiding or showing of “invalid codes” (i.e. codes which are not valid for the current time period). By default, this is set to show invalid codes – rendered in grey. If this is set to “hide” then only currently valid codes are shown (rows which would have been rendered in grey are now not shown).
For viewing Hierarchical Codelists, Code Maps and Concept Schemes the same controls as in the Codelist will be available for those structures that have items with Validity Periods.
Codelist Templates
A Codelist Template allows Codelists to be generated from a definition, and for that definition to persist in the Fusion Registry to allow the Codelists to be re-generated at a future date.
A Codelist is defined as a Codelist Template is defined as such on Step 1 of the wizard. Validity Periods may be set on individual items in the same manner as a Codelist defined as “Item Validity”. When the template is submitted to the Registry, the Registry constructs further Codelists from the template. Each of these generated Codelists will be assigned a specific version and will have start and end dates on the Codelist (rather than the individual codes). These generated Codelists are identified in the Fusion Registry as having a type of “generated”. A generated Codelist is not editable or deletable in the Registry and attempting to perform either operation will affect the Template instead.
The following illustrates an example of how Codelist Templates work:
A Codelist Template is created with Agency “SDMX”, ID: “CL_TEMPLATE”. The Registry automatically assigns this a version of 0.1. The Codelist Template refers to 3 codes: Code A with no validity periods; Code B with a validity period of 2000-2010; Code C with a validity period where the start date is 2005 but no end date has been specified.
When the Template is submitted to the Registry, the following structures will be created:
- Final codelist SDMX:CL_TEMPLATE(1.0) – valid up to 2000, contains code A only
- Final codelist SDMX:CL_TEMPLATE(1.1) – valid from 2000 to 2005, contains codes A and B
- Final codelist SDMX CL_TEMPLATE(1.2) – valid from 2005 to 2010, contains codes A, B and C
- Final codelist SDMX:CL_TEMPLATE(2.0) – valid from 2010 onwards, contains codes A and C
The reason that there is a jump in version in the generated Codelists from 1.2 to 2.0 is that the 2.0 Codelist represents a non-backwardly-compatible change.
If the template is modified and then re-submitted, the existing generated templates are removed from the Registry and replaced by the new Template submission.
Deleting a Codelist Template from the Registry, will delete all Codelists that were generated from that Template. Templates may not be referenced by other structures and it is also not possible to convert a Template into another type of Codelist (e.g., Item Validity).
