Changelog for Fusion Metadata Registry

Version 11.6.0 (Released: 2023/4/13)

New Feature:
FMR11-44 Data Registration
FMR11-258 Web Service to refresh caches

Improvements:
FMR11-253 Update Apache POI to version 5
FMR11-254 Update Commons-fileupload third-party dependency https://github.com/bis-med-it/fmr/issues/391
FMR11-255 Update Jettison third-party dependency https://github.com/bis-med-it/fmr/issues/392
FMR11-256 Update Spring dependencies https://github.com/bis-med-it/fmr/issues/393

Bug Fixes:
FMR11-259 UI of Web Service Structure Query page does not implement "latest" correctly https://github.com/bis-med-it/fmr/issues/285
FMR11-260 Item Validity not returning most recent entry prior to today's date https://github.com/bis-med-it/fmr/issues/387
FMR11-261 Subscript out of range error with FusionXL when carriage return is in the name of a structure
FMR11-262 No transactions returned on SDMX Transactions page when using date slider
FMR11-264 Publication Table Editor: Dependent Variables, remove variable button does not work
FMR11-265 FusionXL resolve carriage returns in concept names

Version 11.5.2 (Released: 2023/3/8)

Changes:
FMR11-248 - Modification of interpration of SDMX-CSV V2 row https://github.com/bis-med-it/fmr/issues/349

Improvements:
FMR11-245 - Improve Performance of large Codelists which have Item Validity https://github.com/bis-med-it/fmr/issues/353
FMR11-246 - Update dependencies: spring, jstl, hibernate, httpcore, jackson, jersey,
FMR11-252 - Fusion Excel Export Data API now takes an additional 'attachment' parameter

Bug Fixes:
FMR11-240 - EDI should prevent Quadruplet attributes in Footnote Section https://github.com/bis-med-it/fmr/issues/347
FMR11-242 - EDI file missing mandatory Period Information validates successfully https://github.com/bis-med-it/fmr/issues/348
FMR11-243 - Improve validation of EDI DTM segment https://github.com/bis-med-it/fmr/issues/351 https://github.com/bis-med-it/fmr/issues/351
FMR11-244 - Temporary "MV" files not being cleared up after data transformation https://github.com/bis-med-it/fmr/issues/346 https://github.com/bis-med-it/fmr/issues/346
FMR11-247 - Unable to load Structure File when Mapping refers to DSD with ValueList coded dimension https://github.com/bis-med-it/fmr/issues/359
FMR11-250 - Parsing EDI fails to detect malformed period https://github.com/bis-med-it/fmr/issues/362
FMR11-251 - EDI REL segment must be followed by ARR segment https://github.com/bis-med-it/fmr/issues/322

Version 11.5.1 (Released: 2023/1/30)

New Feature:
FMR11-226 - Publication Table - support a scaling function applied to numerical observations https://github.com/bis-med-it/fmr/issues/300

Changes:
FMR11-232 - Annotations are ignored when importing 2.1 XML Hierarchical Codelists https://github.com/bis-med-it/fmr/issues/335
FMR11-238 - Use the standard "core" fastUtil library https://github.com/bis-med-it/fmr/issues/338

Improvements:
FMR11-236 - Ensure Implicit Mappings in Structure Maps validate codelists for valid intersection https://github.com/bis-med-it/fmr/issues/129
FMR11-237 - Provide option to skip Validation phase in Transform Web Service https://github.com/bis-med-it/fmr/issues/237

Bug Fixes:
FMR11-227 - Publication Table - Scientific Notation shows NaN in UI and is not applied to observation values https://github.com/bis-med-it/fmr/issues/300
FMR11-228 - Unable to import Codelists with multiple levels of inheritence in same submission as structures that use inherited codes https://github.com/bis-med-it/fmr/issues/308 https://github.com/bis-med-it/fmr/issues/103
FMR11-229 - Bulk actions - detail=raw is ignored https://github.com/bis-med-it/fmr/issues/103
FMR11-230 - REST query: detail=raw is not applied to referenced codelists https://github.com/bis-med-it/fmr/issues/103
FMR11-231 - IncValid output may contain empty series when it should not be generated https://github.com/bis-med-it/fmr/issues/332
FMR11-233 - Parent codes should be included in reference partial https://github.com/bis-med-it/fmr/issues/342
FMR11-234 - Fix 'too much recursion' error with reverse engineer data structure with large number of Dimensions https://github.com/bis-med-it/fmr/issues/339
FMR11-235 - Validate submission of Structure Map's Implicit mappings to ensure Coded mappings have a valid intersection from source to target https://github.com/bis-med-it/fmr/issues/129
FMR11-239 - Codelists with Annotations not being correctly parsed in SDMX-JSON https://github.com/bis-med-it/fmr/issues/345
FMR11-241 - Duplicate elements in SDMX CSV result in unexpected output https://github.com/bis-med-it/fmr/issues/343

Version 11.5.0 (Released: 2022/12/16)

New Features:
FMR11-208 - Audit Structure Queries FMR11-209 - Publication Table - Support Numerical Observation formatting https://github.com/MEDIT-Org/fmr/issues/300
FMR11-212 - Support Categorise Publication Table via the UI https://github.com/MEDIT-Org/fmr/issues/312
FMR11-223 - Improve Metrics Report with IncUnmappedReport option https://github.com/MEDIT-Org/fmr/issues/262 https://github.com/MEDIT-Org/fmr/issues/306 https://github.com/MEDIT-Org/fmr/issues/318 https://github.com/MEDIT-Org/fmr/issues/319
FMR11-209 - Publication Tables set scaling factor https://github.com/MEDIT-Org/fmr/issues/325
FMR11-225 - Support for "References-partial" with regular expressions https://github.com/MEDIT-Org/fmr/issues/298

Changes:
FMR11-213 - In SDMX CSV, SERIES_KEY and OBS_KEY will output with the quote marks around the whole key https://github.com/MEDIT-Org/fmr/issues/313
FMR11-219 - Validation and Transformation should preserve original format unless output format specified https://github.com/MEDIT-Org/fmr/issues/117

Improvements:
FMR11-221 - Update dependencies: hsqldb; jackson-databind; jettison; opencsv; spring; spring-kafka; spring-security
FMR11-222 - Improve Performance of Viewing a Hierarchy https://github.com/MEDIT-Org/fmr/issues/315

Bug Fixes:
FMR11-206 - Support for extended UTF-8 characters in FMR https://github.com/MEDIT-Org/fmr/issues/297
FMR11-207 - Transforming Data with "Delete" action can leave erroneous output in EDI format https://github.com/MEDIT-Org/fmr/issues/186
FMR11-210 - Fix an issue when "edi.datagroup.identifier" specified with value of asterisk
FMR11-211 - Validation on Group ID should ignore case https://github.com/MEDIT-Org/fmr/issues/301
FMR11-214 - When loading via Registry UI, Data file with Duplicate Series results in MV Store Error
FMR11-215 - Viewing a Representation Map has issue with "valid from" on Mappings https://github.com/MEDIT-Org/fmr/issues/310
FMR11-216 - Support dataset attribute consolidation (detection of duplicates) https://github.com/MEDIT-Org/fmr/issues/302
FMR11-217 - Publication Tables: Numerical Observation, "Add Rule" button does not work
FMR11-218 - Observation deletions are no longer written in EDI format https://github.com/MEDIT-Org/fmr/issues/303
FMR11-224 - Failure to report unmapped data when using TIME_PERIOD mapping https://github.com/MEDIT-Org/fmr/issues/305

Version 11.4.0 (Released: 2022/10/14)

Important Note for MySQL users: due to changes in the distribution of the MySQL connector (change FMR11-192 below), please read this article.

New Feature:
FMR11-194 - Publication Table - support casting dates more then once in same expression

Changes:
FMR11-190 - Open source licensing compatibility changes: Removal of alertifyjs dependency https://github.com/MEDIT-Org/fmr/issues/280
FMR11-191 - Open source licensing compatibility changes: Removal of dependency: jscolor https://github.com/MEDIT-Org/fmr/issues/282
FMR11-192 - Open source licensing compatibility changes: Remove MySQL Connector https://github.com/MEDIT-Org/fmr/issues/283
FMR11-193 - Open source licensing compatibility changes: Change FontAwesome to MIT Licensed version https://github.com/MEDIT-Org/fmr/issues/281

Improvement:
FMR11-202 - Update dependencies: Hibernate, Spring-Kafka, Jettison and Jersey https://github.com/MEDIT-Org/fmr/issues/294

Bug Fixes:
FMR11-189 - Behaviour of web service resolving multiple Codelist inheritance with "details=raw" https://github.com/MEDIT-Org/fmr/issues/257
FMR11-196 - Unable to read Excel Spreadsheet where all Frequencies are Formulas
FMR11-197 - Request for Codelist in SDMX-JSON, "details=raw", references="children" doesn't return children as raw https://github.com/MEDIT-Org/fmr/issues/257
FMR11-198 - Request for Codelist in SDMX-JSON, "details=raw", references="descendants" doesn't return descendants https://github.com/MEDIT-Org/fmr/issues/257
FMR11-199 - "Metadata Provider Scheme" not listed on Web Service -> Structures page
FMR11-200 - "Metadata Provider Wizard" does not allow Providers to start with a numeric yet the SDMX Standard allows this
FMR11-201 - "Metadata Provider Scheme" not listed on Cross References page
FMR11-202 - Update dependencies: Hibernate, Spring-Kafka, Jettison and Jersey https://github.com/MEDIT-Org/fmr/issues/294
FMR11-203 - Attaching Reference Metadata to Items in an item scheme fails to add link (cache issue)
FMR11-205 - Version Number not Displayed when Selected Item on Page Load

Version 11.3.0 (Released: 2022/09/23)

New Feature:
FMR11-187 - Support recovery of structure transactions from backup on installation https://github.com/MEDIT-Org/fmr/issues/277

Improvement:
FMR11-183 - Update dependencies https://github.com/MEDIT-Org/fmr/issues/277

Bug Fixes:
FMR11-181 - SDMX CSV v 2.0 Data Format does not report its format https://github.com/MEDIT-Org/fmr/issues/110
FMR11-184 - Support '*' for structure type in REST API https://github.com/MEDIT-Org/fmr/issues/273
FMR11-185 - Failure on Compare Revision shows Stack Trace https://github.com/MEDIT-Org/fmr/issues/278
FMR11-186 - SDMX-JSON Writer not outputting DSD as stub https://github.com/MEDIT-Org/fmr/issues/274

Version 11.2.2 (Released: 2022/09/13)

Improvement:
FMR11-182 - Modify ID generation strategy for importing Reference Metadata at v2.1

Version 11.2.1 (Released: 2022/09/09)

New Features:
FMR11-174 - Publication Tables: Formatting of Missing Values support ObsValue=NaN https://github.com/MEDIT-Org/fmr/issues/253
FMR11-175 - Support Reading of Reference Metadata (with restrictions) in version 2.1 SDMX https://github.com/MEDIT-Org/fmr/issues/250

Bug Fixes:
FMR11-169 - SDMX CSV v1 Data Reader does not allow header fields (row 1) to be double quoted
FMR11-170 - Bulk Export - spelling mistake on Raw help text
FMR11-171 - Order of revisions not as expected
FMR11-176 - OBS_PRE_BREAK validation corrected https://github.com/MEDIT-Org/fmr/issues/65
FMR11-178 - Quadruplet in EDI vs the other formats in the context of data consolidation or integration https://github.com/MEDIT-Org/fmr/issues/118
FMR11-179 - SDMX DSD output may include invalid attribute "isMultiLingual" https://github.com/MEDIT-Org/fmr/issues/255

Version 11.2.0 (Released: 2022/07/22)

New Features:
FMR11-164 - Support Reference Metadata
FMR11-165 - Support Certificate Authentication https://github.com/MEDIT-Org/fmr/issues/234 https://fmrwiki.sdmxcloud.org/Configure_X509_Certificate_Authentication

Improvement:
FMR11-146 - FMR dependency updates https://github.com/MEDIT-Org/fmr/issues/232
FMR11-159 - Add "Raw" option to Download structures and remove "Include internal settings" https://github.com/MEDIT-Org/fmr/issues/227
FMR11-160 - Allow a Concept Representation to be a ValueList https://github.com/MEDIT-Org/fmr/issues/83

Bug Fixes:
FMR11-162 - SERIES_KEY column should be reserved for series keys https://github.com/MEDIT-Org/fmr/issues/128
FMR11-161 - Issue loading Data Constraint with a Component element under Cube Region element https://github.com/MEDIT-Org/fmr/issues/210
FMR11-163 - Attribute variable does not work in table heading https://github.com/MEDIT-Org/fmr/issues/240
FMR11-166 - Duplicate attribute is not detected anymore in FMR https://github.com/MEDIT-Org/fmr/issues/143
FMR11-167 - Publication Table - Array Variables fail to save https://github.com/MEDIT-Org/fmr/issues/239
FMR11-168 - Error encountered whist attempting to deserialize structure https://github.com/MEDIT-Org/fmr/issues/210

Version 11.1.7 (Released: 2022/07/06)

New Feature:
FMR11-153 - Add Environment Sync back in from FR10 https://fmrwiki.sdmxcloud.org/Synchronise_structural_metadata_between_FMR_environments#Overview_-_FMR_Version_11.2

Improvement:
FMR11-158 - Publication Table: Footnotes work https://github.com/MEDIT-Org/fmr/issues/169

Bug Fixes:
FMR11-152 - Comma not being processed correctly in Web Service query for ItemBeans.
FMR11-154 - Unexpected structure validation error from FMR https://github.com/MEDIT-Org/fmr/issues/144
FMR11-155 - Structure validation errors are not detected anymore in FMR https://github.com/MEDIT-Org/fmr/issues/148
FMR11-156 - v2 structure REST API does not support queries for multiple items using the comma syntax https://github.com/MEDIT-Org/fmr/issues/220
FMR11-157 - Typo in header of Representation Maps page https://github.com/MEDIT-Org/fmr/issues/224

Version 11.1.6 (Released: 2022/06/23)

New Features:
FMR11-150 - Add "Duplicate Behaviour" Header Parameter to Web Services for Validation and Transformation - https://github.com/MEDIT-Org/fmr/issues/219
FMR11-151 - MVStore temporary store - https://github.com/MEDIT-Org/fmr/issues/158

Improvement:
FMR11-140 - Revert xsi:type=SiblingType, change the schema to output the Group as Sibling not SiblingType

Bug Fixes:
FMR11-141 - Re-opening of Alternate Half year Issue - https://github.com/MEDIT-Org/fmr/issues/141
FMR11-143 - TIME_FORMAT not being generated correctly
FMR11-144 - Problem with Date data type when reverse engineering a DSD from a CSV dataset - https://github.com/MEDIT-Org/fmr/issues/212
FMR11-145 - Problem with Description data type when reverse engineering a DSD from a CSV dataset - https://github.com/MEDIT-Org/fmr/issues/213
FMR11-147 - DSD Wizard step 3 - not allowing Enumerated Representation - https://github.com/MEDIT-Org/fmr/issues/216
FMR11-148 - Schema Query in Structure Format does not work

Version 11.1.5 (Released: 2022/06/01)

New Feature:
[FMR11-124] - Added support for MySQL 8
[FMR11-135] - Publication Table: Option to output rows with no data - https://github.com/MEDIT-Org/fmr/issues/172
[FMR11-136] - Dimension Value based on another Dimenisons variable value - https://github.com/MEDIT-Org/fmr/issues/173
[FMR11-137] - Publication Table: CPMI - Dynamic Rows and columns - https://github.com/MEDIT-Org/fmr/issues/170

Improvement:
[FMR11-138] - Modify FusionXL ready to open source - https://github.com/MEDIT-Org/fmr/issues/203
[FMR11-139] - Port FusionXL Data Author / Data read code and web services to FMR - https://github.com/MEDIT-Org/fmr/issues/202

Bug Fixes:
[FMR11-130] - Change UI and code to allow Measures to be mandatory / optional - https://github.com/MEDIT-Org/fmr/issues/197
[FMR11-131] - Add support for gzip encoding to FMR SDMX 2.0.0 REST endpoint - https://github.com/MEDIT-Org/fmr/issues/198
[FMR11-129] - Rename Hierarchical Codelists to Hierarchies - https://github.com/MEDIT-Org/fmr/issues/195
[FMR11-134] - KeyValueImpl has only partial support for multi-valued attributes - https://github.com/MEDIT-Org/fmr/issues/119

Version 11.1.4 (Released: 2022/05/20)

New Feature:
Data Publication Tables

Bug Fixes:
[FMR11-118] - Parsing of XLSX file not secure
[FMR11-120] - Metadata in SDMX-ML 3.0 format fails to parse due to unexpected annotations
[FMR11-123] - Update some third-party dependencies which have active vulnerabilities

Version 11.1.3 (Released: 2022/05/05)

Change:
[FMR-154] - Public License Agreement updated

Bug Fixes:
[FMR11-111] - NPE in FMR 11.1.1 when converting a flat CSV file
[FMR11-112] - V2 Structure Query Web Service not honouring Locale Filter
[FMR11-114] - FXL can not get structure beloning to SubAgency
[FMR11-115] - V2 Web service saveas=x.zip outputs a corrupt file (not a zip, not a file)
[FMR11-116] - UI crash on Report Template Wizard with HCL references
[FMR11-117] - FXL Validation can fail causing error

Version 11.1.2 (Released: 2022/04/11)

Improvement:
[FMR11-108] - Updated third-party dependencies

Bug Fixes:
[FMR11-109] - Corrected issue connecting using OpenLDAP
[FMR-107] - Unable to read / write Attributes where AttributeRelationship is at Dataflow level
[FMR-109] - Obs Key missing when onverting to SDMX CSV 2.0.0
[FMR-127] - Keys should use a dot in SDMX-CSV 2.0.0
[FMR-131] - Structure maintenance step 1 name and description language indicator is sometimes incorrect
[FMR-136] - Problem with importing CSV in Codelist : Use of Validity Version corrupts name in Codelist Wizard
[FMR-137] - Data Messages SDMX 1.0 should only support Update and Delete
[FMR-141] - Property TF_ALTERNATE_HALF_YEAR is not honored in FMR
[FMR-150] - Unable to connect to OpenLDAP with FMR version 11.1
[DEV-1063] - Forgotten password link removed from user interface

Version 11.1.1 (Released: 2022/02/18)

Bug Fixes:
[FMR11-102] - View structure map does not include time period mapping rules in table
[FMR11-101] - Convert data using mapping - error thrown
[FMR11-100] - Time Period map saved as Monthly Frequency gets Converted to Annual
[FMR11-99] - Unable to convert using mappings
[FMR11-98] - XML Codelist reader cannot read Codelists with multiple extensions
[FMR-96] - StaxDsdReaderEngineV3 forces MeasureRelationships to be at the Observation attachment level

Version 11.1.0 (Released: 2022/02/15)

Change:
FMR now uses the io.sdmx packages

New Feature:
[FMR11-89] - Add data validation rule to check that OBS_PRE_BREAK is consistent with OBS_STATUS

Bug Fixes:
[FMR11-97] - Dataset Attributes being lost when converting to SDMX JSON format
[FMR11-96] - Half-Yearly not working correctly in particular data files
[FMR11-95] - Structure Map Wizard cannot map Measures
[FMR11-94] - Multi-value attribute can get lost when converting CSV to CSV
[FMR11-92] - Zombie Structures are not detected

Version 11.0.3 (Released: 2022/02/04)

Bug Fixes:
[FMR11-84] - NullPointerException when writing Group Attributes
[FMR11-82] - Data supplied with bad time format do not error

Change:
[FMR11-83] - Remove support for the old "Fusion Security" security mechanism

Version 11.0.2 (Released: 2022/01/14)

Bug Fixes: [FMR11-80] - Validation Error with multiple measures and no time period
[FMR11-78] - MSD Wizard errors when opened
[FMR11-77] - Validation of OBS_VALUE and OBS_STATUS

Version 11.0.1 (Released: 2022/01/07)

Improvement:
[FMR11-66] - SDMX 2.1 Categorisations against Herarchical Code Lists should be applied to all relevant SDMX 3 Hierarchies

Bug Fix:
[FMR11-63] - Specifying a Custom Content Type throws a NullPointerException

Version 11.0.0 (Released: 2021/12/23)

New Features:

Supports SDMX 3.0 internal information mode
Support for input, storage and retrieval of all SDMX 3.0 structures with the exception of Metadata Constraints, Concept Scheme Map, Category Scheme Map
Automatic conversion of SDMX 2.1 structure sets to the SDMX 3.0 mapping model on load
Semantic versioning - three-digit version numbers (e.g. 1.3.2)
JNDI database connection support

Improvements:
[FR-5066] - Carriage Returns in Constraint Values should be ignored
[FR-5063] - Allow specification of Kafka behaviour on Registry Startup
[FR-5062] - Improve CPU performance and Memory Usage
[FR-5048] - Stack Trace not produced for Structure queries that provide no results
[FR-5046] - Add Wildcards support to Content Constraints
[FR-5044] - Update vulnerable dependencies
[FR-5036] - Quieten Stack Trace when "illegal" Contact ID specified in EDI file
[FR-5035] - Re-assigning Dataflow reference to DSD should check constraints
[FR-5039] - Add support for "dot" delimiter to Series Constraint Wizard

Changes:

Change to the structure of the operating database - making it incompatible with version 10 (see https://fmrwiki.sdmxcloud.org/Upgrading_to_Version_11)
Environment synchronisation function temporarily removed

[FR-5061] - Limit Registry to only generate first 1000 structure items for Fusion XL

Bug Fixes:
[FR-5051] - Constraints not being applied correctly
[FR-5049] - Reporting Template - Read Obs Attributes when no Observation Value reported
[FR-5022] - "Valid To" Validity Code does not set date correctly
[FR-5023] - Entering invalid dates on Step 2 of the Codelist Wizard hangs the User Interface
[FR-5025] - Value List Wizard Step 3 table rendering incorrectly
[FR-5034] - Registry Error Dialog can execute scripts which can lead to XSS attacks
[FR-5018] - Correct Transaction ID generated on Kafka notification
[FR-5004] - Kafka Structure Producer page, message detail drop-down not set correctly

Version 10.7.0 (Released 2021/05/13)

Improvements:
[FR-4977] - Option to send structures as stubs on Kafka
[FR-4979] - Data Validation new Option to supply the Sender Id
[FR-4984] - Mapping should be allowed to be applied to multiple datasets if they refer to the same dataset
[FR-4993] - Support the creation of the series attribute TIME_FORMAT from an EDI file

Change:
[FR-4904] - Change Kafka Service to allow status to be Publicly Available
[FR-4915] - Bulk Export download of illegal EDI structures do not clearly report the error
[FR-4917] - Add "keys" and "action" parameter to SDMX CSV Web Service request
[FR-4918] - Fix Issues in Transaction Web Services
[FR-4919] - Modification of a Maintainable should occur in 2 Transactions
[FR-4920] - Handle Large Loads in Reporting Constraint UI
[FR-4971] - Environment Sync - Version information on Item list set to 1.0 when maintainable parent is not 1.0
[FR-4978] - SDMX CSV validation: output DataSetId should be the Dataflow ID value

Bug Fixes:
[FR-4906] - Requesting an SDMX file which cannot be described in SDMX throws an error
[FR-4907] - Saving a DSD with a non-standard Measure Dimension prevents the DSD from being subsequently edited
[FR-4909] - DatasetID should be output in a Data Load when the input is CSV
[FR-4910] - Extend EDI Lenience to Codelists and Concept Schemes
[FR-4913] - HierarchicalCodeList does not update when order of hierarchy changed
[FR-4914] - Compare view not showing correct values for master and target
[FR-4923] - A colon ":" should always follow deptId, even when it's empty
[FR-4926] - Validation Rules Wizard displays data tables error on Step 2
[FR-4927] - Unable to load a generated dataset if the OBS_VAL dimension has been changed
[FR-4928] - Changing Dataflow does not update Valid Calculations on Dataset Details page
[FR-4929] - Validation Scheme Step 3 ignores a number as the Output value
[FR-4930] - Equality in SdmxBeansImpl has error
[FR-4967] - Environment Sync - Javascript error when connecting to endpoint
[FR-4972] - Comparing Reporting Constraints does not show differences
[FR-4973] - Reporting Template - trailing whitespace on HEX value causes obs attributes reported using colour to be missed
[FR-4980] - Codelist Wizard - adding a validity period caused Show CSV button to become inactive
[FR-4981] - Attempting to convert data file with multiple datatsets to SDMX CSV throws error
[FR-4982] - Unable to specify a DSD Dimension as both Coded and with restriction of Year
[FR-4986] - Enumerations of GregorianYearMonth are not present when viewing DSD
[FR-4988] - SDMX CSV file contains rows of different lengths
[FR-4989] - Group Level Attributes Lost in SDMX CSV
[FR-4990] - Dataset Level Attributes Lost writing to SDMX CSV
[FR-4994] - Enumerated Items should not have DateTime and Time as Data Format options
[FR-4995] - GregorianYearMonth is not shown when viewing the DSD
[FR-4997] - EDI Writer only writes a maximum of 3 contacts
[FR-4998] - Generated OBS_KEY in SDMX-CSV output incorrectly when no obs
[FR-4999] - Environmental Sync is unable to resolve Codelists which extend other Codelists
[FR-5000] - Comparing ContentConstraints does not give correct report
[FR-5001] - Comparing ReportingTemplates does not give correct report

Version 10.6.3.1 (Released 2021/04/25)

Bug Fixes:
[FR-4982] - Unable to specify a DSD Dimension as both Coded and with restriction of Year
[FR-4984] - Enumerations of GregorianYearMonth are not present when viewing DSD
[FR-4985] - DSD Wizard: 'dateTime' and 'time' should not be options for Enumerated format

Version 10.6.3 (Released 2021/01/22)

New Features:
[FR-4904] - Change Kafka Service to allow status to be Publicly Available
[FR-4863] - Kafka - add email user on forced producer termination \(when fatal error occurs\)
[FR-4861] - Kafka - support filter structures from notification by structure type
[FR-4860] - Kafka - record notification status against registry transaction status. Restart structure producer from last tx.
[FR-4859] - Kafka - support all connection properties when defining kafka server connection

Click here to read more about Kafka.

Bug Fixes:
[FR-4907] - Saving a DSD with a non-standard Measure Dimension prevents the DSD from being subsequently edited
[FR-4906] - Requesting an SDMX file which cannot be described in SDMX throws an error
[FR-4893] - Activity -> Structure Submissions page show a spinner if item selected in table and user not logged in
[Fr-4891] - NPE on startup when extended Codelist no longer exists
[FR-4884] - DSD Wizard issues with re-assigning Time Dimension
[FR-4879] - Schema Generation: Group Type does not match standards
[FR-4867] - UI delete and add TIME_PERIOD Concept with different Id causes Dimension to be misclassified
[FR-4866] - DSD Measure can be lost when updating structure
[FR-4840] - Loading an EDI file which has invalid values returns a 500 rather than throwing a FormatSpecific Error
[FR-4837] - Ensure CSV Dump honors double quotes around values containing commas
[FR-4830] - Outputting JSON data with Annotations can result in a -1 being returned
[FR-4787] - EDI File containing no department ID in CTA block cannot be validated
[FR-4788] - Dataset ID is not copied to outputted SDMX JSON
[FR-4789] - Activity -> Structure Submission Page - Compare button: nothing happens when clicked
[FR-4790] - If a user with no permissions is created, attempting to log on results in a Stack Trace on the screen
[FR-4792] - Unable to create a Dynamic DSD Attribute with multiple component inputs
[FR-4793] - EDI files are being created with message identity blocks greater than 35 characters and these are not erroring when read back in
[FR-4780] - The Registry can unify monthly data so reading and writing comes out in a consistent fashion
[FR-4771] - Temporary file not cleared away after reading Excel file

Version 10.5.7 (Released 2020/11/13)

New Features:
[FR-4768] - REST - normalise frequency support first in period and last in period functions
[FR-4753] - Registry now has option to clean up temporary directory on startup
[FR-4712] - Format of Code IDs can be restricted for a Codelist
[FR-4689] - Data Validation - support multiple structure references on header to identify which Dataflows or Provisions to use in validation when multiple datasets are in file
[FR-4673] - New parameter on Data Conversion to Merge Datasets
[FR-4666] - Add SSL support to Kafka connection
[FR-4670] - Add a System Property to ignore EDI Agency checks
[FR-4623] - Reporting Template: Hierarchy in row, codes with no data which have child codes with data to be displayed as a presentation node
[FR-4640] - Reporting Template Wrap text if it exceeds 'n' characters
[FR-4617] - Excel Reporting Template: Support Instruction sheets
[FR-4653] - Create property allowing DSI field in EDI file to be resolved to a Dataflow
[FR-4616] - Excel Reporting Template: Display of Codes and Description in the same field
[FR-4619] - Excel Reporting Templates: Protect the workbook so users cannot change the name of the worksheets or add/delete new worksheets
[FR-4597] - Web Service for Modifying and Obtaining System Properties
[FR-4477] - Allow HTTP Read Timeout duration to be viewed and modified via UI
[FR-4478] - Allow HTTP Connect Timeout duration to be viewed and modified via UI
[FR-4460] - View and modify User-Agent used in HTTP Requests from the Registry via the User Interface
[FR-4378] - REST API - Support path parameter: allowedconstraint
[FR-4379] - REST API - Support path parameter: allowedconstraint
[FR-4380] - REST API - Support query parameter: allcompletestubs
[FR-4381] - REST API - Support query parameter: referencecompletestubs
[FR-4394] - Exporting a DSD to EDI - support feature to enforce Primary Measure [1]
[FR-4409] - EDI Lenient mode - allow DSDs to be read and written with uncoded dimensions [2]
[FR-4408] - Add Value list to Bulk Export
[FR-4369] - Support Constraints against Value List
[FR-3302] - DSD Changes: Support zero to many measures - as per SDMX 3.0 working specification
[FR-4276] - Bulk Download offers SDMX-JSON in addition to Fusion-JSON as a format
[FR-4336] - Support Enumeration of items which can have non-SDMX valid IDs (Value List)
[FR-4337] - Reverse Engineer DSD from CSV Dataset
[FR-3305] - Populate an Excel Reporting Template from an SDMX Data File
[FR-2879] - full support for the official SDMX-JSON 1.0 standard
[FR-4114] - AD role mappings CSV import and export
[FR-3304] - Codelist inheritance
[FR-4115] - External table name mapping
[FR-4029] - CORS policy improvements
[FR-3946] - Stronger AES256 encryption
[FR-3561] - Kafka integration – Fusion Registry producer for structural metadata (including Kerberos integration for security)
[FR-4159] - Change the ID and Agency of existing structures
[FR-4072] - Secondary level caching for External SQL datastores, and mapped Dataflows
[FR-4099] - Environment Synchronisation accepts user log in allowing it to work with private or content-restricted registries
[FR-3967] - Active Directory configuration custom search filter
[FR-3951] - New Data Validator: Structure Validator
[FR-3962] - New Data Validator: Obs Status Validation
[FR-3892] - "Missing Time Periods" is its own validator

Improvement:
[FR-4770] - Add support to Fusion CSV to load a CSV file declaring Dimensions and measure dimensions only
[FR-4755] - New Web Service to allow users to 'Close' loaded Data
[FR-4732] - Adding Error Codes to Fusion Registry Validation Response
[FR-4748] - Support Loading of Fusion CSV Data
[FR-4750] - Report errors in EDI attribute validation more clearly
[FR-4730] - Enforce that the EDI FNS block is followed by the correct value
[FR-4688] - Remove 'zombie structures' from registry content, enable users to export or delete via a separate API
[FR-4303] - Kafka Connection Configuration - support multiple hosts
[FR-4660] - Support Quarterly and Half-Yearly formats in SDMX 1.0
[FR-4661] - Upgrade Excel to Structure Map creator to support Fusion Data Mapper v1.1.0
[FR-4652] - Report Templates: Conditional Colour on Attributes - make optional (opt in)
[FR-4632] - Improve Export Structures modal to warn if structures will not be exported in chosen format
[FR-4639] - Enhancement to calculation and interpretation of EDI DSI segment
[FR-4549] - Add 'Reference Partial' option to the export modal on bulk actions
[FR-4615] - Excel Reporting Template: Do not use color on attributes section
[FR-4581] - Add support for Groups in SDMX CSV
[FR-4587] - SDMX JSON Does not support the writing of Groups
[FR-4588] - SDMX JSON Does not support the reading of Groups
[FR-4590] - Issues with Monthly format and SDMX 2.0
[FR-4579] - Enhance the RSS feed to list URLs of added, modified and deleted structures
[FR-4580] - Improved performance in EDI Data Reader
[FR-4511] - Change VND Header for EDI to be consistent with SDMX structure and data syntax
[FR-4512] - Order Excel Report Template Colors in Legend by Codelist Code order
[FR-4214] - Add 'Export SDMX-ML' button to structure pages
[FR-4411] - Improve performance of cross reference resolution
[FR-4363] - Throw HTTP 406 (Not Acceptable) if a structure query explicitly requests the response in a format which is understood but not supported by any structures in the output
[FR-4361] - Simplify install wizard
[FR-4324] - Performance Improvements for Data Convert
[FR-4331] - Show the Hierarchical Codelist Agency/ID/Version of available Hierarchies in Validation Rules Wizard
[FR-4321] - EDI to treat DSI as Dataflow Identifier on Data Read
[FR-4322] - Report Template vnd header - if Data Provider Id is supplied and Agency is not, default Agency to SDMX
[FR-4329] - Remove formula embedded in Report Templates (replaced with checking table)
[FR-4263] - Reporting Template Worksheets to be built from a Dataflow as oppose to a Content Constraint
[FR-4227] - Sdmx JSON meta tag needs to have required node of "sender"
[FR-3882] - Registry should allow Half-Yearly to be specified as code "H"

Removed Feature:
[FR-4081] - Remove: In-memory structures storage

Bug Fixes:
[FR-4759] - Header ID can be incorrectly longer than 10 characters
[FR-4763] - CTA Segment should only be output in EDI if it is valid
[FR-4731] - InputStreams being incorrectly left open
[FR-4733] - Temporary files being created and not cleared away
[FR-4749] - POST requests ending with a slash prevent POST operation from completing successfully
[FR-4727] - Ensure logout after install of Fusion Registry
[FR-4707] - Structure Map - Value Map with Regular expression rejected as expression exceeds max length of Component
[FR-4709] - Unify Rest API VND Headers
[FR-4718] - Issue with format conversion to EDI when an observation level attribute is the only content of an observation
[FR-4701] - Codelist Wizard: re-ordering codes loses non-English languages for those codes
[FR-4702] - Codelist Wizard Step 4, clicking outside of the pop-up window dismisses window and reverts user changes
[FR-4703] - Saving a CSV Locale with 'no value' for the Group Separator value fails
[FR-4704] - StaxReader keeping InputStream open preventing temp file deletion
[FR-4705] - Codelist Wizard Step 4 - Show CSV does not honour languages correctly
[FR-4706] - Concept Scheme Wizard Step 4 - Show CSV does not honour languages correctly
[FR-4698] - Data Provider Constraint linked to Dimension which no longer exists causes registry startup to fail
[FR-4685] - EDI - observations not included in output when a series is written which has a start date earlier then any previous series of the same freequency, AND an end date later then any series of the same frequency
[FR-4674] - SDMX 2.0 data incorrectly stating KeyFamilyURI is an invalid attribute
[FR-4664] - Null pointer when saving a DSD with only 1 measure which does not have Id of OBS_VALUE
[FR-4658] - EDI DSI block should not be permitted to be longer than 18 characters
[FR-4655] - Install page of Fusion Registry - entering an invalid server URL completes the installation
[FR-4656] - Unable to specify space as locale Group Separator on CSV Locale page
[FR-4523] - Updating a DSD with optional Observation Attribute causes Index Out Of Bounds error if data already exists
[FR-4624] - IndexOutOfBounds can be thrown by validating data file
[FR-4626] - SDMXJsonDataReaderEngine is not correctly parsing observations when data file has groups
[FR-4631] - Changing the ID of a structure does not allow it to be obtained by ETag
[FR-4633] - Reporting Template - universe of series miscalculation when including a excluded series constraint with wildcards
[FR-4605] - CSV format (series per column) does not support hebrew characters
[FR-4607] - SDMX JSON - Unknown Attributes at the Series Level should throw an error
[FR-4612] - Half-Yearly issue error in Data handling
[FR-4613] - Issues with Monthly format and SDMX 2.1
[FR-4554] - When editing DSD, unable to change OBS_VALUE to another agency
[FR-4584] - EDI issue - FNS block must be followed by a REL
[FR-4596] - Recursive loop on reindex when 2 provisions linked to the same mapped dataflow
[FR-4598] - Ensure SDMX JSON and SDMX CSV do not support the 1 digit week format
[FR-4599] - Creating Schema can throw Null Pointer Exception
[FR-4524] - Weekly format incorrect for single digit weeks
[FR-4526] - Transforming data files with multiple Datasets should set the header action correctly
[FR-4527] - Converting file should create Group Type of SiblingType
[FR-4535] - Converting file to EDI loses Weekly Data
[FR-4537] - Ensure Generic Data Readers are version independent
[FR-4545] - Support SDMX Dates for Monthly format with the "M" character
[FR-4517] - Empty IDE block following NAD+MS should not be treated as an error
[FR-3281] - Organisations tab - Data Providers not properly sorted
[FR-4495] - JSON data file returning Format Specific error rather than Representation Error
[FR-4496] - Format Specific Errors should not all be at Dataset Level
[FR-4497] - EDI data file returning Format Specific Error rather than Representation error
[FR-4508] - Validation Service should return a 400 when supplied DSD is unknown
[FR-4510] - Role Mappings Organisations should be sorted alphabetically
[FR-4513] - Memory leak when generating Reporting Templates
[FR-4515] - Series Constraint Wizard - step 2 - requires the ability to select multiple Provision Agreements
[FR-4495] - JSON data file returning Format Specific error rather than Representation Error
[FR-4501] - gracefully handle startup failures due to missing internal references
[FR-4474] - Editing a DSD can cause Series and Group information to be lost
[FR-4479] - Hierarchical Codelist fusion-json level incorrectly outputs number, not the level Id that is referenced
[FR-4401] - Validation service: Duplicate attribute values
[FR-4458] - FormatSpecificError expected when reading CSV file
[FR-4461] - JSON: Unknown Attribute at the Observation level should throw Structure Error
[FR-4400] - EDI: Fix to reading of characters which fall outside of the ASCII range
[FR-4459] - EDI: TimePeriodFormat exception should be thrown when analysing attributes
[FR-4462] - EDI: Extra FNS segment causes issues
[FR-4463] - EDI: No IDE segment following the ARR one
[FR-4464] - EDI: Data segment allowed in Attributes section
[FR-4465] - SDMX-CSV data loaded via User Interface as Data Format: CSV with No DSD selected can cause Dataflow lookup error when enclosed in double quotes
[FR-4466] - Unable to load SDMX-CSV via UI when explicit DSD is selected
[FR-4467] - Data files with Weekly frequency can fail to be processed by the Registry
[FR-4471] - CSV - format conversion service is failing due to BOM issue
[FR-4470] - JSON: Format conversion issue - dataset attributes are dropped when converting to JSON data format
[FR-4475] - JSON: Representation error expected due to facet violation but no error thrown
[FR-4480] - SDMX-JSON Hierarchical Codelist Level reference should be by URN - currently it is by Id
[FR-4483] - User unable to create DSD when using "Reverse Engineer DSD from CSV Dataset" due to unclear Error
[FR-4487] - Bug in legacy mode (sdmx-json=fusion-json) for Reference Metadata
[FR-4488] - Bug in legacy mode for availability API
[FR-4482] - REST API references=all is returning descendants of parent structures, should only be children
[FR-4489] - REST API -with Fusion Store: 'offset' parameter used in conjunction with 'max' parameter, is not honoured when 'lastNObservations' is included in query
[FR-4268] - Converting Data - timeFormat = "normalized" - should always return time in ISO 8601
[FR-4292] - HCL Wizard formal Levels, resave causes error in level ID
[FR-4391] - Default value for version not respected in SDMX 2.1 dataset - structure reference is changed to latest version (should be 1.0)
[FR-4392] - IDE segment should only be used to look up DataStructrure (previously defaulted to Dataflow if a match was found)
[FR-4393] - Regression - Data validation for Representation missing from on Group attributes
[FR-4395] - Generic 2.0 files must permit "DataSet" attribute to be in namespace "Generic"
[FR-4399] - EDI: Uncoded Attribute Values must be checked to ensure they are valid EDI characters
[FR-4407] - WS Structure Query - Value List is not on dropdown
[FR-4412] - REST API Delete Method does not support non-SDMX structures
[FR-4420] - Value List - Export to Excel (from VL Page) not working
[FR-4422] - Value List - Annotations - Post add - Page not found
[FR-4423] - View Provision Agreement - Spinner does not disappear (intermittent)
[FR-4441] - referencepartial to treat constraint references as a union, not intersection
[FR-4449] - EDI Delete messages should throw an error if they contain observations or observation attributes
[FR-4453] - Constraint Wizard - edit constraint to add new attachment and remove another results in server side error
[FR-4454] - Illegal Value in EDI contact not reported correctly
[FR-4455] - Supplying an invalid DataStructure in EDI header causes a NullPointerException
[FR-4456] - converting a version 1.0 dataset to a version 2.0 keeps the action as Update (should convert to replace)
[FR-4373] - EDI CTA segment should throw error if Contact Name contains invalid characters
[FR-4376] - Display name and Username in AD and LDAP is wrong
[FR-4386] - SDMX-JSON structures - some keys are the wrong case with respect to the schema
[FR-4397] - SDMX-JSON syntax error on contact names, departments, and roles on Organisations
[FR-4295] - UI - Dataset Details page - View data button should only view the data for the currently selected dataset
[FR-4344] - Validate SDMX-JSON data not reporting error with missing Sender ID
[FR-4349] - Unable to save a DSD with a Primary Measure which uses a Value List for one of the Components
[FR-4350] - Unable to create a DSD when there is no concept with OBS_VALUE in the Registry
[FR-4351] - Primary Measure has incorrect URN with new DSD built through UI Wizard
[FR-4352] - EDI - report error when writing output with unsupported characters
[FR-4354] - EDI file - CTA segment should not restrict contact-id and contact-name to be of IDType
[FR-4358] - Server error when modifying report template from having an Attribute in a worksheet to becoming a fixed attribute
[FR-4359] - Client side issue - unable to modify Report template Attribute from being included in worksheet to having a fixed value
[FR-4362] - Do not output EDI structures with non-1.0 versions, or references to structures with non-1.0 versions, as they are not compatible with the EDI specification
[FR-4365] - structure query, implement references=organisationscheme, currently returns HTTP 400
[FR-4366] - structure query, references=all does not return children of parents
[FR-4315] - EDI issue: Missing REL segment after FNS+Attributes is not detected
[FR-4317] - EDI issue: FTX segments should not be present in EDI deletion messages but no error reported
[FR-4318] - EDI issue: CSV segments should not be present in EDI deletion messages but no error reported
[FR-4270] - EDI time formats are not set correctly when using EDIDataReader
[FR-4224] - Can not convert data to JSON format when no dataflow selected
[FR-4225] - "304 Not Modified" no longer being returned when header includes "If-Modified-Since"
[FR-4226] - Issue with reading SDMX-CSV with very large datasets
[FR-4228] - ByteOrderMark (BOM UTF8) preventing CSV read
[FR-4238] - SDMX JSON should report error when attribute indexes are wrong
[FR-4215] - Enable a dataflow mapping to be saved which is linked to a database, even if there is no data available
[FR-4205] - ws/fusion/info/product web service to include link for current user details
[FR-4187] - When specifying "Structure" in transform call, HTTP 400 is thrown rather than HTTP 200
[FR-4155] - EDI Interchange Header - Application Reference value should accept GESMES/CB and return HTTP 200 if illegal
[FR-4158] - HTTP 200 should be returned when header of EDI file has incorrect elements (ID, preparation date, test value)
[FR-4161] - Validating an EDI file ignores the specified Structure in favour of Structure referenced in EDI file itself
[FR-4058] - Mapping to EDI output is not putting the correct organisation in the sender field
[FR-4071] - Exception generated when mapping file
[FR-4100] - SDMX 1.0 Structure file with Receiver Element with Contact Element cannot be processed by Registry
[FR-4101] - Support SDMX 1.0 'Update' Action
[FR-4057] - xsi:schemaLocation should be specifiable at the Dataset element since this is valid XML
[FR-4035] - Validating Data - requesting Invalid values for EDI file with missing dimension causes 500 error
[FR-4038] - Validation EDI Time Series should report error if range specified but first and last values are not supplied
[FR-4043] - Enforce that TIME_FORMAT is never a mandatory attribute
[FR-4044] - Mapping Dataset Attributes to Series Attributes does not produce correct output
[FR-4028] - Data Validation of EDI series with missing dimensions should report error of type "Structure"
[FR-4033] - Reverting EDI TIME_FORMAT processing
[FR-4000] - Obs Status Validator is not honouring NaN as a missing value
[FR-4002] - Data Validation - EDI files should not report error for missing TIME_FORMAT attribute
[FR-4015] - Reading EDI files does not attempt to resolve Dataflow reference
[FR-4021] - Data Validation need to report EDI missing series attributes more clearly
[FR-3884] - Validation Service - ability to skip Mandatory Attributes data validator should be provided
[FR-3886] - Validation Service - Supplying invalid time definitions should return an error report
[FR-3887] - Validating data in EDI format should behave in the same manner as Structure Specific format
[FR-3889] - Validating data in Generic Data format should behave in the same manner as the Structure Specific format
[FR-3924] - Reported legal characters for ID state that asterisk may be used, when it shouldn't be
[FR-3930] - Dataflow Map - Combination Mapping shows in Mapped Components table in step 3 of the wizard when in edit mode
[FR-3931] - Data Structure and DataFlow Map Wizard - language issue
[FR-3938] - Data Validation - converting valid / invalid series to EDI
[FR-3941] - Data Validation - invalid EDI output missing any form of value
[FR-3964] - Unable to read Excel data file is FREQ is a Dataset Attribute
[FR-3965] - Dataset Attributes are not output when output format is specified as Excel
[FR-3972] - Errors in validating EDI headers should return HTTP 200 rather than 400
[FR-3973] - If EDI Message Trailer Segment (UNT) is incorrect, error is not clear to end user
[FR-3980] - Data Validation of EDI file that has duplicate Series Attributes returns HTTP 400
[FR-3875] - Provision Agreement Wizard does not honour language correctly on step 1
[FR-3925] - Adding a space to an Active Directory Role Mapping definition removes it
[FR-3929] - Fusion Registry Sdmx CSV Format can not be published to Registry if Dataset Attributes are included
[FR-3948] - Metadata Reports not showing Structure labels if language is not available
[FR-3953] - Data Sources Manager 'Table Names' modal has the 'Obs Table' and 'Key Table' header names transposed
[FR-3950] - Metadata Management Interface - Fix multi-page pagination on Manage Metadata
[FR-3949] - Metadata Management Interface - filters broken on manage metadata table for dimensions with numeric codes
[FR-3927] - Oracle: Error shown when starting tomcat

Change Log