Change Log

Changelog for Fusion Metadata Registry

Version 11.5.2 (Released: 2023/3/8)

Changes:
FMR11-248 - Modification of interpration of SDMX-CSV V2 row https://github.com/bis-med-it/fmr/issues/349

Improvements:
FMR11-245 - Improve Performance of large Codelists which have Item Validity https://github.com/bis-med-it/fmr/issues/353
FMR11-246 - Update dependencies: spring, jstl, hibernate, httpcore, jackson, jersey,
FMR11-252 - Fusion Excel Export Data API now takes an additional 'attachment' parameter

Bug Fixes:
FMR11-240 - EDI should prevent Quadruplet attributes in Footnote Section https://github.com/bis-med-it/fmr/issues/347
FMR11-242 - EDI file missing mandatory Period Information validates successfully https://github.com/bis-med-it/fmr/issues/348
FMR11-243 - Improve validation of EDI DTM segment https://github.com/bis-med-it/fmr/issues/351 https://github.com/bis-med-it/fmr/issues/351
FMR11-244 - Temporary "MV" files not being cleared up after data transformation https://github.com/bis-med-it/fmr/issues/346 https://github.com/bis-med-it/fmr/issues/346
FMR11-247 - Unable to load Structure File when Mapping refers to DSD with ValueList coded dimension https://github.com/bis-med-it/fmr/issues/359
FMR11-250 - Parsing EDI fails to detect malformed period https://github.com/bis-med-it/fmr/issues/362
FMR11-251 - EDI REL segment must be followed by ARR segment https://github.com/bis-med-it/fmr/issues/322

Version 11.5.1 (Released: 2023/1/30)

New Feature:
FMR11-226 - Publication Table - support a scaling function applied to numerical observations https://github.com/bis-med-it/fmr/issues/300

Changes:
FMR11-232 - Annotations are ignored when importing 2.1 XML Hierarchical Codelists https://github.com/bis-med-it/fmr/issues/335
FMR11-238 - Use the standard "core" fastUtil library https://github.com/bis-med-it/fmr/issues/338

Improvements:
FMR11-236 - Ensure Implicit Mappings in Structure Maps validate codelists for valid intersection https://github.com/bis-med-it/fmr/issues/129
FMR11-237 - Provide option to skip Validation phase in Transform Web Service https://github.com/bis-med-it/fmr/issues/237

Bug Fixes:
FMR11-227 - Publication Table - Scientific Notation shows NaN in UI and is not applied to observation values https://github.com/bis-med-it/fmr/issues/300
FMR11-228 - Unable to import Codelists with multiple levels of inheritence in same submission as structures that use inherited codes https://github.com/bis-med-it/fmr/issues/308 https://github.com/bis-med-it/fmr/issues/103
FMR11-229 - Bulk actions - detail=raw is ignored https://github.com/bis-med-it/fmr/issues/103
FMR11-230 - REST query: detail=raw is not applied to referenced codelists https://github.com/bis-med-it/fmr/issues/103
FMR11-231 - IncValid output may contain empty series when it should not be generated https://github.com/bis-med-it/fmr/issues/332
FMR11-233 - Parent codes should be included in reference partial https://github.com/bis-med-it/fmr/issues/342
FMR11-234 - Fix 'too much recursion' error with reverse engineer data structure with large number of Dimensions https://github.com/bis-med-it/fmr/issues/339
FMR11-235 - Validate submission of Structure Map's Implicit mappings to ensure Coded mappings have a valid intersection from source to target https://github.com/bis-med-it/fmr/issues/129
FMR11-239 - Codelists with Annotations not being correctly parsed in SDMX-JSON https://github.com/bis-med-it/fmr/issues/345
FMR11-241 - Duplicate elements in SDMX CSV result in unexpected output https://github.com/bis-med-it/fmr/issues/343

Version 11.5.0 (Released: 2022/12/16)

New Features:
FMR11-208 - Audit Structure Queries FMR11-209 - Publication Table - Support Numerical Observation formatting https://github.com/MEDIT-Org/fmr/issues/300
FMR11-212 - Support Categorise Publication Table via the UI https://github.com/MEDIT-Org/fmr/issues/312
FMR11-223 - Improve Metrics Report with IncUnmappedReport option https://github.com/MEDIT-Org/fmr/issues/262 https://github.com/MEDIT-Org/fmr/issues/306 https://github.com/MEDIT-Org/fmr/issues/318 https://github.com/MEDIT-Org/fmr/issues/319
FMR11-209 - Publication Tables set scaling factor https://github.com/MEDIT-Org/fmr/issues/325
FMR11-225 - Support for "References-partial" with regular expressions https://github.com/MEDIT-Org/fmr/issues/298

Changes:
FMR11-213 - In SDMX CSV, SERIES_KEY and OBS_KEY will output with the quote marks around the whole key https://github.com/MEDIT-Org/fmr/issues/313
FMR11-219 - Validation and Transformation should preserve original format unless output format specified https://github.com/MEDIT-Org/fmr/issues/117

Improvements:
FMR11-221 - Update dependencies: hsqldb; jackson-databind; jettison; opencsv; spring; spring-kafka; spring-security
FMR11-222 - Improve Performance of Viewing a Hierarchy https://github.com/MEDIT-Org/fmr/issues/315

Bug Fixes:
FMR11-206 - Support for extended UTF-8 characters in FMR https://github.com/MEDIT-Org/fmr/issues/297
FMR11-207 - Transforming Data with "Delete" action can leave erroneous output in EDI format https://github.com/MEDIT-Org/fmr/issues/186
FMR11-210 - Fix an issue when "edi.datagroup.identifier" specified with value of asterisk
FMR11-211 - Validation on Group ID should ignore case https://github.com/MEDIT-Org/fmr/issues/301
FMR11-214 - When loading via Registry UI, Data file with Duplicate Series results in MV Store Error
FMR11-215 - Viewing a Representation Map has issue with "valid from" on Mappings https://github.com/MEDIT-Org/fmr/issues/310
FMR11-216 - Support dataset attribute consolidation (detection of duplicates) https://github.com/MEDIT-Org/fmr/issues/302
FMR11-217 - Publication Tables: Numerical Observation, "Add Rule" button does not work
FMR11-218 - Observation deletions are no longer written in EDI format https://github.com/MEDIT-Org/fmr/issues/303
FMR11-224 - Failure to report unmapped data when using TIME_PERIOD mapping https://github.com/MEDIT-Org/fmr/issues/305

Version 11.4.0 (Released: 2022/10/14)

Important Note for MySQL users: due to changes in the distribution of the MySQL connector (change FMR11-192 below), please read this article.

New Feature:
FMR11-194 - Publication Table - support casting dates more then once in same expression

Changes:
FMR11-190 - Open source licensing compatibility changes: Removal of alertifyjs dependency https://github.com/MEDIT-Org/fmr/issues/280
FMR11-191 - Open source licensing compatibility changes: Removal of dependency: jscolor https://github.com/MEDIT-Org/fmr/issues/282
FMR11-192 - Open source licensing compatibility changes: Remove MySQL Connector https://github.com/MEDIT-Org/fmr/issues/283
FMR11-193 - Open source licensing compatibility changes: Change FontAwesome to MIT Licensed version https://github.com/MEDIT-Org/fmr/issues/281

Improvement:
FMR11-202 - Update dependencies: Hibernate, Spring-Kafka, Jettison and Jersey https://github.com/MEDIT-Org/fmr/issues/294

Bug Fixes:
FMR11-189 - Behaviour of web service resolving multiple Codelist inheritance with "details=raw" https://github.com/MEDIT-Org/fmr/issues/257
FMR11-196 - Unable to read Excel Spreadsheet where all Frequencies are Formulas
FMR11-197 - Request for Codelist in SDMX-JSON, "details=raw", references="children" doesn't return children as raw https://github.com/MEDIT-Org/fmr/issues/257
FMR11-198 - Request for Codelist in SDMX-JSON, "details=raw", references="descendants" doesn't return descendants https://github.com/MEDIT-Org/fmr/issues/257
FMR11-199 - "Metadata Provider Scheme" not listed on Web Service -> Structures page
FMR11-200 - "Metadata Provider Wizard" does not allow Providers to start with a numeric yet the SDMX Standard allows this
FMR11-201 - "Metadata Provider Scheme" not listed on Cross References page
FMR11-202 - Update dependencies: Hibernate, Spring-Kafka, Jettison and Jersey https://github.com/MEDIT-Org/fmr/issues/294
FMR11-203 - Attaching Reference Metadata to Items in an item scheme fails to add link (cache issue)
FMR11-205 - Version Number not Displayed when Selected Item on Page Load

Version 11.3.0 (Released: 2022/09/23)

New Feature:
FMR11-187 - Support recovery of structure transactions from backup on installation https://github.com/MEDIT-Org/fmr/issues/277

Improvement:
FMR11-183 - Update dependencies https://github.com/MEDIT-Org/fmr/issues/277

Bug Fixes:
FMR11-181 - SDMX CSV v 2.0 Data Format does not report its format https://github.com/MEDIT-Org/fmr/issues/110
FMR11-184 - Support '*' for structure type in REST API https://github.com/MEDIT-Org/fmr/issues/273
FMR11-185 - Failure on Compare Revision shows Stack Trace https://github.com/MEDIT-Org/fmr/issues/278
FMR11-186 - SDMX-JSON Writer not outputting DSD as stub https://github.com/MEDIT-Org/fmr/issues/274

Version 11.2.2 (Released: 2022/09/13)

Improvement:
FMR11-182 - Modify ID generation strategy for importing Reference Metadata at v2.1

Version 11.2.1 (Released: 2022/09/09)

New Features:
FMR11-174 - Publication Tables: Formatting of Missing Values support ObsValue=NaN https://github.com/MEDIT-Org/fmr/issues/253
FMR11-175 - Support Reading of Reference Metadata (with restrictions) in version 2.1 SDMX https://github.com/MEDIT-Org/fmr/issues/250

Bug Fixes:
FMR11-169 - SDMX CSV v1 Data Reader does not allow header fields (row 1) to be double quoted
FMR11-170 - Bulk Export - spelling mistake on Raw help text
FMR11-171 - Order of revisions not as expected
FMR11-176 - OBS_PRE_BREAK validation corrected https://github.com/MEDIT-Org/fmr/issues/65
FMR11-178 - Quadruplet in EDI vs the other formats in the context of data consolidation or integration https://github.com/MEDIT-Org/fmr/issues/118
FMR11-179 - SDMX DSD output may include invalid attribute "isMultiLingual" https://github.com/MEDIT-Org/fmr/issues/255

Version 11.2.0 (Released: 2022/07/22)

New Features:
FMR11-164 - Support Reference Metadata
FMR11-165 - Support Certificate Authentication https://github.com/MEDIT-Org/fmr/issues/234 https://fmrwiki.sdmxcloud.org/Configure_X509_Certificate_Authentication

Improvement:
FMR11-146 - FMR dependency updates https://github.com/MEDIT-Org/fmr/issues/232
FMR11-159 - Add "Raw" option to Download structures and remove "Include internal settings" https://github.com/MEDIT-Org/fmr/issues/227
FMR11-160 - Allow a Concept Representation to be a ValueList https://github.com/MEDIT-Org/fmr/issues/83

Bug Fixes:
FMR11-162 - SERIES_KEY column should be reserved for series keys https://github.com/MEDIT-Org/fmr/issues/128
FMR11-161 - Issue loading Data Constraint with a Component element under Cube Region element https://github.com/MEDIT-Org/fmr/issues/210
FMR11-163 - Attribute variable does not work in table heading https://github.com/MEDIT-Org/fmr/issues/240
FMR11-166 - Duplicate attribute is not detected anymore in FMR https://github.com/MEDIT-Org/fmr/issues/143
FMR11-167 - Publication Table - Array Variables fail to save https://github.com/MEDIT-Org/fmr/issues/239
FMR11-168 - Error encountered whist attempting to deserialize structure https://github.com/MEDIT-Org/fmr/issues/210

Version 11.1.7 (Released: 2022/07/06)

New Feature:
FMR11-153 - Add Environment Sync back in from FR10 https://fmrwiki.sdmxcloud.org/Synchronise_structural_metadata_between_FMR_environments#Overview_-_FMR_Version_11.2

Improvement:
FMR11-158 - Publication Table: Footnotes work https://github.com/MEDIT-Org/fmr/issues/169

Bug Fixes:
FMR11-152 - Comma not being processed correctly in Web Service query for ItemBeans.
FMR11-154 - Unexpected structure validation error from FMR https://github.com/MEDIT-Org/fmr/issues/144
FMR11-155 - Structure validation errors are not detected anymore in FMR https://github.com/MEDIT-Org/fmr/issues/148
FMR11-156 - v2 structure REST API does not support queries for multiple items using the comma syntax https://github.com/MEDIT-Org/fmr/issues/220
FMR11-157 - Typo in header of Representation Maps page https://github.com/MEDIT-Org/fmr/issues/224

Version 11.1.6 (Released: 2022/06/23)

New Features:
FMR11-150 - Add "Duplicate Behaviour" Header Parameter to Web Services for Validation and Transformation - https://github.com/MEDIT-Org/fmr/issues/219
FMR11-151 - MVStore temporary store - https://github.com/MEDIT-Org/fmr/issues/158

Improvement:
FMR11-140 - Revert xsi:type=SiblingType, change the schema to output the Group as Sibling not SiblingType

Bug Fixes:
FMR11-141 - Re-opening of Alternate Half year Issue - https://github.com/MEDIT-Org/fmr/issues/141
FMR11-143 - TIME_FORMAT not being generated correctly
FMR11-144 - Problem with Date data type when reverse engineering a DSD from a CSV dataset - https://github.com/MEDIT-Org/fmr/issues/212
FMR11-145 - Problem with Description data type when reverse engineering a DSD from a CSV dataset - https://github.com/MEDIT-Org/fmr/issues/213
FMR11-147 - DSD Wizard step 3 - not allowing Enumerated Representation - https://github.com/MEDIT-Org/fmr/issues/216
FMR11-148 - Schema Query in Structure Format does not work

Version 11.1.5 (Released: 2022/06/01)

New Feature:
[FMR11-124] - Added support for MySQL 8
[FMR11-135] - Publication Table: Option to output rows with no data - https://github.com/MEDIT-Org/fmr/issues/172
[FMR11-136] - Dimension Value based on another Dimenisons variable value - https://github.com/MEDIT-Org/fmr/issues/173
[FMR11-137] - Publication Table: CPMI - Dynamic Rows and columns - https://github.com/MEDIT-Org/fmr/issues/170

Improvement:
[FMR11-138] - Modify FusionXL ready to open source - https://github.com/MEDIT-Org/fmr/issues/203
[FMR11-139] - Port FusionXL Data Author / Data read code and web services to FMR - https://github.com/MEDIT-Org/fmr/issues/202

Bug Fixes:
[FMR11-130] - Change UI and code to allow Measures to be mandatory / optional - https://github.com/MEDIT-Org/fmr/issues/197
[FMR11-131] - Add support for gzip encoding to FMR SDMX 2.0.0 REST endpoint - https://github.com/MEDIT-Org/fmr/issues/198
[FMR11-129] - Rename Hierarchical Codelists to Hierarchies - https://github.com/MEDIT-Org/fmr/issues/195
[FMR11-134] - KeyValueImpl has only partial support for multi-valued attributes - https://github.com/MEDIT-Org/fmr/issues/119

Version 11.1.4 (Released: 2022/05/20)

New Feature:
Data Publication Tables

Bug Fixes:
[FMR11-118] - Parsing of XLSX file not secure
[FMR11-120] - Metadata in SDMX-ML 3.0 format fails to parse due to unexpected annotations
[FMR11-123] - Update some third-party dependencies which have active vulnerabilities

Version 11.1.3 (Released: 2022/05/05)

Change:
[FMR-154] - Public License Agreement updated

Bug Fixes:
[FMR11-111] - NPE in FMR 11.1.1 when converting a flat CSV file
[FMR11-112] - V2 Structure Query Web Service not honouring Locale Filter
[FMR11-114] - FXL can not get structure beloning to SubAgency
[FMR11-115] - V2 Web service saveas=x.zip outputs a corrupt file (not a zip, not a file)
[FMR11-116] - UI crash on Report Template Wizard with HCL references
[FMR11-117] - FXL Validation can fail causing error

Version 11.1.2 (Released: 2022/04/11)

Improvement:
[FMR11-108] - Updated third-party dependencies

Bug Fixes:
[FMR11-109] - Corrected issue connecting using OpenLDAP
[FMR-107] - Unable to read / write Attributes where AttributeRelationship is at Dataflow level
[FMR-109] - Obs Key missing when onverting to SDMX CSV 2.0.0
[FMR-127] - Keys should use a dot in SDMX-CSV 2.0.0
[FMR-131] - Structure maintenance step 1 name and description language indicator is sometimes incorrect
[FMR-136] - Problem with importing CSV in Codelist : Use of Validity Version corrupts name in Codelist Wizard
[FMR-137] - Data Messages SDMX 1.0 should only support Update and Delete
[FMR-141] - Property TF_ALTERNATE_HALF_YEAR is not honored in FMR
[FMR-150] - Unable to connect to OpenLDAP with FMR version 11.1
[DEV-1063] - Forgotten password link removed from user interface

Version 11.1.1 (Released: 2022/02/18)

Bug Fixes:
[FMR11-102] - View structure map does not include time period mapping rules in table
[FMR11-101] - Convert data using mapping - error thrown
[FMR11-100] - Time Period map saved as Monthly Frequency gets Converted to Annual
[FMR11-99] - Unable to convert using mappings
[FMR11-98] - XML Codelist reader cannot read Codelists with multiple extensions
[FMR-96] - StaxDsdReaderEngineV3 forces MeasureRelationships to be at the Observation attachment level

Version 11.1.0 (Released: 2022/02/15)

Change:
FMR now uses the io.sdmx packages

New Feature:
[FMR11-89] - Add data validation rule to check that OBS_PRE_BREAK is consistent with OBS_STATUS

Bug Fixes:
[FMR11-97] - Dataset Attributes being lost when converting to SDMX JSON format
[FMR11-96] - Half-Yearly not working correctly in particular data files
[FMR11-95] - Structure Map Wizard cannot map Measures
[FMR11-94] - Multi-value attribute can get lost when converting CSV to CSV
[FMR11-92] - Zombie Structures are not detected

Version 11.0.3 (Released: 2022/02/04)

Bug Fixes:
[FMR11-84] - NullPointerException when writing Group Attributes
[FMR11-82] - Data supplied with bad time format do not error

Change:
[FMR11-83] - Remove support for the old "Fusion Security" security mechanism

Version 11.0.2 (Released: 2022/01/14)

Bug Fixes: [FMR11-80] - Validation Error with multiple measures and no time period
[FMR11-78] - MSD Wizard errors when opened
[FMR11-77] - Validation of OBS_VALUE and OBS_STATUS

Version 11.0.1 (Released: 2022/01/07)

Improvement:
[FMR11-66] - SDMX 2.1 Categorisations against Herarchical Code Lists should be applied to all relevant SDMX 3 Hierarchies

Bug Fix:
[FMR11-63] - Specifying a Custom Content Type throws a NullPointerException

Version 11.0.0 (Released: 2021/12/23)

New Features:

• Supports SDMX 3.0 internal information mode
• Support for input, storage and retrieval of all SDMX 3.0 structures with the exception of Metadata Constraints, Concept Scheme Map, Category Scheme Map
• Automatic conversion of SDMX 2.1 structure sets to the SDMX 3.0 mapping model on load
• Semantic versioning - three-digit version numbers (e.g. 1.3.2)
• JNDI database connection support

Improvements:
[FR-5066] - Carriage Returns in Constraint Values should be ignored
[FR-5063] - Allow specification of Kafka behaviour on Registry Startup
[FR-5062] - Improve CPU performance and Memory Usage
[FR-5048] - Stack Trace not produced for Structure queries that provide no results
[FR-5046] - Add Wildcards support to Content Constraints
[FR-5044] - Update vulnerable dependencies
[FR-5036] - Quieten Stack Trace when "illegal" Contact ID specified in EDI file
[FR-5035] - Re-assigning Dataflow reference to DSD should check constraints
[FR-5039] - Add support for "dot" delimiter to Series Constraint Wizard

Changes:

• Change to the structure of the operating database - making it incompatible with version 10 (see https://fmrwiki.sdmxcloud.org/Upgrading_to_Version_11)
• Environment synchronisation function temporarily removed

[FR-5061] - Limit Registry to only generate first 1000 structure items for Fusion XL

Bug Fixes:
[FR-5051] - Constraints not being applied correctly
[FR-5049] - Reporting Template - Read Obs Attributes when no Observation Value reported
[FR-5022] - "Valid To" Validity Code does not set date correctly
[FR-5023] - Entering invalid dates on Step 2 of the Codelist Wizard hangs the User Interface
[FR-5025] - Value List Wizard Step 3 table rendering incorrectly
[FR-5034] - Registry Error Dialog can execute scripts which can lead to XSS attacks
[FR-5018] - Correct Transaction ID generated on Kafka notification
[FR-5004] - Kafka Structure Producer page, message detail drop-down not set correctly

Version 10.7.0 (Released 2021/05/13)

Improvements:
[FR-4977] - Option to send structures as stubs on Kafka
[FR-4979] - Data Validation new Option to supply the Sender Id
[FR-4984] - Mapping should be allowed to be applied to multiple datasets if they refer to the same dataset
[FR-4993] - Support the creation of the series attribute TIME_FORMAT from an EDI file

Change:
[FR-4904] - Change Kafka Service to allow status to be Publicly Available
[FR-4915] - Bulk Export download of illegal EDI structures do not clearly report the error
[FR-4917] - Add "keys" and "action" parameter to SDMX CSV Web Service request
[FR-4918] - Fix Issues in Transaction Web Services
[FR-4919] - Modification of a Maintainable should occur in 2 Transactions
[FR-4920] - Handle Large Loads in Reporting Constraint UI
[FR-4971] - Environment Sync - Version information on Item list set to 1.0 when maintainable parent is not 1.0
[FR-4978] - SDMX CSV validation: output DataSetId should be the Dataflow ID value

Bug Fixes:
[FR-4906] - Requesting an SDMX file which cannot be described in SDMX throws an error
[FR-4907] - Saving a DSD with a non-standard Measure Dimension prevents the DSD from being subsequently edited
[FR-4909] - DatasetID should be output in a Data Load when the input is CSV
[FR-4910] - Extend EDI Lenience to Codelists and Concept Schemes
[FR-4913] - HierarchicalCodeList does not update when order of hierarchy changed
[FR-4914] - Compare view not showing correct values for master and target
[FR-4923] - A colon ":" should always follow deptId, even when it's empty
[FR-4926] - Validation Rules Wizard displays data tables error on Step 2
[FR-4927] - Unable to load a generated dataset if the OBS_VAL dimension has been changed
[FR-4928] - Changing Dataflow does not update Valid Calculations on Dataset Details page
[FR-4929] - Validation Scheme Step 3 ignores a number as the Output value
[FR-4930] - Equality in SdmxBeansImpl has error
[FR-4967] - Environment Sync - Javascript error when connecting to endpoint
[FR-4972] - Comparing Reporting Constraints does not show differences
[FR-4973] - Reporting Template - trailing whitespace on HEX value causes obs attributes reported using colour to be missed
[FR-4980] - Codelist Wizard - adding a validity period caused Show CSV button to become inactive
[FR-4981] - Attempting to convert data file with multiple datatsets to SDMX CSV throws error
[FR-4982] - Unable to specify a DSD Dimension as both Coded and with restriction of Year
[FR-4986] - Enumerations of GregorianYearMonth are not present when viewing DSD
[FR-4988] - SDMX CSV file contains rows of different lengths
[FR-4989] - Group Level Attributes Lost in SDMX CSV
[FR-4990] - Dataset Level Attributes Lost writing to SDMX CSV
[FR-4994] - Enumerated Items should not have DateTime and Time as Data Format options
[FR-4995] - GregorianYearMonth is not shown when viewing the DSD
[FR-4997] - EDI Writer only writes a maximum of 3 contacts
[FR-4998] - Generated OBS_KEY in SDMX-CSV output incorrectly when no obs
[FR-4999] - Environmental Sync is unable to resolve Codelists which extend other Codelists
[FR-5000] - Comparing ContentConstraints does not give correct report
[FR-5001] - Comparing ReportingTemplates does not give correct report

Version 10.6.3.1 (Released 2021/04/25)

Bug Fixes:
[FR-4982] - Unable to specify a DSD Dimension as both Coded and with restriction of Year
[FR-4984] - Enumerations of GregorianYearMonth are not present when viewing DSD
[FR-4985] - DSD Wizard: 'dateTime' and 'time' should not be options for Enumerated format

Version 10.6.3 (Released 2021/01/22)

New Features:
[FR-4904] - Change Kafka Service to allow status to be Publicly Available
[FR-4863] - Kafka - add email user on forced producer termination \(when fatal error occurs\)
[FR-4861] - Kafka - support filter structures from notification by structure type
[FR-4860] - Kafka - record notification status against registry transaction status. Restart structure producer from last tx.
[FR-4859] - Kafka - support all connection properties when defining kafka server connection

Click here to read more about Kafka.

Bug Fixes:
[FR-4907] - Saving a DSD with a non-standard Measure Dimension prevents the DSD from being subsequently edited
[FR-4906] - Requesting an SDMX file which cannot be described in SDMX throws an error
[FR-4893] - Activity -> Structure Submissions page show a spinner if item selected in table and user not logged in
[Fr-4891] - NPE on startup when extended Codelist no longer exists
[FR-4884] - DSD Wizard issues with re-assigning Time Dimension
[FR-4879] - Schema Generation: Group Type does not match standards
[FR-4867] - UI delete and add TIME_PERIOD Concept with different Id causes Dimension to be misclassified
[FR-4866] - DSD Measure can be lost when updating structure
[FR-4840] - Loading an EDI file which has invalid values returns a 500 rather than throwing a FormatSpecific Error
[FR-4837] - Ensure CSV Dump honors double quotes around values containing commas
[FR-4830] - Outputting JSON data with Annotations can result in a -1 being returned
[FR-4787] - EDI File containing no department ID in CTA block cannot be validated
[FR-4788] - Dataset ID is not copied to outputted SDMX JSON
[FR-4789] - Activity -> Structure Submission Page - Compare button: nothing happens when clicked
[FR-4790] - If a user with no permissions is created, attempting to log on results in a Stack Trace on the screen
[FR-4792] - Unable to create a Dynamic DSD Attribute with multiple component inputs
[FR-4793] - EDI files are being created with message identity blocks greater than 35 characters and these are not erroring when read back in
[FR-4780] - The Registry can unify monthly data so reading and writing comes out in a consistent fashion
[FR-4771] - Temporary file not cleared away after reading Excel file

Version 10.5.7 (Released 2020/11/13)

New Features:
[FR-4768] - REST - normalise frequency support first in period and last in period functions
[FR-4753] - Registry now has option to clean up temporary directory on startup
[FR-4712] - Format of Code IDs can be restricted for a Codelist
[FR-4689] - Data Validation - support multiple structure references on header to identify which Dataflows or Provisions to use in validation when multiple datasets are in file
[FR-4673] - New parameter on Data Conversion to Merge Datasets
[FR-4666] - Add SSL support to Kafka connection
[FR-4670] - Add a System Property to ignore EDI Agency checks
[FR-4623] - Reporting Template: Hierarchy in row, codes with no data which have child codes with data to be displayed as a presentation node
[FR-4640] - Reporting Template Wrap text if it exceeds 'n' characters
[FR-4617] - Excel Reporting Template: Support Instruction sheets
[FR-4653] - Create property allowing DSI field in EDI file to be resolved to a Dataflow
[FR-4616] - Excel Reporting Template: Display of Codes and Description in the same field
[FR-4619] - Excel Reporting Templates: Protect the workbook so users cannot change the name of the worksheets or add/delete new worksheets
[FR-4597] - Web Service for Modifying and Obtaining System Properties
[FR-4477] - Allow HTTP Read Timeout duration to be viewed and modified via UI
[FR-4478] - Allow HTTP Connect Timeout duration to be viewed and modified via UI
[FR-4460] - View and modify User-Agent used in HTTP Requests from the Registry via the User Interface
[FR-4378] - REST API - Support path parameter: allowedconstraint
[FR-4379] - REST API - Support path parameter: allowedconstraint
[FR-4380] - REST API - Support query parameter: allcompletestubs
[FR-4381] - REST API - Support query parameter: referencecompletestubs
[FR-4394] - Exporting a DSD to EDI - support feature to enforce Primary Measure [1]
[FR-4409] - EDI Lenient mode - allow DSDs to be read and written with uncoded dimensions [2]
[FR-4408] - Add Value list to Bulk Export
[FR-4369] - Support Constraints against Value List
[FR-3302] - DSD Changes: Support zero to many measures - as per SDMX 3.0 working specification
[FR-4276] - Bulk Download offers SDMX-JSON in addition to Fusion-JSON as a format
[FR-4336] - Support Enumeration of items which can have non-SDMX valid IDs (Value List)
[FR-4337] - Reverse Engineer DSD from CSV Dataset
[FR-3305] - Populate an Excel Reporting Template from an SDMX Data File
[FR-2879] - full support for the official SDMX-JSON 1.0 standard
[FR-4114] - AD role mappings CSV import and export
[FR-3304] - Codelist inheritance
[FR-4115] - External table name mapping
[FR-4029] - CORS policy improvements
[FR-3946] - Stronger AES256 encryption
[FR-3561] - Kafka integration – Fusion Registry producer for structural metadata (including Kerberos integration for security)
[FR-4159] - Change the ID and Agency of existing structures
[FR-4072] - Secondary level caching for External SQL datastores, and mapped Dataflows
[FR-4099] - Environment Synchronisation accepts user log in allowing it to work with private or content-restricted registries
[FR-3967] - Active Directory configuration custom search filter
[FR-3951] - New Data Validator: Structure Validator
[FR-3962] - New Data Validator: Obs Status Validation
[FR-3892] - "Missing Time Periods" is its own validator

Improvement:
[FR-4770] - Add support to Fusion CSV to load a CSV file declaring Dimensions and measure dimensions only
[FR-4755] - New Web Service to allow users to 'Close' loaded Data
[FR-4732] - Adding Error Codes to Fusion Registry Validation Response
[FR-4748] - Support Loading of Fusion CSV Data
[FR-4750] - Report errors in EDI attribute validation more clearly
[FR-4730] - Enforce that the EDI FNS block is followed by the correct value
[FR-4688] - Remove 'zombie structures' from registry content, enable users to export or delete via a separate API
[FR-4303] - Kafka Connection Configuration - support multiple hosts
[FR-4660] - Support Quarterly and Half-Yearly formats in SDMX 1.0
[FR-4661] - Upgrade Excel to Structure Map creator to support Fusion Data Mapper v1.1.0
[FR-4652] - Report Templates: Conditional Colour on Attributes - make optional (opt in)
[FR-4632] - Improve Export Structures modal to warn if structures will not be exported in chosen format
[FR-4639] - Enhancement to calculation and interpretation of EDI DSI segment
[FR-4549] - Add 'Reference Partial' option to the export modal on bulk actions
[FR-4615] - Excel Reporting Template: Do not use color on attributes section
[FR-4581] - Add support for Groups in SDMX CSV
[FR-4587] - SDMX JSON Does not support the writing of Groups
[FR-4588] - SDMX JSON Does not support the reading of Groups
[FR-4590] - Issues with Monthly format and SDMX 2.0
[FR-4579] - Enhance the RSS feed to list URLs of added, modified and deleted structures
[FR-4580] - Improved performance in EDI Data Reader
[FR-4511] - Change VND Header for EDI to be consistent with SDMX structure and data syntax
[FR-4512] - Order Excel Report Template Colors in Legend by Codelist Code order
[FR-4214] - Add 'Export SDMX-ML' button to structure pages
[FR-4411] - Improve performance of cross reference resolution
[FR-4363] - Throw HTTP 406 (Not Acceptable) if a structure query explicitly requests the response in a format which is understood but not supported by any structures in the output
[FR-4361] - Simplify install wizard
[FR-4324] - Performance Improvements for Data Convert
[FR-4331] - Show the Hierarchical Codelist Agency/ID/Version of available Hierarchies in Validation Rules Wizard
[FR-4321] - EDI to treat DSI as Dataflow Identifier on Data Read
[FR-4322] - Report Template vnd header - if Data Provider Id is supplied and Agency is not, default Agency to SDMX
[FR-4329] - Remove formula embedded in Report Templates (replaced with checking table)
[FR-4263] - Reporting Template Worksheets to be built from a Dataflow as oppose to a Content Constraint
[FR-4227] - Sdmx JSON meta tag needs to have required node of "sender"
[FR-3882] - Registry should allow Half-Yearly to be specified as code "H"

Removed Feature:
[FR-4081] - Remove: In-memory structures storage

Bug Fixes:
[FR-4759] - Header ID can be incorrectly longer than 10 characters
[FR-4763] - CTA Segment should only be output in EDI if it is valid
[FR-4731] - InputStreams being incorrectly left open
[FR-4733] - Temporary files being created and not cleared away
[FR-4749] - POST requests ending with a slash prevent POST operation from completing successfully
[FR-4727] - Ensure logout after install of Fusion Registry
[FR-4707] - Structure Map - Value Map with Regular expression rejected as expression exceeds max length of Component
[FR-4709] - Unify Rest API VND Headers
[FR-4718] - Issue with format conversion to EDI when an observation level attribute is the only content of an observation
[FR-4701] - Codelist Wizard: re-ordering codes loses non-English languages for those codes
[FR-4702] - Codelist Wizard Step 4, clicking outside of the pop-up window dismisses window and reverts user changes
[FR-4703] - Saving a CSV Locale with 'no value' for the Group Separator value fails
[FR-4704] - StaxReader keeping InputStream open preventing temp file deletion
[FR-4705] - Codelist Wizard Step 4 - Show CSV does not honour languages correctly
[FR-4706] - Concept Scheme Wizard Step 4 - Show CSV does not honour languages correctly
[FR-4698] - Data Provider Constraint linked to Dimension which no longer exists causes registry startup to fail
[FR-4685] - EDI - observations not included in output when a series is written which has a start date earlier then any previous series of the same freequency, AND an end date later then any series of the same frequency
[FR-4674] - SDMX 2.0 data incorrectly stating KeyFamilyURI is an invalid attribute
[FR-4664] - Null pointer when saving a DSD with only 1 measure which does not have Id of OBS_VALUE
[FR-4658] - EDI DSI block should not be permitted to be longer than 18 characters
[FR-4655] - Install page of Fusion Registry - entering an invalid server URL completes the installation
[FR-4656] - Unable to specify space as locale Group Separator on CSV Locale page
[FR-4523] - Updating a DSD with optional Observation Attribute causes Index Out Of Bounds error if data already exists
[FR-4624] - IndexOutOfBounds can be thrown by validating data file
[FR-4626] - SDMXJsonDataReaderEngine is not correctly parsing observations when data file has groups
[FR-4631] - Changing the ID of a structure does not allow it to be obtained by ETag
[FR-4633] - Reporting Template - universe of series miscalculation when including a excluded series constraint with wildcards
[FR-4605] - CSV format (series per column) does not support hebrew characters
[FR-4607] - SDMX JSON - Unknown Attributes at the Series Level should throw an error
[FR-4612] - Half-Yearly issue error in Data handling
[FR-4613] - Issues with Monthly format and SDMX 2.1
[FR-4554] - When editing DSD, unable to change OBS_VALUE to another agency
[FR-4584] - EDI issue - FNS block must be followed by a REL
[FR-4596] - Recursive loop on reindex when 2 provisions linked to the same mapped dataflow
[FR-4598] - Ensure SDMX JSON and SDMX CSV do not support the 1 digit week format
[FR-4599] - Creating Schema can throw Null Pointer Exception
[FR-4524] - Weekly format incorrect for single digit weeks
[FR-4526] - Transforming data files with multiple Datasets should set the header action correctly
[FR-4527] - Converting file should create Group Type of SiblingType
[FR-4535] - Converting file to EDI loses Weekly Data
[FR-4537] - Ensure Generic Data Readers are version independent
[FR-4545] - Support SDMX Dates for Monthly format with the "M" character
[FR-4517] - Empty IDE block following NAD+MS should not be treated as an error
[FR-3281] - Organisations tab - Data Providers not properly sorted
[FR-4495] - JSON data file returning Format Specific error rather than Representation Error
[FR-4496] - Format Specific Errors should not all be at Dataset Level
[FR-4497] - EDI data file returning Format Specific Error rather than Representation error
[FR-4508] - Validation Service should return a 400 when supplied DSD is unknown
[FR-4510] - Role Mappings Organisations should be sorted alphabetically
[FR-4513] - Memory leak when generating Reporting Templates
[FR-4515] - Series Constraint Wizard - step 2 - requires the ability to select multiple Provision Agreements
[FR-4495] - JSON data file returning Format Specific error rather than Representation Error
[FR-4501] - gracefully handle startup failures due to missing internal references
[FR-4474] - Editing a DSD can cause Series and Group information to be lost
[FR-4479] - Hierarchical Codelist fusion-json level incorrectly outputs number, not the level Id that is referenced
[FR-4401] - Validation service: Duplicate attribute values
[FR-4458] - FormatSpecificError expected when reading CSV file
[FR-4461] - JSON: Unknown Attribute at the Observation level should throw Structure Error
[FR-4400] - EDI: Fix to reading of characters which fall outside of the ASCII range
[FR-4459] - EDI: TimePeriodFormat exception should be thrown when analysing attributes
[FR-4462] - EDI: Extra FNS segment causes issues
[FR-4463] - EDI: No IDE segment following the ARR one
[FR-4464] - EDI: Data segment allowed in Attributes section
[FR-4465] - SDMX-CSV data loaded via User Interface as Data Format: CSV with No DSD selected can cause Dataflow lookup error when enclosed in double quotes
[FR-4466] - Unable to load SDMX-CSV via UI when explicit DSD is selected
[FR-4467] - Data files with Weekly frequency can fail to be processed by the Registry
[FR-4471] - CSV - format conversion service is failing due to BOM issue
[FR-4470] - JSON: Format conversion issue - dataset attributes are dropped when converting to JSON data format
[FR-4475] - JSON: Representation error expected due to facet violation but no error thrown
[FR-4480] - SDMX-JSON Hierarchical Codelist Level reference should be by URN - currently it is by Id
[FR-4483] - User unable to create DSD when using "Reverse Engineer DSD from CSV Dataset" due to unclear Error
[FR-4487] - Bug in legacy mode (sdmx-json=fusion-json) for Reference Metadata
[FR-4488] - Bug in legacy mode for availability API
[FR-4482] - REST API references=all is returning descendants of parent structures, should only be children
[FR-4489] - REST API -with Fusion Store: 'offset' parameter used in conjunction with 'max' parameter, is not honoured when 'lastNObservations' is included in query
[FR-4268] - Converting Data - timeFormat = "normalized" - should always return time in ISO 8601
[FR-4292] - HCL Wizard formal Levels, resave causes error in level ID
[FR-4391] - Default value for version not respected in SDMX 2.1 dataset - structure reference is changed to latest version (should be 1.0)
[FR-4392] - IDE segment should only be used to look up DataStructrure (previously defaulted to Dataflow if a match was found)
[FR-4393] - Regression - Data validation for Representation missing from on Group attributes
[FR-4395] - Generic 2.0 files must permit "DataSet" attribute to be in namespace "Generic"
[FR-4399] - EDI: Uncoded Attribute Values must be checked to ensure they are valid EDI characters
[FR-4407] - WS Structure Query - Value List is not on dropdown
[FR-4412] - REST API Delete Method does not support non-SDMX structures
[FR-4420] - Value List - Export to Excel (from VL Page) not working
[FR-4422] - Value List - Annotations - Post add - Page not found
[FR-4423] - View Provision Agreement - Spinner does not disappear (intermittent)
[FR-4441] - referencepartial to treat constraint references as a union, not intersection
[FR-4449] - EDI Delete messages should throw an error if they contain observations or observation attributes
[FR-4453] - Constraint Wizard - edit constraint to add new attachment and remove another results in server side error
[FR-4454] - Illegal Value in EDI contact not reported correctly
[FR-4455] - Supplying an invalid DataStructure in EDI header causes a NullPointerException
[FR-4456] - converting a version 1.0 dataset to a version 2.0 keeps the action as Update (should convert to replace)
[FR-4373] - EDI CTA segment should throw error if Contact Name contains invalid characters
[FR-4376] - Display name and Username in AD and LDAP is wrong
[FR-4386] - SDMX-JSON structures - some keys are the wrong case with respect to the schema
[FR-4397] - SDMX-JSON syntax error on contact names, departments, and roles on Organisations
[FR-4295] - UI - Dataset Details page - View data button should only view the data for the currently selected dataset
[FR-4344] - Validate SDMX-JSON data not reporting error with missing Sender ID
[FR-4349] - Unable to save a DSD with a Primary Measure which uses a Value List for one of the Components
[FR-4350] - Unable to create a DSD when there is no concept with OBS_VALUE in the Registry
[FR-4351] - Primary Measure has incorrect URN with new DSD built through UI Wizard
[FR-4352] - EDI - report error when writing output with unsupported characters
[FR-4354] - EDI file - CTA segment should not restrict contact-id and contact-name to be of IDType
[FR-4358] - Server error when modifying report template from having an Attribute in a worksheet to becoming a fixed attribute
[FR-4359] - Client side issue - unable to modify Report template Attribute from being included in worksheet to having a fixed value
[FR-4362] - Do not output EDI structures with non-1.0 versions, or references to structures with non-1.0 versions, as they are not compatible with the EDI specification
[FR-4365] - structure query, implement references=organisationscheme, currently returns HTTP 400
[FR-4366] - structure query, references=all does not return children of parents
[FR-4315] - EDI issue: Missing REL segment after FNS+Attributes is not detected
[FR-4317] - EDI issue: FTX segments should not be present in EDI deletion messages but no error reported
[FR-4318] - EDI issue: CSV segments should not be present in EDI deletion messages but no error reported
[FR-4270] - EDI time formats are not set correctly when using EDIDataReader
[FR-4224] - Can not convert data to JSON format when no dataflow selected
[FR-4225] - "304 Not Modified" no longer being returned when header includes "If-Modified-Since"
[FR-4226] - Issue with reading SDMX-CSV with very large datasets
[FR-4228] - ByteOrderMark (BOM UTF8) preventing CSV read
[FR-4238] - SDMX JSON should report error when attribute indexes are wrong
[FR-4215] - Enable a dataflow mapping to be saved which is linked to a database, even if there is no data available
[FR-4205] - ws/fusion/info/product web service to include link for current user details
[FR-4187] - When specifying "Structure" in transform call, HTTP 400 is thrown rather than HTTP 200
[FR-4155] - EDI Interchange Header - Application Reference value should accept GESMES/CB and return HTTP 200 if illegal
[FR-4158] - HTTP 200 should be returned when header of EDI file has incorrect elements (ID, preparation date, test value)
[FR-4161] - Validating an EDI file ignores the specified Structure in favour of Structure referenced in EDI file itself
[FR-4058] - Mapping to EDI output is not putting the correct organisation in the sender field
[FR-4071] - Exception generated when mapping file
[FR-4100] - SDMX 1.0 Structure file with Receiver Element with Contact Element cannot be processed by Registry
[FR-4101] - Support SDMX 1.0 'Update' Action
[FR-4057] - xsi:schemaLocation should be specifiable at the Dataset element since this is valid XML
[FR-4035] - Validating Data - requesting Invalid values for EDI file with missing dimension causes 500 error
[FR-4038] - Validation EDI Time Series should report error if range specified but first and last values are not supplied
[FR-4043] - Enforce that TIME_FORMAT is never a mandatory attribute
[FR-4044] - Mapping Dataset Attributes to Series Attributes does not produce correct output
[FR-4028] - Data Validation of EDI series with missing dimensions should report error of type "Structure"
[FR-4033] - Reverting EDI TIME_FORMAT processing
[FR-4000] - Obs Status Validator is not honouring NaN as a missing value
[FR-4002] - Data Validation - EDI files should not report error for missing TIME_FORMAT attribute
[FR-4015] - Reading EDI files does not attempt to resolve Dataflow reference
[FR-4021] - Data Validation need to report EDI missing series attributes more clearly
[FR-3884] - Validation Service - ability to skip Mandatory Attributes data validator should be provided
[FR-3886] - Validation Service - Supplying invalid time definitions should return an error report
[FR-3887] - Validating data in EDI format should behave in the same manner as Structure Specific format
[FR-3889] - Validating data in Generic Data format should behave in the same manner as the Structure Specific format
[FR-3924] - Reported legal characters for ID state that asterisk may be used, when it shouldn't be
[FR-3930] - Dataflow Map - Combination Mapping shows in Mapped Components table in step 3 of the wizard when in edit mode
[FR-3931] - Data Structure and DataFlow Map Wizard - language issue
[FR-3938] - Data Validation - converting valid / invalid series to EDI
[FR-3941] - Data Validation - invalid EDI output missing any form of value
[FR-3964] - Unable to read Excel data file is FREQ is a Dataset Attribute
[FR-3965] - Dataset Attributes are not output when output format is specified as Excel
[FR-3972] - Errors in validating EDI headers should return HTTP 200 rather than 400
[FR-3973] - If EDI Message Trailer Segment (UNT) is incorrect, error is not clear to end user
[FR-3980] - Data Validation of EDI file that has duplicate Series Attributes returns HTTP 400
[FR-3875] - Provision Agreement Wizard does not honour language correctly on step 1
[FR-3925] - Adding a space to an Active Directory Role Mapping definition removes it
[FR-3929] - Fusion Registry Sdmx CSV Format can not be published to Registry if Dataset Attributes are included
[FR-3948] - Metadata Reports not showing Structure labels if language is not available
[FR-3953] - Data Sources Manager 'Table Names' modal has the 'Obs Table' and 'Key Table' header names transposed
[FR-3950] - Metadata Management Interface - Fix multi-page pagination on Manage Metadata
[FR-3949] - Metadata Management Interface - filters broken on manage metadata table for dimensions with numeric codes
[FR-3927] - Oracle: Error shown when starting tomcat