View source for FMR 11.0 release notes

FMR 11.0 implements the SDMX 3.0 information model and differs fundamentally from FMR 10 in that respect which uses the SDMX 2.1 information model. 

=Backward compatibility=
Backward compatibility with SDMX 2.1 and FMR 10 has been preserved where possible:
* All SDMX 2.1 stuctures can be loaded into a FMR 11 registry, although some such as Structure Maps and Hierarchical Codelists are converted irreversably to the equivalent SDMX 3.0 model structures.
* SDMX 2.1, 2.0 and EDI structure and data formats continue to be supported where possible. Structures such as Representation Maps which only exist in the SDMX 3.0 information model cannot be exported in legacy structure formats. Similarly, data that uses some of the features introduced in SDMX 3.0 for handling microdata cannot be converted to an SDMX 2.1 or older data format.
* The SDMX 2.1 structure REST API syntax and entry point is supported meaning that most systems and processes designed for use with FMR 10 will continue to work with FMR 11.

=Breaking change - FMR 11 incompatible with FMR 10 databases=
FMR 11.0 is not compatible with FMR 10 operating databases. Upgrading from FMR 11 therefore requires the software to be installed on a new database schema and existing structural metadata migrated by hand. [[Upgrading to Version 11]] explains how.

=Environment Synchronisation temporarily removed=
FMR 11.0 does not provide an[[Synchronise structural metadata between FMR environments|Environment Synchronisation]] tool for moving structures between two Registry installations. 

=Features and changes=
{| class="wikitable"
|-
|
Reference

|


|
Summary

|
Description

|-
|
FMR11-1

|
New Feature

|
SDMX 3.0 internal information model

|
The SDMX 3.0 information model is used internally.

|-
|
FMR11-2

|
New Feature

|
Storage and retrieval of all SDMX 3.0 structures with the exception of Metadata Constraints, Concept Scheme Map, Category Scheme Map

|
Support for input, storage and retrieval of all SDMX 3.0 structures with the exception of Metadata Constraints, Concept Scheme Map, Category Scheme Map

|-
|
FMR11-3

|
New Feature

|
SDMX 3.0 structure query REST API

|
Structure query REST API to the SDMX 3.0 specification.

|-
|
FMR11-4

|
New Feature

|
SDMX 3.0 schema query REST API

|
Schema query REST API to the SDMX 3.0 specification.

|-
|
FMR11-5

|
New Feature

|
SDMX 2.1 compatibility structure query REST API

|
Structure query REST API to the SDMX 2.1 specification for backward compatibility purposes. 



The following  are not supported:



* Querying for SDMX 2.1 structure classes that do not exist in the SDMX 3.0 model, e.g. structure sets

* Querying for SDMX 3.0 structure classes that are not supported by the SDMX 2.1 structure query REST API specification e.g. representation maps

|-
|
FMR11-6

|
New Feature

|
SDMX-ML (XML) 3.0 structure format

|
Structures can be read and written in SDMX 3.0 SDMX-ML.

|-
|
FMR11-7

|
New Feature

|
Read and write structures in supported legacy SDMX 2.1, 2.0 and 1.0 structure formats

|
Structures can be read and written in supported legacy SDMX 2.1, 2.0 and 1.0 formats where the models align sufficiently e.g. simple Codelists, Concept Schemes, Dataflows.

The following SDMX 3.0 structures cannot be written in legacy structure formats:


* SDMX 3.0 structures that have no equivalent in SDMX 2.1 or earlier, e.g. Value Lists, Metadata Provider Schemes
* SDMX 3.0 structures using features that are not possible to describe in SDMX 2.1 or earlier, e.g. DSDs with multiple measures

SDMX 3.0 extended Codelists are materialised when written in legacy formats

SDMX 3.0 geospatial Codelists loose their geofeature and geogrid information. 

Legacy structures that do not exist in the SDMX 3.0 information model are converted on read where possible, including:

* Structure Sets are converted to the SDMX 3.0 mapping model using Structure Maps and Representation Maps
* Hierarchical Codelists are converted to the SDMX 3.0 equivalent Hierarchy artefacts

|-
|
FMR11-8

|
New Feature

|
Fusion-JSON 3.0 structure format

|
Fusion-JSON is Metadata Technology's proprietary JSON format for serialising SDMX 3.0 object collections. It is critical for driving the FMR user interfaces, and can also be used for transmission of structural metadata between FMR environments. It differs from SDMX standard formats in that it is able to represent all of the possible structures, including any structures that are not part of the official SDMX standard. 

|-
|
FMR11-9

|
New Feature

|
SDMX-CSV 3.0 data format (partial)

|
Read and write data in SDMX 3.0 CSV for the purposes of validation, mapping and conversion to other formats.



The majority of the SDMX 3.0 CSV specification is supported with the exception of some advanced features including:



* multi-lingual attributes
* multi-lingual measures
* hierarchical reference metadata

|-
|
FMR11-10

|
New Feature

|
SDMX 3.0 data validation

|
Data validation respecting the new SDMX 3.0 model features, principally:

* multiple measures, array attributes / measures, 
* the enhanced constraints model - % operator, validity of constraining terms, discriminated union
* extended codelists 
* value lists
* the new component representation types (e.g. geospatialinformation)


Validation of both traditional aggregated time series and SDMX 3.0 micro-datasets is supported.

|-
|
FMR11-11

|
New Feature

|
SDMX 3.0 data mapping

|
Map loaded datasets using the SDMX 3.0 mapping model including:



* Mapping of micro-datasets
* Absolute time mapping

Excluding:

* Relative time mapping

The mapped resultset can be written out in any of the supported SDMX 3.0 data formats, or a supported legacy data format provided that the selected format is capable representing it.

|-
|
FMR11-12

|
New Feature

|
Automatic conversion of SDMX 2.1 structure sets to the SDMX 3.0 mapping model on load

|
SDMX 2.1 Structure Sets are automatically converted to the SDMX 3.0 mapping model (Structure Maps and Representation Maps) on load. 



The conversion is one-way meaning that it is not possible to retrieve the original Structure Set.

|-
|
FMR11-13

|
New Feature

|
Read and write data in supported legacy SDMX 2.1 and 2.0 data formats (legacy data conversion)

|
Datasets can be:

* Read in a SDMX 3.0 format and converted to a supported legacy format provided that it is capable of representing the data.
* Read in a supported legacy format and converted to a SDMX 3.0 format.
* Read in a supported legacy format, mapped to a different structure and written out as a supported SDMX 3.0 data format, or a supported legacy format provided that it is capable of representing the data.

SDMX 3.0 aggregated time-series datasets can usually be represented in legacy formats, provided they do not use multi-value attributes or measures which are features specific to SDMX 3.0.

SDMX 3.0 micro-datasets cannot be represented in legacy data formats.

Datasets loaded in legacy formats can generally be converted to SDMX 3.0 data formats without issue.

|-
|
FMR11-14

|
New Feature

|
Automatic conversion of SDMX legacy Hierarchical Codelists to the SDMX 3.0 Hierarchy model on load

|
SDMX 2.1 Hierarchical Codelists are automatically converted to the SDMX 3.0 Hierarchy model on load.



The conversion is two-way meaning that it is possible to retrieve the original Hierarchical Codelist once the load is complete. However, the conversion is lossy such that HCL's are ungrouped on the conversion to Hierarchies.

|-
|
FMR11-15

|
New Feature

|
EDI legacy structure format

|
Read and write structures in pre-SDMX 3.0 EDI format where possible.

|-
|
FMR11-16

|
New Feature

|
SDMX-ML (XML) 2.1, 2.0 and 1.0 legacy structure formats

|
Read and write structural metadata in pre-SDMX 3.0 XML structure formats where possible.

|-
|
FMR11-18

|
New Feature

|
SDMX-ML (XML) 2.1 and 2.0 legacy data formats

|
Read and write data in the following legacy SDMX-ML formats where possible:

* SDMX 2.1 Structure Specific
* SDMX 2.1 Generic
* SDMX 2.0 Compact
* SDMX 2.0 Generic

FMR11's data conversion rules define when datasets can be read and written in legacy formats.

|-
|
FMR11-21

|
New Feature

|
SDMX-JSON 2.1 legacy structure format

|
Read and write structural metadata in pre-SDMX 3.0 JSON structure formats where possible.

|-
|
FMR11-22

|
New Feature

|
Read and write structures in supported SDMX 3.0 structure formats

|
Structures can be read and written in supported SDMX 3.0 structure formats.

|-
|
FMR11-23

|
New Feature

|
Read and write data in supported SDMX 3.0 data formats

|
Read and write data in supported SDMX 3.0 data formats

|-
|
FMR11-28

|
New Feature

|
UI - support annotations on Representation Maps

|
<nowiki>GitHub issue cross reference: [https://github.com/MEDIT-Org/fmr/issues/48|https://github.com/MEDIT-Org/fmr/issues/48|smart-link] </nowiki>

|-
|
FMR11-50

|
New Feature

|
Series constraint expression using the + operator
|
Use the '+' operator to specify multiple values for series constraint values to simplify maintenance and reduce the verbosity.

Example SDMX-ML
 <str:Key>
   <com:KeyValue id="REF_AREA">
      <com:Value>DE+FR+GB</com:Value>
   </com:KeyValue>
   <com:KeyValue id="INDICATOR">
      <com:Value>ABC_123</com:Value>
   </com:KeyValue>
 </str:Key>

The above evaluates to the following series keys:
 DE:ABC_123
 FR:ABC_123
 GB:ABC_123

|-
|
FMR11-51

|
New Feature

|
Semantic versioning - three-digit version numbers (e.g. 1.3.2)

|
Structures can be versioned using the three-digit semantic versioning syntax.

|-
|
FMR11-52

|
New Feature

|
Semantic versioning - REST API structure query using semantic versioning syntax

|
REST API structure queries support the semantic versioning selection syntax using the '+' operator.

e.g.

+ latest stable version

1.3+ any 1 version greater than 1.3 (e.g. 1.5, but not 1.1)

1.0.0,2.1.7 either version 1.0.0 or 2.1.7

|-
|
FMR11-62

|
New Feature

|
JNDI data sources - Apache Tomcat and RedHat JBoss support

|
JNDI data source has been added to the list of FMR database connection options in addition to Oracle, MySQL and SQL Server. Apache Tomcat and RedHat JBoss are supported. Support for Oracle Weblogic is planned for a future release.

|-
|
FMR11-60

|
Change

|
Change to the structure of the operating database

|
The structure of the operating database has changed in FMR 11 making it incompatible with version 10. An existing FMR 10 installation cannot be directly upgraded to FMR 11, and an FMR 11 database cannot be used with version 10 of the software.

To upgrade from 10 to 11:

* Create a new FMR 11 installation using a separate database schema
* Export the structural metadata from FMR 10 to file using the Fusion-JSON format
* Import the structural metadata into FMR11 which will perform an automatic conversion to the SDMX 3.0 model

|-
|
FMR11-61

|
Change

|
Environment synchronisation function temporarily removed

|
The Environment Synchronisation function has been temporarily removed in FMR 11.0 but will be replaced in a future 11 released with a new design.

|}