Issue #15: Referential Metadata Content Item & Referential Metadata Subject Item

Meeting 6 June

Examples would be helpful to clarify the difference, but do we really need both? the way it was modeled seems over-complicated. Maybe we can remove Referential Metadata Content Item?

Action: 1) Need use cases from countries who implemented (experiences? - Eva Holm, Catrin Karling, Alistair Hamilton); 2) Check with SDMX people for initial rationale 

Excerpt from GSIM:

108. The Referential Metadata Content Item is the actual metadata for the identified Referential Metadata Subject Item. Each Referential Metadata Content Item contains the reported referential metadata for one Referential Metadata Attribute specified in the Referential Metadata Structure. 

Table 1. Example of Use of GSIM Referential Metadata Objects

ONS Statistical bulletin: Public Sector Finances, October 2013:Table 1

Table 1 ‘Key Measures of the Public Sector Finances’ presents the latest headline figures for the Public Sector Finances. The table compares the figures for the latest month with the same month a year ago and cumulative figures for the financial year to date compared with the same period in the last financial year. A time series presentation of these fiscal measures can be found in table PSF1 of this bulletin.

Table 1: Key Measures of the Public Sector (PS) Finances by month and financial year-to-date

Excluding the temporary effects of financial interventions

Table source: Office for National Statistics

Table notes:

1. Unless otherwise stated
2. Current Budget is the difference between current receipts and current expenditure
3. Net Investment is investment less depreciation
4. Net Borrowing is Current Budget less Net Investment
5. RM = Royal Mail Pension Plan transfer
6. APF = Bank of England Asset Purchase Facility Fund transfers
7. Net Debt is financial liabilities less liquid assets
8. GDP = Gross Domestic Product

ABS has steered clear of practical implementation of Referential Metadata to date (but it is still in our AIM model).  This is because of the complexity of the model.

The most fundamental complexity for us is that the Referential Metadata (eg an annotation/footnote) tells you what it attaches to (via Referential Subject Item) rather than the subject "knowing" it has Referential Metadata associated with it.

Taking the ONS example, if PS Net Borrowing (PSNB ex) ex RM and APF is a Code Item I've decided to include in an output structure, the Code Item doesn't know it has Referential Metadata associated with it.  We need to trawl through a potential "cloud" of Referential Metadata to see if any of it has this Code Item as its Referential Metadata Subject Item.

There are various things we can do in lower level implementation design (eg at a database and footnote management system design level rather than in GSIM) to take the association and reverse it so a Code Item "knows" which Referential Metadata Content Items are associated with it at the time of dissemination.

A further complexity in end to end management is that we might start with an annotation on one Code Item ("XYZ") in Code List A but then due to restructuring of other Code Items, we end up with Code List B.  Conceptually Code Item "XYZ" hasn't changed but each Code Item "belongs" (composition relationship) to its Code List so "XYZ" has a new identify in Code List B.  We somehow need to ensure that any Referential Metadata Content Items that related to "XYZ URN1" as their Referential Metadata Subject Item now also have as a subject item "XYZ URN2". 

(I realise this goes away if the Referential Metadata attaches to the Category rather than the Code Item but ABS business folk found it hard to manage Categories independently of Code Items.)

However, if Referential Metadata remains modelled as is currently the case then absolutely both Subject Item and Content Item are required.

As a further example of complexity, the ONS example would need a lot of "fancy footwork" if all the footnotes apply to "Data Structure Components". 

Firstly, I suspect that the rows are Code Items within a single Identifier Component by this stage (given they are having a heap of different measures reported for them across the columns) in which case they would need a Referential Metadata Subject of Code Item.

Even if that is not the case, however, GSIM has a composition relationship between Data Structure Components and Data Structures.  This means if I reuse the same piece of information (eg PS Net Debt (PSND ex)) in multiple outputs which have multiple structures (depending on what else I want to include in the particular output) it is a new Data Structure Component each time. 

I probably, however, want to explain the concept of "Net Debt" (footnote 7 in the above example) each time I use it.  So once again, probably the Referential Metadata Subject Item was something different initially, and was in a Referential Metadata Set with a Metadata Referential Structure that had a Referential Metadata Subject of something like "Represented Variable" rather than "Data Structure Component".

Either that or every time I define a Data Structure for output I manually "sprinkle" the right footnotes over its Components?

We are implementing this part of GSIM informally (no proper mapping between the GSIM classes and the SDMX ones) using an RDF flavour of the SDMX Information Model (SDMX-IM) which inspired this part of GSIM I guess.

My understanding is that the Referential Metadata Content Item and the Referential Metadata Subject Item correspond to the SDMX the ReportAttribute and targetObjectKey classes. However, SDMX-IM has an additional class, the MetadataReport which associates the Reported Attribute values which are to be associated with the object or objects identified by the Target Object Key. So I agree with Alistair, both the Referential Metadata Content Item and the Referential Metadata Subject Item should be kept.

Our use case is quite different from the ONS example, since it deals with quality declarations/statements documented with the European Single Integrated Metadata Structure (SIMS).

A MetadataStructureDefinition structures a MetadataFlowDefinition and contains one or more MetadataTarget composed of TargetObject. So basically it is a heap of metadata atttributes gathered in a set that targets a flow. It is then quite far from the ONS example which tries to apply footnotes to the different components of the Data Structure which requires a lot of sleights of hand as Alistair says. The problem in this example is that we are mixing the DataStructure and the MetadataStructure areas without being as complete as the SDMX-IM on the MetadataStructure/Set part.

I would change the example based on something more streamlined and maybe simpler as the SIMS: we stick to reference metadata and attach a bunch of metadata attributes to a metadata flow even though that latter class does not exist per se in GSIM and is somewhat split between the GSIM ReferentialMetadataStructure and ProvisionAgreement.

Here is a very rough example. I took a SIMS web page on the Eurostat website and added the GSIM classes. I am not sure that the layout fits in the GSIM documentation .

Note that there are here ReferentialMetadataSubjectItem: the dataflow, the data provider and the time dimension. Their combination constitutes the information required to identify the object for which the Referential Metadata Content Items are reported.

In SDMX terms,an additional class, the identified MetadataTarget specifies the expected content of the TargetObjectKey i.e. it specifies the information required to identify the object for which the ReportedAttributes are reported.