istat: use of GSBPM

Since the late ’90s, Istat has developed an information system (SIDI) for documenting the quality of statistical production processes, i.e. processes aimed at producing statistical information such as surveys and statistical compilations. The system documents standard quality indicators and reference metadata supporting the correct interpretation of the quality measures. In 2006 a navigation system, named SIQual, was released on Istat website. The English version was made available in 2009 (SIQual).

The Istat business process model is coherent with GSBPM and can be mapped to GSBPM items at any level (Brancato and Simeoni, 2012).

Briefly, the most important feature of Istat current model for describing the statistical business processes is the definition of a multilayer framework identifying three main strata, strictly connected one each other: i) phases and operations; ii) quality control actions; iii) generalised software. In detail, for each statistical production process the following entities are documented: the activities manipulating data or oriented to their production, namely operations (hierarchically organized into phases); the quality control actions aimed at preventing, monitoring and evaluating errors that affect accuracy; the generalised software used to perform both of them. In addition, some general metadata regarding the process, its content, its periodicities and the reporting units as well as the analysis units are documented. Finally, some topics considered particularly relevant and requiring articulated metadata, are treated with more in-depth documentation (e.g. sampling design and index production methodology).

All the metadata items defined in SIDI/SIQual are characterised by a “validity period”, i.e. the time reference in which that piece of information is indeed valid.

The items to be used for documenting a statistical production process are included in a “thesaurus”, that is a list of standard terms, shared among different processes. These lists are continuously updated by the quality pilots in charge of documenting Istat processes. They can add new items that are then subject to a centralised control and validation procedure, ascertaining the need for their inclusion, position in the hierarchy (if applicable), wording and English translation. In this way, it is ensured on the one hand the standardisation of the documentation and on the other hand the flexibility to adapt the documentation system to changing methods and techniques.

In detail, the phases included in Istat model are:

The phases “Planning”, “Frame development” and “Re-planning” match with the first three phases of GSBPM (“Specify needs”, “Design” and “Build”). Some of the “Build” activities are documented as quality control actions in Istat model. The phases “Data pre-processing”, “Editing and Imputation” and part of “Data processing” in Istat model are included in GSBPM phase 5 “Process”. Istat model phases “Data validation”, “Statistical disclosure” and part of “Data processing” are included in GSBPM phase 6 “Analyse”. Istat model quality control actions are partly included in GSBPM phase 8 “Evaluate”.

For each phase of Istat model, the statistical operations, e.g. how the phase is carried out, are documented in a standard and flexible way. In addition the quality control actions are documented, i.e. how the errors are prevented, monitored or evaluated. Finally the generalized software used to perform the statistical operation is documented. As an example, see picture below taken from the paper by Brancato, Simeoni (2012).

The metadata and the quality indicators of SIDI/SIQual are standardised, homogeneous and consistently managed for all statistical processes, independently from the statistical domain. However some thesaura, i.e. lists of items used to document the statistical operations, are tailored according to the type of process (direct survey vs. statistical compilation) as well as some areas of the system (e.g. data collection phase, sampling design).

The majority of the metadata and quality indicators to be documented in the Istat system are entered in the system by the metadata and quality documentation pilots. However, the SIDI/SIQual system is highly integrated with other Istat systems managing metadata. In particular, the list of the processes to be carried out annually are taken from Statistical Annual Plan (PSN), that is the legislation establishing the information needs and the statistical processes of interest for the National Statistical System.  Metadata on the organizational structure and Istat personnel, useful to attribute responsibilities of the statistical processes are shared with Istat personnel management system. In SIDI/SIQual the processes are related to their outputs (publications, press releases, data warehouses). The  relation with Istat outputs is ensured by the connection with the list of the publications, of any type, released throughout Istat official website. The list of the standard production software, i.e. the software that can be applied to implement methodologies during the process execution is periodically updated with the list managed by Istat structure in charge of it. Process demography is managed in conjunction with Istat structure in charge of storing validated microdata (ARMIDA). Since the new structural metadata component of the wider SUM (Sistema Unitario di Metatati) has been implemented (Signore, Scanu, Brancato, 2015), it creates the lists analysis units used and connected to other reference metadata in  SIDI/SIQual.

With regard to the quality indicators, some of them are automatically produced and loaded in the SIDI/SIQual system, using software interfacing the production phases with the metadata system (e.g. for the data collection phase the coverage and unit non response rates are derived with standard systems for a subset of processes and for the editing and imputation phase the quality indicators can be produced using a standard software, named IDEA). An attempt to obtain indicators on costs from the system managing Istat activities (PTA, Triannual Plan of the Activities).

The SIDI/SIQual metadata system supports some external  internal processes and respond to some requirements of the external users.

It supports the dissemination process, by providing to the users accessing the Istat corporate data warehouse (I.Stat) with a standard documentation regarding the processes generation the data and their quality. The system contributes also to increase transparency, throughout the publication of a page of short quality reports, named “Quality at a glance”  that contains a selection of the metadata and of the quality indicators drawn from the SIDI/SIQual system and properly arranged. Similarly, a methodological note documenting all the relevant aspect of the process is drawn from the SIDI/SIQual and published in the Statistical Yearbook. The available metadata are being integrated with additional ones and a system to draw, arrange and transmit the required information to Eurostat (ESMS and ESQRS) is being developed.

The system also supports internal assessment and quality planning processes. Metadata and quality indicators are the information material used as basis for the assessment in the statistical auditing program carried out at Istat. The quality indicators are annually analysed for monitoring and setting quality strategies.