Contact person* | |
|---|---|
| Job title | |
Email | |
| Telephone |
Statistical business process model
Statistics New Zealand's business process model is illustrated here.
Extensive consultation was undertaken across the organisation to develop the generic Business Process Model (gBPM), including many business process modelling sessions. Having undertaken the consultative processes it was agreed that the gBPM will fit the direction that Statistics NZ was taking. We are working on some minor variations of the gBPM to better fit Data Integration and Feasibility Studies.
Also refer to this diagram illustrating data flow through the statistical production cycle .
Mapping to the CMF generic survey life cycle model:
Common Metadata Framework | Statistics NZ business process model |
|---|---|
1. Need | Need |
2. Develop and design | Develop and design |
3. Build | Build |
4. Collect | Collect |
5. Process | Process |
6. Analyse | Analyse |
7. Disseminate | Dissseminate |
8. Archive | within the sub-processes so that archiving is considered at all relevant stages |
9. Evaluate | not explicit in Statistics NZ model |
Metadata used/created at each phase
Phase: Need
Description:
- Need is an ongoing process to determine the statistical needs of Statistics New Zealand's stakeholders
Description of main developed functionalities:
- Online Consultation/Submission Tool (Census)?
- Documentation Storage: Lotus Notes
Metadata used (inputs): table (below) with groups of metadata (rather than individual metadata items), examples for each group
Group | Description | Example | Source (if re-used) |
|---|---|---|---|
Conceptual Metadata | Concepts of interest. Previously available information. | Global variables, themes, subject areas, statistical objects types | Disseminate (previous cycle/collection) |
Operational Metadata | Analyse processes of previous collections/ available data. | Study, Study Method, Statistical Process | Process (previous cycle/collection) |
Quality Metadata | Quality of previous collections, or available data | Data Quality | Process and Analyse (previous cycle/collection) |
Physical Metadata | Locate available data through physical metadata | Datasets, server locations, software, access rights | Disseminate (previous cycle/collection) |
Metadata produced (outputs): table (below) with groups of metadata (rather than individual metadata items), examples for each group
Group | Description | Example | Source (if re-used) |
|---|---|---|---|
Conceptual Metadata | Defined concepts high level | Global variables, themes, subject areas, statistical objects types |
|
Operational Metadata | High level strategy for meeting need. | Business Case, Study Method, Statistical Process |
|
Quality Metadata | Consultation process | Reports |
|
Physical Metadata | Document storage, submission storage | Locations, References |
|
Phase: Develop and Design
Description:
- Develop and Design describes the research, development and design activities to define the statistical outputs, methodologies, collection instruments, sample, operational processes and end-to-end (E2E) solution
Description of main developed functionalities:
- Documentation Storage: Lotus Notes
Metadata used (inputs): table (below) with groups of metadata (rather than individual metadata items), examples for each group
Group | Description | Example | Source (if re-used) |
|---|---|---|---|
Conceptual Metadata | Need to refine the concepts to determine the variables to be collected and the population of interest. | Includes concepts such as object variables, value domains, classifications, measure units, context data elements. | Need (High level concepts) |
Operational Metadata | This includes designing the sample methodology, collection methodology and the statistical processes. | Includes concepts such as statistical process, process implementation, data collection methodology. | Need (strategy) |
Quality Metadata | Quality metadata from previous collections may be used in the design of sample and collection methodologies. | statistical process, process implementation, data collection methodology. | Collect/Process (previous collections) |
Physical Metadata | May begin to define/ identify where data will be stored. | software, data sets, access package, |
|
Metadata produced (outputs): table (below) with groups of metadata (rather than individual metadata items), examples for each group
Group | Description | Example | Source (if re-used) | |
|---|---|---|---|---|
Conceptual Metadata | Defined concepts | Includes concepts such as object variables, value domains, classifications, measure units, context data elements. |
| |
Operational Metadata | Completed end to end design and methodology | statistical process, process implementation, data collection methodology, business rules, transformations. |
| |
Quality Metadata | Details of the design process used. Quality introduced by proposed standards. | Design process. Quality measures of standards. |
| |
Physical Metadata | Storage plan. Document storage | Software, Access Package, Data Set |
|
|
Phase: Build
Description:
- Build produces the components needed for the end-to-end solution, and tests that the solution works
Description of main developed functionalities:
- Dashboard - For configuring and monitoring processes
- Workflow tool - for developing statistical processes
- Questionnaire Design, CAI tools, Scanning Software - For survey collections
- CRM, Q-Master (call centre) - For running collections
- Transformation Tools - imputation, editing, coding, etc.
- Data Environments - for storage of data (in development)
- Metadata Environment Components - for storage of metadata (developing high level design)
Metadata used (inputs): table (below) with groups of metadata (rather than individual metadata items), examples for each group
Group | Description | Example | Source (if reused) |
|---|---|---|---|
Conceptual Metadata | May need to refine/adapt concepts due to feedback from testing or errors detected. | Data Elements, Classifications, Value Domains | Develop and Design |
Operational Metadata | Build and configure storage structures, collection instruments, processing requirements. Includes concepts such as record variables, record types, matrix operations. | Question, questionnaire, data collection methodology, statistical process. | Develop and Design |
Quality Metadata | The final version of a questionnaire will be decided during this stage. | Methodology and process design. Testing plans. | Develop and Design |
Physical Metadata | Determine the physical location where the data will be stored | servers, software, access packages. | Develop and Design |
Metadata produced (outputs): table (below) with groups of metadata (rather than individual metadata items), examples for each group
Group | Description | Example | Source (if reused) | |
|---|---|---|---|---|
Conceptual Metadata | Finalised concepts | Data Elements, Classifications, Value Domains, Statistical Unit |
| |
Operational Metadata | End to End components built and tested | Built Components, Processes. Statistical process, data collection methodology. |
| |
Quality Metadata | Quality metrics about the build process and testing report | Test reports, validated processes. |
| |
Physical Metadata | Complete Application Architecture | Software, Storage, Access Packages, Access Rights, Versions etc. |
|
|
Phase: Collect
Description:
- Collect acquires collection data each collection cycle and manages the providers of that data
Description of main developed functionalities:
- Questionnaire Design, CAI tools, Scanning Software - For survey collections
- CRM, Q-Master (call centre) - For running collections
- Data Environments - for storage of data (in development)
- Metadata Environment Components - for storage of metadata (developing high level design)
Metadata used (inputs): table (below) with groups of metadata (rather than individual metadata items), examples for each group
Group | Description | Example | Source (if reused) |
|---|---|---|---|
Conceptual Metadata | As data is collected it will be allocated against concepts. Some details about the relevant concepts may be used in respondent management strategies. | Data Elements, Classifications, Value Domains, Statistical Unit | Develop and Design |
Operational Metadata | Utilise the collection processes outlined in the operational metadata. | Data Collection Methodology, Questionnaire, Collection Strategy. | Devlop and Design, Build |
Quality Metadata | Collect data for each instance of the survey. Quality metadata will be populated based on the collection instance. | Collection strategy, Data Collection Methodology, | Devlop and Design, Build |
Physical Metadata | Physical datasets will be populated with data at this stage. | Software, Storage, Access Packages, Access Rights, Versions etc. | Build |
Metadata produced (outputs): table (below) with groups of metadata (rather than individual metadata items), examples for each group
Group | Description | Example | Source (if reused) | |
|---|---|---|---|---|
Quality Metadata | Operational Processes used. Quality measures of collection instance. | Collection Report, Data Quality - response rate, item non response. |
| |
Physical Metadata | Data collected to measure collection concepts. | Software, Storage, Access Packages, Access Rights, Versions etc. |
|
|
Phase: Process
Description:
- Process describes cleaning the detailed data records and preparing them for analysisFor each phase provide:
Description of main developed functionalities:
- Dashboard - For configuring and monitoring processes
- Workflow tool - for developing statistical processes
- Transformation Tools - imputation, editing, coding, etc.
- Data Environments - for storage of data (in development)
- Metadata Environment Components - for storage of metadata (developing high level design)
Metadata used (inputs): table (below) with groups of metadata (rather than individual metadata items), examples for each group
Group | Description | Example | Source (if reused) |
|---|---|---|---|
Conceptual Metadata | Conceptual metadata are used to classify and code open responses. Derive new concepts, aggregate data etc. | Classification, Correspondence, Data Elements, Classifications, Value Domains, Statistical Unit | Develop and Design |
Operational Metadata | Further statistical processes are used in order to process the data. This may include the creation of cubes and registers, aggregating results using derivation rules, applying editing and imputation strategies, applying confidentiality rules, etc. | Matrix, Cube, Register, Statistical Process, Process Implementation, Operation Implementation, Derivation Rules, Computation Implementation. | Develop and Design, Build, Collect |
Metadata produced (outputs): table (below) with groups of metadata (rather than individual metadata items), examples for each group
Group | Description | Example | Source (if reused) | |
|---|---|---|---|---|
Quality Metadata | Further quality metadata will be populated at this stage based on the processes applied. | Processing reports, Data Quality - imputation rates, editing rates etc. | Develop and Design, Collect | |
Physical Metadata | If processed data is stored in different locations, new physical metadata will be defined. | Storage, Software, Access Package etc. | Build |
|
Phase: Analyse
Description:
- Analyse is where the statistics are produced, examined in detail, interpreted, understood and readied for dissemination
Description of main developed functionalities:
- Analytical Environment (strategy still in development)
- Information Portal (strategy still in development)
- Data Environments - for storage of data (in development)
- Metadata Environment Components - for storage of metadata (developing high level design)
Metadata used (inputs): table (below) with groups of metadata (rather than individual metadata items), examples for each group
Group | Description | Example | Source (if reused) |
|---|---|---|---|
Conceptual Metadata | In the analysis stage the processed data will be used to analyse the concepts defined in the need and develop/design stages. | Data Elements, Classifications, Value Domains, Statistical Unit | Need, Develop and Design |
Operational Metadata | Further processes may be used to generate tables for analysis. | Tables, Statistical Processes, Confidentiality Rules | Develop and design |
Quality Metadata | Quality metadata will be used at this stage to assess how well the data represents the concepts outlined in the needs stage. | Statistical Activity, Study, Statistical Process. | Collect, Process |
Physical Metadata | Physical metadata will be required to locate the data, and any additional data for comparisons. | Storage, Software, Access Package etc. | Build, Collect, Process |
Metadata produced (outputs): table (below) with groups of metadata (rather than individual metadata items), examples for each group
Group | Description | Example | Source (if reused) | |
|---|---|---|---|---|
Quality Metadata | Analysis of quality metadata against concept defined in need stage. | Data Elements, Classifications, Value Domains, Statistical Unit, Statistical Activity, Study, Statistical Process. |
| |
Physical Metadata | Produce analysis reports, output products. | Data Set, Publications |
|
|
Phase: Disseminate
Description:
- Disseminate manages the release of the statistical products to the customers.
Description of main developed functionalities:
- Integrated Publishing Environment - Tool for configuring and disseminating analysis.
- CRM/ Job Tracking Systems - for recording customer usage.
- Data Environments - for storage of data (in development)
- Metadata Environment Components - for storage of metadata (developing high level design)
Metadata used (inputs): table (below) with groups of metadata (rather than individual metadata items), examples for each group
Group | Description | Example | Source (if reused) |
|---|---|---|---|
Quality Metadata | Technical audiences and professional audiences may be interested in the quality metadata in order to understand the characteristics of each instance of data collection. | Analysis Reports, Output Releases, Quality measures/ reports, Needs definition | Need, Analysis |
Physical Metadata | Physical metadata may be required when locating data in response to customer queries. | Server locations, access rights, access packages, systems and tools, online catalogues | Build, Analyse |
Metadata produced (outputs): table (below) with groups of metadata (rather than individual metadata items), examples for each group
|
|
| Source (if reused) |
|---|---|---|---|
Conceptual Metadata | Identification of new needs for collection. | Global variables, themes, subject areas, statistical objects types |
|
Operational Metadata | Development of new/changed standards for other collections | Study Methods, Statistical Processes |
|
Quality Metadata | Details of products produced, queries received and products used. | Dissemination process, Usage data |
|
Physical Metadata | Output products available for promotion. | Tables, Reports, Brochures etc. |
|
Metadata relevant to other business processes
Group | Description | Example |
|---|---|---|
Conceptual Metadata | Information/ records defining organisational concepts, structures etc | Corporate directories, Corporate glossary, Organisational Structure charts |
Operational Metadata | Information/ records defining how corporate processes are applied | Corporate policies, Guides, Contracts, Memorandums of understanding (MOU) |
Quality Metadata | Information/ records related to specific instances | Invoices, Annual Reports, Budgets |
Physical Metadata | Information relating to phyisical item | Catalogues, Asset Management Lists etc. |
Lessons learned
| Links: |
|---|