1. Increasingly, new data sources are becoming available to statistical organisations. This comes at a time when modern technologies are available to support data integration. Data integration provides the potential to produce timelier, more disaggregated statistics at higher frequencies than traditional approaches alone.
2. It can be used to provide new official statistics, address new or unmet data needs, lower response burden, overcome the effects of reducing response rates, and address quality and bias issues in surveys.
3. Statistical organisations are challenged to integrate diverse sets of inconsistent data and to produce stable outputs with sometimes unstable, ever-changing inputs. Instead of trying to produce the best possible statistics from a single survey, it is necessary to try to find the best combination of sources to deliver the indicator/statistics that best satisfy the users' needs.
4. There are some potential challenges related to data integration including:
- New skills, new methods and new information technology approaches
- Designing new concepts or aligning existing statistical concepts to the concepts in new data sources
- Measuring, managing and publishing the quality of both the data sources and the statistics produced
- Governance for data integration projects
- Managing public perception and communication
- Avoiding duplication of effort across countries and organisations and using the collective experience of the official statistics community.
5. A survey conducted in 2017 by the Data Integration Project found that public acceptance and trust issues caused more significant/moderate barriers to data integration compared to issues such as methodologies, skills and budget.
6. This guide provides information on planning for data integration activities, issues related to data and the data integration methods and tools.