Background to UNECE's Big Data work
In May 2013 a temporary task team was set up to work out the key issues with using Big Data for official statistics, identify priority actions and formulate a project proposal. The team worked virtually, by Wiki and Webex.
Throughout 2013, UNECE's statistical division coordinated activities designed to explore the current situation in statistical organizations with respect to Big Data:
- identifying the main categories of challenges and opportunities presented by the emergence of Big Data for the official statistics community
- developing a general classification of Big Data sources and uses with respect to the production of official statistics
- determining current and planned uses of Big Data for official statistics and collating them in an inventory
- identifying relevant research, experiments, projects and papers: both in the inventory and in a list of related resources
- developing a proposal for a project under the auspices of the HLG (High-Level Group for the Modernisation of Statistical Production and Services) which could help to address some of the challenges and answer some of the questions identified
- refining and adding detail to certain aspects of this proposal (the 'sandbox' work package)
- and finally presenting it to the HLG for approval in November 2013.
Once approved, the project was launched at the beginning of 2014. The objectives of the project are:
**NEW** March 2016 International collaboration project on The Role of Big Data in the Modernisation of Statistical Production - This project, overseen by the High-Level Group for the Modernisation of Official Statistics, is running during 2014, and aims to:
UNECE work relating to Big Data
Article on the Outcomes of the UNECE Project on Using Big Data for Official Statistics January 2016 November 2015 Competency profiles for Big Data teams and team leaders October 2015 Release of the sandbox prospectus, inviting statistical organisations to participate under the new governance arrangements September 2015 Project workshop in Ljubljana to review progress and decide on priority source(s) to use to produce statistics by November 2015. June 2015 Sprint session in Cork produced an options paper on the future of the Sandbox. This paper and its recommendations were approved by the HLG-MOS Executive Board, which oversees the Big Data project. Throughout 2015 New Big Data project focusing on the Sandbox - New experiments, new data, and a new challenge: "Create a set of internationally comparable statistics based on one or more Big Data sources" April 2015 Launch of the updated . The inventory replaces the initial inventory and includes all project information collected in the joint UNECE-UNSD survey. March 2015 Workshop on Big Data, 9 March, Brussels January 2015 November 2014 The HLG and representatives of expert groups review the progress so far on the 2014 Big Data project, and decide to launch a new project for 2015, with the main aim of producing and publishing multi-national statistics from a Big Data source September 2014 Enhancement of the in collaboration with the United Nations Statistics Division and the HLG Modernisation Committee on Products and Sources: a new, more detailed questionnaire is under development with the aim of obtaining a more comprehensive picture of current and planned projects and facilitating exchange of knowledge and experience. August-October 2014 Preliminary results of the survey "Skills necessary for people working with Big Data in Statistical Organisations". More detailed analysis will be prepared by October 2014. Throughout 2014 June 2013 Draft classification of types of Big Data March 2013 Position paper What does Big data mean for official statistics? (March 2013) drafted for the High Level Group for the Modernization of Official Statistics (HLG-MOS).
International collaboration project on The Role of Big Data in the Modernisation of Statistical Production - This project, overseen by the High-Level Group for the Modernisation of Official Statistics, is running during 2014, and aims to: