Web-scraping strategy case studies: Approaches to gaining web-scraped data for official statistics – case study of NZ and survey of other countries’ experiences. (Copy)

(from Frances Krsinich, Statistics NZ)

Experiment Proposal                                    Work Package(s): WP2

Name:

Web-scraping strategy case studies:

Approaches to gaining web-scraped data for official statistics – case study of NZ and survey of other countries’ experiences.

Description:

Using the New Zealand experience as a case-study this will be a report on the considerations associated with deciding how to get web-scraped online price data for use in production.

Expected Benefits:

To help other statistical agencies when thinking through the same issues.

Data characteristics:

This will be a report, based on the experience of Statistics New Zealand and surveying the experience of other statistical agencies.

Activities:

Write a case study of SNZ’s experience, and contact other statistical agencies to find out their experience around the same issue (we already have anecdotal information on a range of approaches and changing strategies over time for other offices who have been researching and/or using web-scraped prices data).

Issues and Risks:

We may have limited success in getting information from other statistical agencies.

Outputs

Report including the NZ case-study and survey of other countries experiences.

Lodgement of set of relevant methods (though this may be covered already by the outputs of WP2 ‘Integrated big-data price measurement’).