How might the tension between NSO and researcher perspectives be resolved?
71. This will most effectively be done by NSOs moving from a risk avoidance strategy to a risk management strategy. How to do this is discussed in more detail in the following paragraphs.
72. There are definitely risks that have to be managed. The rapid expansion of databases, containing data about identifiable persons, means that it is virtually impossible to completely avoid identification through data matching of a significant number of persons even though names and addresses are removed, particularly if household structure is contained in the files. Many of these databases are held by the private sector where controls on their use are generally less stringent than for the public sector. Furthermore, technology advances have made data matching easier, whether by exact matching or statistical matching techniques (which can lead to exact matches in unique cases). Risk avoidance in essence means not allowing identifiable microdata to leave the premises of the NSO unless other steps such as data swapping or data perturbation are undertaken. (Note that risks will vary according to the size of country among other things. In smaller countries, the risk will be relatively higher because there are relatively more unique cases.)
73. Nevertheless, the microdata access provided by NSOs does not seem to have been an area of public controversy. Implicitly, there seems to be a reasonably high level of public acceptance of current practices although we are not aware of countries where there has been an extensive public debate. But general community concerns about privacy suggest there is a limit to what the public is likely to accept. A debate could be easily triggered (across national boundaries) by one unfortunate incident. The level of cooperation in statistical collections could be subsequently affected.
74. Transparency is important in order to avoid accusations of secrecy. Therefore, it is good practice for NSOs to be transparent in outlining that one of the valued uses of the data from some statistical collections will be to provide researcher access to confidentialised microdata under controlled conditions for specific purposes. This has to be managed carefully or the privacy advocates could sway public opinion. Support from respected and authoritative persons is very important.
How do NSOs manage the risks of microdata access?
75. Some suggestions are outlined below:
- (i) agree on a set of principles that should be followed in the provision of access to microdata (such as those outlined in Chapter 3);
- (ii) ensure there is a sound legal and ethical base (as well as the technical and methodological tools) for protecting confidentiality. This legal and ethical base requires a balanced assessment between the public good of confidentiality protection on the one hand, and the public benefits of research on the other. A decision on whether or not to provide access might depend on the merits of specific research proposals and the credibility of the researcher, and there should be some allowance for this in the legal arrangements. Access should not be regarded as automatic;
- (iii) have an arms-length process for balancing these two public goods. It is good practice to set up an internal committee to debate these matters and make recommendations to the head of the NSO. Judgements are involved and ethics committees or similar bodies may be able to assist in situations where there is discretion in deciding whether to provide access or not. The public good arguments are much stronger if the results of the research are to be placed in the public domain;
- (iv) be completely transparent about specific uses of microdata to avoid suspicions of misuse;
- (v) be prepared to provide more access through remote access facilities and data laboratories if completely unidentifiable microdata for public release may not be possible without considerable distortion of the data. Explore other opportunities to use technological developments to improve access to microdata in such a way that adequate confidentiality protection is provided;
- (vi) pass some of the onus of responsibility to the research community. Ensure researchers understand the reasons NSOs are so protective of confidentiality. Ensure researchers are aware of the consequences to them and their institution if there are breaches. Follow up with appropriate retribution if there are breaches.
76. The last point requires some comment. The culture and value system of the research community is very different to that of an NSO. Researchers often regard some of the 'controls' inherent in the microdata access arrangements as unnecessary bureaucracy. Whilst there are no known incidents of researchers using their access to microdata to deliberately identify individuals, there have been incidents where microdata provided to them on an exclusive basis has been provided to other researchers without permission, or cases where microdata have been statistically matched without permission with other data to produce richer data sets. The researchers in question may feel they have done nothing wrong, as they have not tried to identify individuals. However, incidents of this type, if they become public, can undermine public confidence and should be treated seriously. NSOs and researchers operate in different cultures and take different views of risks from incidents. This has to be taken into account in the determination of procedures for release of microdata.
77. How can NSOs pass some of the risk back to researchers? Actions might include:
- (i) asking them to prove their bona fides as researchers and to demonstrate the public benefits of their research and that the microdata are necessary for this research;
- (ii) making them sign a legally binding undertaking with similar penalties to those operating for NSO staff if they breach confidentiality provisions;
- (iii) explaining the reasons NSOs are cautious. Ensuring researchers are fully aware of their obligations through appropriate education. Follow up with effective audit and monitoring procedures. It may be useful to establish a Code of Conduct in collaboration with the research community;
- (iv) where offences occur, withdrawing all current and future services from the researcher and possibly their institution for a period of time (e.g. until the institution has undertaken appropriate disciplinary action against the offender). Make them realise that the future release of microdata to any researcher may be at risk if there is strong public criticism. Undertaking legal action where appropriate.
78. The potential harm from an unauthorised disclosure cannot be underestimated, particularly if done deliberately. Such situations must be treated very seriously.
79. The reality is that a combination of legal, administrative and technical measures will be necessary to ensure public confidence in the arrangements. Furthermore, the research community must accept that it has no automatic right of access. The NSOs may be enabled to provide access but researcher access should be at the discretion of the NSO. There will be responsibilities associated with access. In particular, researchers should accept that they will have a shared responsibility to maintain and uphold the conditions under which they have been provided access. The limitations and safeguards may be more restrictive than exist with other data sets to which they have access but there is a good reason.
80. It is sometimes argued that respondent consent should be sought before release of microdata outside of the NSO. They argue that respondents have a right to decide how their data should be used even if it is not identifiable. This should be discouraged, as:
- (i) there are significant practical issues associated with seeking and managing consent;
- (ii) data being provided are unidentifiable and are only being used for statistical purposes, consistent with the purpose of the data collection;
- (iii) it is very difficult to provide all the information required for a respondent to make a really informed decision, - and so many respondents will say "No" just as a precaution. The sample will soon become unrepresentative if it is reliant on just those who give consent.
However, there is an obligation, as stated elsewhere in these guidelines, to be transparent about the arrangements to respondents. By this means, it can be argued that passive consent has been obtained.
(Note: If allowed by law, informed consent would be appropriate in a situation where the publication of small aggregates allows users to infer the situation of a single sample unit (e.g. person or business) that is part of this aggregate. This situation is more likely to apply to business statistics.)
81. There is another perspective on the consent issue. The data of a NSO can comprise data collected directly by themselves and data collected by administering authorities and passed on to the NSO. Unless there is specific provision in legislation or a protocol to the contrary, a NSO should not release data from administrative sources in microdata form without the consent of the administering authority (who may feel unable to give consent because of promises made to their respondents). Even when administrative data are already in the public domain, it would be courteous to advise the administering authorities to give them an opportunity to comment. Otherwise difficulties may arise with the supply of the administrative data. Administering agencies also have to manage their own privacy and confidentiality issues.
82. It is important that NSOs do some contingency planning in the event the microdata access becomes an issue for public debate. They should not assume such a debate will not happen. What are some of the key defences?
- (i) NSOs can point to the care they take in providing confidentiality protection through devices such as anonymising the microdata, providing strong physical security protection and the care taken in devising a process for the assessment of the balance between the conflicting public goods of confidentiality protection and the public benefits of research.
- (ii) If an offence has occurred and a NSO is questioned, it should be open about the offences and the penalties that have been invoked; it should make clear that the breach is the responsibility of the researcher but that the NSO will take appropriate action in response to the breach.
- (iii) NSOs should point to the overall public benefits of providing microdata access, particularly for the situation where the offence has occurred, and give some good examples
- (iv) Well-known and respected people who are prepared to publicly support the arrangements should have been arranged. Senior privacy officials may be of particular importance in this regard.