A Scotland-wide Data Linkage Framework for Statistics and Research: Consultation Paper on the Aims and Guiding Principles

The main purpose of this consultation is to seek views on the aims of the Data Linkage Framework and a draft set of guiding principles.

3. Data Linkage Framework

The foundation of the framework will be a set of principles on which the other elements - The Analytical Privacy Advisory Service and The National Data Linkage Centre - can be built. The framework will be governed by a Steering Group.

Data Linkage Framework

3a. Guiding Principles

Before the details of a National Data Linkage Centre and an Analytical Privacy Advisory Service can be established it is necessary to have a set of principles on which all subsequent plans and implementation will be based. These principles should protect individuals and their data and be underpinned by the requirement to act lawfully.

The principles should flow from Human Rights Legislation, the Data Protection Act, common law requirements, guidance issued by the Information Commissioner (specifically the Data Sharing Code of Practice), and the Scottish Government Identity Management and Privacy Principles.

The draft set of principles below is an edited version of the SHIP Guiding Principles [10]. As to date SHIP has primarily been concerned with sharing and linking of health records for ad-hoc research projects the edits mainly reflect a broadening to other sectors and the production of official statistics.

As explained in the SHIP document, principles are not rules. Principles sometimes conflict. This is why they are starting points for deliberation or action. Because of their fundamental importance, however, it is expected that they are followed where they are relevant to a given data use, storage, sharing or linkage practice. The value is that such principles provide a common framework of reference so that all parties can agree what is at stake; moreover, they provide a statement of the standards against which all parties agree to be bound.

The Principles

Resource allocated to the application of the principles should in proportion to the risk involved in the linkage proposed.

Public Interest

1. Scientifically sound and ethically robust research and statistics are in the interest of the public. The objective of the Data Linkage Framework is to facilitate scientifically sound and ethically robust research and statistics through the appropriate and safe use of data.

2. The rights of individuals should be respected with adequate privacy protection, while at the same time the benefits for all in the appropriate use of data for research and statistical purposes should be recognised.

3. The production of scientifically sound and ethically robust research and statistics should be considered necessary grounds for data sharing, providing that ethical and legal standards are met.

4. The production and dissemination of statistics through data linkage
should be in accordance with the Code of Practice for Official Statistics, The Pre-release access to Official Statistics Order (Scotland) 2008 and National Statistician's Guidance on Confidentiality of Official Statistics.

5. Where linkages resulting in commercial gain are envisaged, this should be clearly and publicly articulated and widely communicated.

6. Benefits arising from linkage of personal data are public goods and should be shared as widely as possible.

Governance and Public Transparency

7. Data sharing and linkage should be carried out under transparent controls and security processes, and the purposes and protection mechanisms should be communicated publicly and to oversight bodies/individuals with responsibility for data processing.

8. Information about all approved linkages; all privacy impact risk assessments; and all data sharing agreements for linkage purposes, should be made publicly available.

9. All practices, including all data linkages, shall be appropriately monitored and regulated by a relevant individual, organisation or governance body as appropriate. It is possible that these activities will be monitored at an individual and organisational level simultaneously.

10. There should be a clear distinction in roles between those carrying out linkages, analyses and those policing governance and enforcing sanctions.

11. As far as possible, account should be taken of the full range of stakeholder positions in the development and implementation of governance arrangements.

12. The interests of one (or a few) stakeholder(s) should not dominate use/linkages or the conditions of the same, especially where this might be at the expense of other stakeholder interests.


13. Data controllers should demonstrate their commitment to privacy protection through the development and implementation of appropriate and transparent policies.

14. Every effort should be made to consider and minimise risks of identification (or re-identification) to data subjects and their families arising from all aspects of data handling.

15. Serious consideration should be given to carrying out privacy-impact risk assessments, following the most up to date ICO guidance. Where a PIA is not considered feasible or necessary that should be clearly and publicly articulated. PIAs should be made publicly available (excluding sections as necessary for reasons of security), before any cross-sectoral linkage occurs.

16. Linked datasets should be kept for the minimal time necessary for the original purpose of the linkage to be met. If a secondary purpose arises, a new Privacy Impact Risk Assessment should be conducted, and data sharing agreements revised.

Removal of names and direct identifiers

17. The default position should be that data users have access only to data from which names and direct identifiers have been removed, and data users should be subject to an obligation not to attempt to re-identify individual data subjects. Any requirement for researchers to have access to data containing identifiers should be fully justified and risk assessed.

18. Data controllers should determine and agree upon the appropriate extent of anonymisation to be applied to any given dataset or linkage exercise.

19. The risk of re-identification of data subjects must be assessed by a body/individual with the relevant expertise to make such judgments.


20. Where possible and practicable, explicit consent should be obtained from each data subject prior to the linkage of personal data for statistical and research purposes. Personal data are those from which an individual is identifiable or is likely to be identifiable.

21. Where possible and practicable, individuals collecting data should adequately inform data subjects of all material issues relating to the storage and use of their data. Material issues are those likely to affect a person in a non-trivial way.

22. Where personal data are used, the minimum amount of personal data should be used to achieve the stated objective; the reasons and justification for its use should be adequate and clearly explained; and reasonable efforts should be made to inform data subjects of the purposes of the use.

23. Where obtaining explicit consent is not possible/practicable, and in all uses of data which are beyond those specified when consent was obtained then (a) removal of direct identifiers should occur as soon as is reasonably practicable and/or (b) authorisation from an appropriate oversight body should be obtained which can confirm that the public interest in data linkage is met and appropriate safeguards are in place.


24. Data linkage activity must be governed by an Information Security Policy appropriately implementing HMG Information Security Policy Framework .

25. Appropriate physical and technical security measures should be applied to ensure the confidentiality, integrity and availability of information and reflecting the assessed risk level of information assets.

26. All personnel involved in data linkage activities should be properly trained on the data security policies and procedures, and should undertake periodic refresher training.

27. The importance of data security should be reflected in the business objectives of all organisations involved in data linkage.

28. Information about data security policies and procedures should be highly visible within organisations conducting indexing or linking or sharing of personal data.

Access and Personnel

29. Access policies should be developed in a transparent and open manner; these should also be subject to public scrutiny and review.

30. All data recipients should be appropriately vetted to ensure they have adequate training. Vetting procedures should be robust and transparent and proportionate to the requests made and the sensitivity of the data requested.

31. All personnel involved in data linkages should be fully aware of their roles and responsibilities, including these principles, and must abide by the relevant Data Sharing Agreement.

32. These roles and responsibilities should be subject to robust governance mechanisms designed to ensure that these roles are being carried out appropriately and to the standards legally and ethically required.

33. Whether a single data controller or otherwise, a clear distinction should be maintained between each of the functions of linker, indexer, recipient and data custodian. Linkers should be responsible only for linking data.

Clinical Trials

34. Mechanisms for linkages involving clinical trials must permit re-identification by the principal data source, this is particularly important for pharmacovigilance purposes.

35. The specific circumstances and conditions governing whether or not patients involved in clinical trials can be contacted and by whom, should be clearly set in place in transparent policies.

36. Researchers should only seek to contact participants directly with respect to information arising from a clinical trial in which they took part where prior consent to be contacted for specific purposes has been obtained.

Data sharing agreements and sanctions

37. Roles and responsibilities of parties to data linkages should be identified from the outset, terms and conditions for data sharing should also be agreed upon in the form of a memorandum of understanding.

38. Where researchers wish to deviate from/modify the terms of the data use/sharing agreement at any time, new terms must be agreed upon by all parties concerned and such changes should be monitored by the relevant oversight body/mechanisms.

39. Sanctions for failure to respect terms and conditions should be clearly stipulated in all data use/sharing documentation.

40. Sanctions should be enforced by a body/individual independent of those granting permissions for access to data sets (i.e. data controllers)

Consultation question 3:

Are the guiding principles sufficient and appropriate?

Please explain your answer fully and make suggestions for improvement.

3b. Governance

The Data Linkage Framework is a fully collaborative, cross-sectional, multi-agency approach to enabling and facilitating data linkage for statistical and research purposes in Scotland.

A Data Linkage Steering Group has been established to oversee and guide delivery of the framework, including the strategic direction of the National Data Linkage Centre and The Analytical Privacy Advisory Service.

Chaired by Director General: Governance and Communities (Scottish Government), the Steering Group comprises academic experts on medical research and jurisprudence, as well as representatives from:

The Information Commissioner's Office
National Records of Scotland
NSS: Information Services Division
Scottish Collaboration for Public Health Research and Policy
Scottish Government (various directorates)
Society of Local Authority Chief Executives
Research Councils.

The UK Context

In government, law, education, health and many other spheres Scotland has distinct systems from those in England, Wales and Northern Ireland, as well as correspondingly different needs for statistical evidence. The Scottish Government is however keen to work collaboratively with other administrations across the UK to share best practice and to reap the benefits of co-ordinated approaches where sensible to do so.

Such partnership working is desirable where UK level statistical outputs are of mutual value, such as in making better use of existing data to develop and test more efficient methods of producing census type information and for producing summary statistics about the size and structure of the population; in understanding migration between parts of the UK; and on issues reserved to Westminster.

The Data Linkage Steering Group is working with partners across the UK through the Administrative Data Task Force. Established in October 2011, this Task Force will consider data sharing and linkage for research and statistical purposes across the UK. It is expected to report in September 2012 and to make recommendations to UK and Devolved Government Ministers.

3c. Privacy Advisory Service

One of the challenges that can cause difficulties in establishing efficient and effective data linkage projects is the considerable variation in the interpretation of the legal and regulatory environment. Data custodians may often be unsure whether they can legally and appropriately make data available for linkages and so, to be on the safe side, turn down requests for access to data.

To address this, a National Privacy Advisory Service is proposed, with the aims of helping all those involved in data linkage projects strike the right balance between safeguarding individuals' right to privacy and the efficient use of data for statistical and research purposes through careful application of the Guiding Principles.

This section provides a summary of early ideas about a Privacy Advisory Service. Firm proposals will be consulted on at a later stage, when this consultation on Principles has been completed and the UK wide Administrative Data Task Force has made its recommendations. An important consideration will be avoiding any increase in regulatory burden and complicating the landscape further.

Early ideas, on which views are welcome, are that the Service will provide a one-stop-shop for quality advice on linkages and associated ethical, legal and social issues applying advice on data linkage projects that is proportionate to the risks to privacy and potential for public benefits[11]. Objectives for the Service could include:

  1. Helping data custodians consider the privacy implications of applications to link their data to other data
  2. Helping researchers navigate through the ethical, regulatory and legal issues when planning and conducting data linkage projects, in particularly by making recommendations to mitigate risks to the privacy of individuals and maximising the public good.
  3. Making recommendations as to whether or not data linkages are conducted, giving advice that is proportionate to the risks involved, such as privacy and reputational risks, and in light of the public benefits that would accrue if the linkage took place.
  4. Making technical suggestions for improving security, methodology and analysis of any linkage.
  5. Publicly demonstrate high standards of decision making about the use of personal data for cross-sectoral linkage projects.

A Service that can advise on data linkages across sectors as well as within them would need to have all sectors within its scope, and would need to incorporate people with experience and expertise of legal, technical and public acceptability issues from different organisations. Whether any or all appointments to the Privacy Advisory Service should be appointed through the Public Appointments Scheme and according to the Nolan Principles is under consideration, as is the potential for the service to have some statutory footing to approve data linkages at some point in the future.

Consultation question 4a:

Are the objectives for a Privacy Advisory Service set out in section 3c the right ones? Please explain your answer fully and make suggestions.

Consultation question 4b:

Do you wish to be consulted on firmer proposals for a Privacy Advisory Service as and when they are developed?

3d. National Data Linkage Centre

A National Data Linkage Centre will be established to provide a clear focus of activity and development. It will function according to the Guiding Principles under consultation.

The centre will build on the experience and expertise of organisations already involved in data linkage, and will be a collaboration between NSS:ISD and National Records of Scotland in the first instance. A collaborative approach to further developing data linkage infrastructure, methods and expertise in Scotland will realise efficiency savings.

The Centre will provide those functions that are best hosted centrally and/or are integral to developing alternatives to the decennial census. This includes:

  1. Leading development of data linkage IT and expertise
  2. Development and maintenance of methods for read-through between different individual referencing systems
  3. A linkage service: conducting approved within and cross-sector data linkages where necessary and efficient
  4. A trusted data-exchange service
  5. Development and maintenance of a 'population spine'
  6. Co-ordination and support for any 'satellite' data linkage units/safe havens that continued to function in other bodies (for example ScotXed)

Data security will be a high priority for the Data Linkage Centre and there will be provision to fund information security management activities with visible support and commitment from all levels of management. Accreditation of IT systems to formally assess them against their information assurance requirements, resulting in the acceptance of residual risks in the context of business requirements will be properly considered.

Consultation question 5a:

Are the functions that will be led by the National Data Linkage Centre set out in section 3d the right ones? Please explain your answer fully and make suggestions.

Consultation question 5b:

Do you wish to be consulted on firmer proposals for a National Data Linkage Centre as and when they are developed?


Email: Andrew Paterson

Back to top