Designing a social capital dashboard: study

1. Introduction

Background

The Scottish Government is in the process of developing and improving its National Performance Framework. As part of this work it has identified a need for improved shared understanding of, and data on, social capital in Scotland.

Definitions of social capital vary considerably though discussions of the concept commonly centre around the importance of networks which, in combination with shared social values and norms, such as trust, are viewed as enabling citizens to act together to achieve shared goals (e.g. see Putnam, 1996 and 2000, Cote and Healy, 2001). Building strong, resilient communities, high in ‘social capital’, is central to multiple Scottish Government Strategic Objectives and National Outcomes. Social Capital is seen as having intrinsic value as an indicator of social wellbeing in its own right and is strongly related to positive social outcomes such as lower crime rates, improved health, higher educational achievement, strong democracy and good governance, as well as providing an important means of preventing a range of negative outcomes. The Scottish Government has also made a commitment to trust and empower communities to control their own affairs through the Community Empowerment Act, and this depends on the existence of a strong infrastructure of connections and networks across communities. In terms of the new National Performance Framework, data on social capital will be particularly important in terms of assessing progress towards the new National Outcome, ‘We live in communities that are inclusive, empowered, resilient and safe.’

Given the complexity of the concept of social capital, and the range of indicators that have been used to measure it in Scotland (and more widely) to date, a key consideration for the Government is how a range of data might be brought together within the new National Performance Framework to improve understanding of the levels of social capital in different settings and over time at national and local authority level.

To inform these discussions, the Scottish Government commissioned Ipsos MORI to undertake scoping work to provide an account of the likely advantages and disadvantages of the development of a social capital headline composite measure or dashboard, and to provide information about what the process might be for designing and implementing this. The Government anticipated that learnings from this work would also be generalisable to other areas in terms of thinking about combining and presenting complex data in robust but accessible formats, and developing internal and external capacity for this.

Research questions

The specific research questions set for the research were:

• What existing examples are there of composite measures and dashboards, and similar methods; what are their perceived benefits and disadvantages?
• What data sources could be included in combination to provide a measurement of social capital? – e.g. SHS, other quantitative and possibly qualitative sources, also to consider how approaches could be appropriately inclusive or aspects of social capital that may not have traditionally been measured successfully or quantitatively, or may present other challenges in terms of their measurement.
• How can aspects of the design of composite and dashboard measurement enable confidence, transparency and robustness in the final design, implementation and use of any new measures?
• How could design of composite measures help inform research on social capital, or use of other social capital data sources, at a local level?
• Who are the key audiences for these measures and how could they be used at different levels i.e. Scottish Government, local government, third sector?
• What methods and technical skills are typically required in developing these measures?
• What are the options for developing a composite or dashboard measure if decisions were made to do this?
• What might be likely timescales and resources for development, testing, implementing and maintaining new measures?
• Can reflections on examples and expertise from other areas help us to think about how other data might help us to estimate more local-area social capital levels?

Previous research has highlighted weaknesses of composite measures (e.g. Ormston and Hope, 2017) – in particular, that multiple questions are rarely in a consistent format, it is often unclear whether particular measures should be given more weight than others, and there can be large amounts of missing data. In recognition of these issues, a decision was taken during the early stages of the current project to focus the research more squarely on dashboards, albeit with composite measures considered among the range of potential components of dashboards.

Methodology

The research comprised two main components: a scoping workshop with relevant Scottish Government stakeholders; and a review of comparable dashboards. In addition, we carried out a brief review of issues surrounding, and approaches to, measuring social capital, as context for the dashboard review and to further inform discussions of measures to include in a potential dashboard.

Stakeholder workshop

The purpose of the stakeholder workshops was two-fold: to enhance internal stakeholders’ awareness and appreciation of each other’s perspectives and priorities in this area; and to enable the Ipsos MORI research team to develop a clear understanding of these perspectives and priorities, with a view to refining the focus of the project. Among the key areas addressed at the workshop were:

• Which specific aspects of social capital people were most interested in capturing through any dashboard and what their views were about survey measures vs. potential ‘objective’ measures of social capital?
• Who they saw as the key users of a social capital dashboard, and what they might use it for?
• Their priorities for analytic functionality and design aspects of any dashboard tool – for example, it terms of how interactive it might be; what level of sub-group or cross-analysis might be possible; and how summary measures be presented.

The workshop also sought stakeholders’ views of some specific examples of existing dashboards and how these compared with what they would (ideally) like to see in a potential social capital dashboard.

Review of comparable existing dashboards

Based on initial proposals put forward by Ipsos MORI, together with suggestions made in the stakeholder workshop and the findings of the rapid review, six existing dashboards were selected for detailed consideration. These were:

• The Active Scotland Outcomes Framework
• The European Commission’s Youth Monitor (The Situation of Young People in Europe)
• The OECD’s ‘How’s Life?’ Dashboard
• ONS’ Measures of National Wellbeing
• Public Health England’s Public Health Dashboard
• The ‘Understanding Glasgow’ website

This selection provided a mix of dashboards in terms of the level of sophistication/design features; approaches to presenting data; levels of analysis possible; and progress monitoring.

Each dashboard was reviewed by a member of the research team using a short proforma. Additionally, in depth interviews were carried out with individuals who had been involved in commissioning/developing, maintaining and using the dashboards. The interviews were conducted by telephone and structured around a discussion guide designed by Ipsos MORI in discussion with the Scottish Government. The guide reflected the research questions set out above, covering issues such as understanding of the purpose of the dashboard; the driver for commissioning; the process of commissioning; the resources and timescales required for its development and maintenance; perceptions of its usefulness; any limitations/areas for improvement; challenges encountered (including data issues, design issues, resources/skills required at design and maintenance stage); and lessons for other public sector dashboards.

In all 14 interviews were conducted; 1-3 per dashboard.

Acknowledgements

Ipsos would like to thank the following Scottish Government representatives for their helpful input into the design of the project:

Elinor Findlay
Elizabeth Fraser
Justine Geyer
Jo Gillies
Anja-Maaike Green
Tom Lamplugh
Iain Murray
Carla Plasberg-Hill
Gordon Paterson
Jacqueline Rae
Sandy Robinson
Euan Shields
Ralph Throp.

We would also like to thank the following individuals who took part in an interview:

Justine Geyer, Niamh Laffan and Caspian Richards of the Scottish Government
Fabienne Metayer of the European Commission
Carrie Exton and Vincent Finat-Duclos of the Organisation for Economic Co-operation and Development (OECD)
Zoe Hartland, Rhian Jones and Chris Randall of the Office for National Statistics (ONS)
Julian Flowers, Daniel Flint and Natasha Roberts of Public Health England
Sheena Fletcher and Bruce Whyte of the Glasgow Centre for Population Health (GCPH)
Louise Blackstock of Hamilton College