We are testing a new beta website for gov.scot go to new site

Archive

This is an archived section of the Scottish Government website. External links, forms and search may not work on archived pages and content/contact details are likely to be out of date.

Menu

Key Sources

Statistics Methodology Glossary

Scottish Government

Social Science Research Methods

Scottish Government

The Green Book

HM Treasury

The Magenta Book

Government Social Research

Public Sector Analytical Peer Support Service

Do you have a piece of analytical work for which you would like advice or support?

Or maybe you have a general question relating to statistical methodology?

Send us an email detailing the work and advice you would like or phone us on 0131 244 0443 and we may be able to help.

Would you like to see any changes to the library?

All contributions are very welcome.

If you have any comments at all on the content or structure of the library, if you would like something added, removed or altered, please don't hesitate to email us.

ANALYTICAL GUIDANCE LIBRARY

library books

This electronic library of analytical guidance material was developed by a working group of the ScotStat Public Sector Analysts Network. The library is intended to help people involved in analytical work across Scotland locate guidance that may be useful.

The Library is arranged alphabetically, first by topic and then by title. You can scroll down through all the records or use the index to the right. The library is updated regularly.

Can you suggest any resources that could be added to the library? - email Analysts.Network@Scotland.gsi.gov.uk

C: Commissioning and Managing Research

 
Commissioning Social Research a good practice guide

Social Research Association (2002)

Synposis

This document considers ways of commissioning social research projects and offers recommendations about good practice. The guidance is not designed to be a beginners step by step guide to commissioning research, rather it offers some general principles and looks at some of the issues involved in the commissioning process.

For

Anyone commissioning research. The guidance is designed for SRA members, commissioners of research, researchers and associated professional associations.

Guidance for social research contractors engaging with the Scottish Government.

Scottish Government

Synopsis

Web page provides researchers with information about engaging with the Scottish Government as well as specific guidance for contractors on Scottish Government research projects, including information on Completing an Expression of Interest (EOI) form, Publication of Social Research, Research Tendering Process and Templates for Social Research Reports and Research Findings.

For

Guidance is designed for researchers and contractors wishing to work with the Scottish Government, however analysts in local authorities may find this information a useful point of reference when developing their own guidance.

Managing Research In Local Government - A Good Practice Guide.

Local Authorities Research and Intelligence Association (LARIA) /Bryan Hall (2005)

Synopsis

This guide has been produced in response to the demand for good practice guides amongst researchers in LARIA. It considers the specific and general management skills required to manage research in local authorities and is linked to the LARIA/LGA toolkit on 'Creating Effective Research in Local Government'. The guide is split into a number of detailed sections, some examples being the Context of Managing Research in Local Government, Making Research Relevant, Managing Staff and Projects and Good Practice in Managing Research.

For

Any research manager, this guide is designed to aid new research managers, while also providing experienced research managers with some suggestions of advice.

Procurement of Government Social Research

Government Social Research Unit

Synopsis

Guidance developed in collaboration between GSRU and the Office for Government Commerce (OGC) for government social researchers procuring government social research. The document is divided into two sections, the first provides guidance on procurement of social research including a general summary of EU and UK procurement rules and the second section offers a question and answer. It is not intended to be a comprehensive guide to commissioning and managing social research in government, but is an agreed interpretation of the OGC and EU procurement guidance.

For

Guidance has been developed for government social researchers, however the guidance may be a useful starting point on procurement for any anyone in local authorities commissioning research.

Social Science Methods Series - Guide 1: Deliberative Methods

Scottish Government

Synopsis

This brief document introduces the use of deliberative methods to engage the public in decision-making, looking at the pros and cons and the main methods of the approach. Deliberative methods are often described as hybrid of consultation and research.

For

Anyone commissioning, managing or undertaking research, who is unfamiliar with the method or approach; for policy-makers interested in approaches to public engagement.

BACK TO TOP

C: Confidence Intervals

Commonly Used Public Health Statistics and their Confidence Intervals

Association of Public Health Observatories (APHO)

Synopsis

This document describes the calculation of the more common types of statistic used within public health intelligence, specifically: rates, proportions, means and age-standardised rates and ratios. It also describes the use of confidence intervals, i.e. what they are and why they are used. It is accompanied by a useful Excel spreadsheet which provides a practical illustration of the concepts being discussed.

For

This document is useful for people who have an understanding of basic statistical concepts and who would like to develop a more detailed understanding of how confidence intervals are calculated.

Methodology Glossary - Confidence Intervals

Scottish Government

Synopsis

The Methodology Glossary is a useful quick reference guide for people who may not be familiar with statistical methodologies.

There are two sections which look at confidence intervals. These explain why confidence intervals are necessary when using survey data and describe how they can be calculated.

For

Tier 1 is useful for people who have never studied statistics before and wish to gain a basic understanding of confidence intervals and their use in statistics. Tier 2 is considerably more advanced and is a step by step guide to calculating confidence intervals with illustrative examples.

BACK TO TOP

C: Correlation

Methodology Glossary - Correlation and Regression

Scottish Government

Synopsis

The Methodology Glossary is a useful quick reference guide for people who may not be familiar with statistical methodologies.

This section explains the concepts of correlation and regression and provides examples of how they can be used. It also considers how to evaluate the effectiveness of a regression model.

For

People who have never studied statistics before and wish to develop an understanding of correlation and regression methods.

BACK TO TOP

D: Data Protection

Data Protection Act 1998 Guidelines for Social Research.

Social Research Association (SRA) and Market Research Society (MRS) (October 2005)

Synopsis

Detailed guidance produced by MRS and SRA on the Data Protection Act 1998 and how it relates to the activities of researchers.

The document provides a summary of key points of the DPA. The subsequent sections are more detailed, discussing the Principles of the Data Protection Act 1998 including its relationship with the Freedom of Information act, and the key data protection implications for each stage of the research project process. Appendixes also provide information on: key principles within the 1998 Act, disclosing personal data, data security, protocols for facilitating the sharing of personal data within the public sector, and frequently asked questions.

For

Anyone working in social research or commissioning social research needing to understand their obligations arising from the Data Protection Act 1998.

Data protection guide

Information Commissioners Office

Synopsis

Webpage with some brief information on the Data Protection Act, its requirements and the role of the Information Commissioners Office in promoting good practice and enforcing the act. The page also lists a number of links to ICO guidance for the public and private sector on how to comply with the act, such as good practice notes, codes of practice, legal obligations and guidance on determining what is personal data and determining what information is 'data' for the purposes of the DPA.

For

This guidance is not specific to analysts, however it provides a starting point and background for anyone working in the public sector who needs to consider their obligations under the Data Protection Act.

Know How No 4: Guidance for research and evaluation in Fife: Data Protection.

Fife Council (January 2005)

Synopsis

Know How guides are produced on behalf of Fife Research Co-ordination Group to provide advice and guidance on planning, carrying out, delivering and disseminating research and evaluation in Fife.

This is a very brief guide to how the Data Protection Act (1998) affects research projects.

For

It is designed for public authorities in Fife, however may be a brief source of information for researchers in other public authorities responsible for conducting or commissioning research.

Standards and Guidelines: Data protection and privacy issues.

Market Research Society

Synopsis

MRS Webpage listing some short key points regarding data protection and privacy issues, privacy and electronic communications, and also frequently asked questions relating to the Data Protection Act. There is also web links of where to find more guidance on the Data Protection Act and social research, Market Research Processes and the DPA, and Privacy and Electronic Communications Regulations.

For

Anyone considering undertaking or commissioning market research or survey work in particular and requiring information on their obligations in relation to data protection.

BACK TO TOP

D: Depreciation

Scottish Government

Synopsis

The Methodology Glossary is a useful quick reference guide for people who may not be familiar with statistical methodologies.

There are two sections which look at depreciation. The first section introduces the concept of depreciation and explains why it is required. The second section looks at several methods of calculating depreciation figures in more detail.

For

Tier 1 is for people who have never studied statistics before and wish to gain an understanding of the basic premise behind depreciation. Tier 2 goes into greater detail about the different approaches that can be used to take account of depreciation.

BACK TO TOP

D: Dissemination and Reporting

Guidance for Communicating Research Evidence

Government Social Research Unit/Canadian Health Services Research Foundation (CHSRF)

Synopsis

Short report containing a suggested structure for research reports.

For

Any-one new to report writing who is unclear how to present key finding and structure their reports.

Writing for Social Research - Contractor Instructions

Scottish Government

Synopsis

Guidance for Scottish Government contractors on how to write up research outputs for the Scottish Government, available as either a 7 page pdf or through links to individual sections. Guidance looks at key stages in writing up the research project, house style, the format of different research outputs (research reports and research findings) and templates.

For

Contractors undertaking social research for the Scottish Government. However, may be a useful reference for anyone in local authorities thinking about dissemination and reporting or producing their own contractor guidance.

BACK TO TOP

E: Equalities

 
Collecting equalities data

Scottish Government (December 2008).

Synopsis

Links to brief guidance notes developed by the Scottish Government designed to assist in the collection of equalities data.

Guidance notes focus on Age, Ethnic Group, Gender and Religion/belief and give advice on recommended questions to use, and how to analyse and disseminate the data.

For

Anyone planning, conducting, analysing or disseminating research requiring guidance on equalities considerations.

 

Getting it Right LGBT Research Guidelines

Stonewall Scotland and NHS Scotland

Synopsis

Guidance for those considering undertaking research involving Lesbian, Gay,Bisexual and Transgender communities, covering ethical issues, classification and definition, sampling, diversity and inclusiveness, methodology and reporting.

For

Anyone considering planning, undertaking or commissioning research involving the groups above.

 

 

Scottish Government

 

Long-Term Monitoring of Health Inequalities: First Report on Headline Indicators

Scottish Government Health Analytical Services Division

(September 2008)

Synopsis

This is the first report on long-term monitoring of health inequalities, produced by the Scottish Government in response to recommendations made by a Ministerial Task Force on Health Inequalities. It presents detailed definitions for each of the proposed indicators of inequalities in health outcomes (as agreed by the Task Force), and current trends to set a baseline for long-term monitoring at Scotland level.

For

This will be of interest to people working in the field of public health and to those who may be setting targets in this area. However, it is noted in the report that the indicators and measures were recommended for monitoring health inequalities at Scotland level and may not be appropriate for use at a local level so it is important to be aware of this.

 

Measuring Socio-Economic Inequalities in Health: A Practical Guide

Scottish Public Health Observatory (ScotPHO)

Synopsis

This document describes and explains some key methods used to measure health inequalities. It outlines the advantages and disadvantages of each, and contains a useful Excel spreadsheet which provides a practical illustration of the concepts being discussed.

For

The document focuses on health inequalities but may also be useful reading for someone working on different inequalities issues. Despite giving practical demonstrations of the examples, this document may be more useful to someone who has some experience of working with data.

Portal to Resources and Information on Mainstreaming Equalities (PRIME)

 

Synopsis

PRIME is a collection of equality related resources designed to help policy designers and analysts understand the broad range of equality issues that have relevancy across policy areas, and how the needs of different equality groups should be considered. PRIME also provides links to a variety of guidelines and guidance to assist everyone mainstream equality.

For

Anyone, the guidance is designed for policy makers and analysts.

 

BACK TO TOP

E: Ethics

Conducting business to business research
Conducting Market Research in Town Centres
Conducting research with children and young people
Conducting research with employees

Market Research Society: Standards and Guidelines

Synopsis

The four guidance documents above give detailed advice on the logistics and ethics of conducting research. The papers focus on issues such as consent, safety, and legalities.

For

Anyone planning or conducting research with the key groups mentioned and unsure about their obligations.

Ethical Assurance

Government Social Research: Professional Guidance

Synopsis

Full guidance and a one page summary providing guidance on ethics for all government departments to use when commissioning, managing and conducting social research.

For

The guidance is designed for government social research, but may be useful to anyone planning, commissioning, managing or conducting research.

Ethical Guidelines

Social Research Association (December 2003)

Synopsis

SRA's detailed ethical guidelines and information on best practice, outlining the principles of the SRA's ethical code, ethical difficulties and dilemmas in ethical decision making, practical advice and guidance, and also links to other web pages and references.

For

The guidance is intended for SRA members, however may be helpful for anyone at any stage of the research process requiring detailed information on ethics.

Ethical Guidelines for Educational Research

Scottish Educational Research Association (SERA) March 2005

Synopsis

Ethical guidelines for educational researchers covering a number of ethical issues including: consent, rights of children, vulnerable young people and vulnerable adults, avoidance of harm, incentives, confidentiality, data protection, disclosure and other responsibilities.

For

Guidelines are intended as a set of standards to guide the research of members of SERA, but could also be of some use to anyone involved in carrying out research in educational settings.

RESPECT Code of Practice for Socio-Economic Research

Social Research Association (2004)

Synopsis

4 page code of practice produced by the RESPECT project funded by the European Commission's Information Society Technologies (IST) Programme, which aims to form the basis of a voluntary code of practice for researchers carrying out socio-economic research in Europe and draws from existing professional and ethical codes of practice and the EU's present legal requirements.

The code discusses and offers advice under its three main principles: upholding scientific standards, compliance with the law and avoidance of social and personal harm.

For

Anyone at any stage of the research process.

Starting points for Research in Schools

Scottish Educational Research Association-Scottish Executive Education Department- Research in Schools Working Group (January 2007)

Synopsis

This document does not specifically focus on ethics. It combines a discussion of principles of good practice, including some of the key issues and responsibilities to consider when conducting research in schools and provides some 'starting points.' Individual sections briefly discuss: purpose of research, ethical considerations, research types, research and teacher development, school research communities and also provides checklists of some of the issues specific to different groups involved in school's research (e.g. head teachers, parents, education authorities)

For

Anyone planning or undertaking research in schools.

BACK TO TOP

E: Evaluation

 
Quality in Qualitative Evaluation: A framework for assessing research evidence

Government Social Research

Synopsis

The basis of this is not how to evaluate, but how to assess the evaluation. The full report is very detailed, but gives good background and reasoning behind the inclusion of elements of the framework, however the framework alone is most useful in assessing an evaluation. The questions it asks in how well an evaluation has been carried out should lead to a greater understanding in how to carry out a well structured evaluation.

For

Those with a general understanding of what is required in an evaluation, this gives in addition a check-list of what should be carried out.

Social Science Methods Series - Guide 3: Natural Experiments

Scottish Government

Synopsis

'Natural experiments' are observational studies to assess the impacts of policy interventions, which are possible where there is divergence in policy, law or practice between nations, regions or other socio-political units. Unlike experimental or quasi-experimental methods, natural experiments occur in situations that have not been designed or influenced by researchers.

For

Policy evaluators, analysts or anyone interested in the evaluation of policies and interventions, especially where a more 'experimental' approach has not been possible.

Social Science Methods Series - Guide 6: Contribution Analysis

Scottish Government

Synopsis

Contribution Analysis (CA) is an approach to assessing the performance of policies and programmes towards realising outcomes, where designing an experiment to assess causation is not possible. This brief introduction outlines the main steps in conducting a CA, how it differs from theory-based evaluation, and some uses in the Scottish policy context. CA does not try to prove that one factor - a policy - 'caused' the desired outcome, but rather explores the contribution that a policy is making to observed results. A 'theory of change' is developed to show the links between the activities, outcomes and policy context and data are gathered to test this theory, with the aim being to build a credible 'performance story'.

For

Anyone interested in the evaluation, design and implementation of policy, especially outcomes-based social policy, and anyone involved in delivering and assessing progress on the National Performance Framework.

The Green Book: Appraisal and Evaluation in Central Government

 

HM Treasury

 

Synopsis

The Green Book describes techniques and issues to be considered when carrying out assessment of the social costs and benefits of new policies, projects and programmes. There is also guidance on how to prepare business cases for economic assessment, supplementary guidance on specific issues and departmental guidance for applying the principles in specific instances.

For

This guidance is aimed at central government analysts but is equally relevant and useful to any one involved in evidence based policy.

The Magenta Book: Experiments and quasi-experiments for evaluating government policies and programmes

Government Social Research

Synopsis

Analytical guidance on the use of social experiments to determine policy impact. It discusses randomised control trials pros and cons, then discusses the use of quasi-experimental designs for policy impact evaluation.

For

Policy evaluators and analysts who undertake, commission or use evaluation. It focuses mainly on evaluating government policies, but could still provide some useful and straightforward technical guidance.

The Magenta Book: Qualitative methods of evaluation

Government Social Research

Synopsis

A detailed description of the methods that might be used in evaluation along with the principles involved in each method, some case studies and implications it has for research, evaluation and policy.

For

Those about to embark on evaluation to be aware of what is required, and what the consequences might, or should be.

The Magenta Book: What is policy evaluation?

Government Social Research

Synopsis

An excellent introduction to policy evaluation. Highlights why evaluation is needed, what it is and the role that it plays.

For

Those needing a background on evaluation - no previous skills or experience necessary.

Trying it Out: The role of pilots in policy-making

Government Social Research

Synopsis

The report is long (nearly 50 pages) and much of it covers the history and politics of pilot studies in government. There is little direct guidance on how to conduct pilots but the section on pilot methodologies (p.16) may provide food for thought.

For

Anyone involved in developing evidence based policies or programmes. No previous experience or skills is necessary to understand the report.

BACK TO TOP

F: Factor Analysis

Scottish Government

Synopsis

The Methodology Glossary is a useful quick reference guide for people who may not be familiar with statistical methodologies.

This section looks at factor analysis and at how different variables can be combined into one useful summary measure.

For

People who wish to learn the basic ideas involved in factor analysis.

BACK TO TOP

F: Freedom of Information

Freedom of Information (Scotland) Act 2002 and Environmental Information (Scotland) Regulations 2004

Scottish Government

Synopsis

Information about both pieces of legislation, their policy background and how to access information held by the Scottish Government. There is also a web link to guidance material which provides a list of training materials for Scottish public authorities and also internal guidance for SG staff.

For

Anyone requiring background information on the acts or the duties of the SG and public authorities under FOI, the guidance is not specific to research.

Know How No 7: Guidance for research & evaluation in Fife: Freedom of information (Scotland) Act 2002

Fife Council (January 2005)

Synopsis

Know How guides are produced on behalf of Fife Research Co-ordination Group to provide advice and guidance on planning, carrying out, delivering and disseminating research and evaluation in Fife.

This short guide provides some information on the FOI act and its implications for research and evaluation in public authorities.

For

It is designed for public authorities in Fife, however may be a brief source of information for researchers in other public authorities.

BACK TO TOP

G: General

Methodology Glossary - Describing Numerical Data

Scottish Government

Synopsis

The Methodology Glossary is a useful quick reference guide for people who may not be familiar with statistical methodologies.

This section introduces basic statistical terms.

For

This resource is useful for people who have never studied statistics before and wish to gain a basic understanding of the terms.

Statistically Informed

Napier University

Synopsis

This is a usefull website for Local Authority staff to find information on statistics. It includes sections covering: Useful Statistical Materials; Useful Links; and Learning Materials. The site will develop to be a comprehensive statistical repository for Local Authorities. To use the site you must register.

For

Local Authority staff who are interested in finding information about statistics in general.

StatSoft Electronic Statistics Textbook

StatSoft, Inc. (2007)

Synopsis

This electronic textbook aims to provide training in the understanding and applications of statistics. It contains explanations of a very wide range of methods and techniques, from basic introductions to key statistical concepts to more complex ideas like discriminant analysis and distribution fitting. It also contains an overview of elementary concepts, a glossary of statistical terms as well as references for further study.

For

Anyone carrying out statistical analysis and wanting information on how certain applications of statistics can be undertaken. Also useful for anyone not sure of which techniques and methods can be used for certain types analysis. The variety of material means that this resource will be useful for users with different levels of expertise.

BACK TO TOP

I: Imputation

Methodology Glossary - Imputation

Scottish Government

Synopsis

The Methodology Glossary is a useful quick reference guide for people who may not be familiar with statistical methodologies.

This section looks at missing and empty data and how to deal with it.

For

People who have never studied statistics before and wish to gain an understanding of imputation. Especially useful if you are going to conduct a survey and want to find out how to deal with missing data.

BACK TO TOP

I: Index Numbers

Scottish Government

Synopsis

The Methodology Glossary is a useful quick reference guide for people who may not be familiar with statistical methodologies.

There are two sections which look at index numbers. The first introduces the concept of index numbers and explains why they can often be a very useful way of working with data, especially when comparing values with a baseline time period.

For

Tier 1 is for people who have never studied statistics before and wish to gain an understanding of the basic ideas involved in index numbers. Tier 2 is considerably more advanced and goes into greater detail about the calculations involved in using index numbers.

BACK TO TOP

M: Modelling

Scottish Government

Synopsis

The Methodology Glossary is a useful quick reference guide for people who may not be familiar with statistical methodologies.

This section provides a brief introduction to the idea of statistical modelling and outlines some examples of situations where modelling is currently used.

For

People who have never studied statistics before and wish to get an overview of why it is useful to use models in statistics.

BACK TO TOP

P: Performance Indicators

 


Methodology Glossary - Measuring Change and Monitoring Targets

Scottish Government

Synopsis

The Methodology Glossary is a useful quick reference guide for people who may not be familiar with statistical methodologies.

This section looks at the general rules for setting targets and measuing change when using both administrative source and survey data.

For

This resource is useful for people who have never studied statistics before and wish to gain a basic understanding of target setting and measuring change.

Performance Indicators: Good, Bad and Ugly

The Royal Statistical Society (2005)

 

Synopsis

 

This is the report of a Working Party on Performance Monitoring in the Public Services. It contains high level descriptions of how to define and use performance indicators and targets with an emphasis on data quality. It contains sections on integrity, confidentiality and ethics of performance monitoring.

 

For

 

This is a really useful report which contains some useful practical guidance on developing indicators. It is a very detailed paper so may be more useful for people who already have a basic understanding of the issues involved in target-setting.

 

Target setting in a multi-agency environment

Association of Public Health Observatories (APHO) (August 2008)

Synopsis

This document provides a very useful overview of target-setting and covers things like selecting a baseline, setting SMART targets and different ways of expressing targets. It contains useful examples of good and bad practice.

For

This will be useful for people who need to set targets for their organisation/role. For people who are new to this field of work, the document complements the next item in the Analytical Library called 'The Good Indicators Guide' which it may be best to read first.

The Good Indicators Guide: Understanding how to use and choose indicators

Association of Public Health Observatories (APHO)

Synopsis

This is a useful document which considers developing and using indicators to measure performance. It comprehensively covers things like how indicators should be structured, how to get people on board and how to deal with variation. It also contains sections on Frequently Asked Questions and common myths associated with indicators, as well as a useful appendix giving more detail on what a good indicator should look like.

For

Despite the fact that the document focuses on health-related indicators, it will be useful for anyone who already works with indicators or is going to be developing new ones. It is clearly written so will be useful for everyone, regardless of their prior experience in target-setting and/or health.

BACK TO TOP

 

Q: Qualitative Research


Know How No 3: guidance for research and evaluation in Fife: Using Qualitative Analysis Software

Fife Council: (August 2003)

Synopsis

An overview of using software to help with qualitative analysis. In particular how software can be used to assist with: data storage and retrieval, developing and testing theory and writing reports. The paper also explains briefly the limitations of using software packages.

For

Those interested in the possibility of using software to aid qualitative analysis.

Methods Spotlight: Action research

Government Social Research

Synopsis

Provides an overview of Action Research and highlights an example of how the Welsh Assembly Government has used it in various programmes. Provides a list of strengths and weaknesses associated with Action Research. Also contains links for further information.

For

Those who want an overview of the qualitative method, Action Research.


Social Science Methods Series - Guide 4: Measuring and Understanding Attitudes

Scottish Government

Synopsis

Attitudinal data are key to the design, implementation, evaluation and refinement of policy interventions, especially those which are intended to have an impact on behaviours. This brief guide introduces methods for gathering attitudinal data; the links between behaviour and attitudes (e.g. Theory of Planned Behaviour); perceptions and satisfaction data; and the uses of attitudinal data by government.

For

Those involved in the development, implementation or evaluation of policies where understanding the social influences and actors which shape how people think and behave is important. Anyone interested in public attitudes and behaviours.

Standards and Guidelines: Qualitative research

Market Research Society

Synopsis

Outlines good practice and rules which members of the Market Research Society must adhere to, which in themselves are ways to proceed with research.

For

Those about to embark on a qualitative research project and wishing to read about ways to carry out the research.

BACK TO TOP

R: Regression Analysis

Scottish Government

Synopsis

The Methodology Glossary is a useful quick reference guide for people who may not be familiar with statistical methodologies.

This section explains the concepts of correlation and regression and provides examples of how they can be used. It also considers how to evaluate the effectiveness of a regression model.

For

People who have never studied statistics before and wish to develop an understanding of correlation and regression methods.

BACK TO TOP

R: Reviews

 
Finding and Reviewing the Evidence

NHS Health Scotland

Synopsis

Sections of the NICE guidance are helpful, some are probably too specific to the Health sector. Despite this, it takes you on a step by step guide through how to carry out a review.

For

Those looking to review a service/situation. Skills are required as although it tells you what to do, it does not tell you how to do it.

Rapid Evidence Assessment Toolkit index

Government Social Research

Synopsis

A thorough reference document on how to collate information that is already out there on a given subject. Gives a variety of choices that are available depending on timescales listing the pros and cons of each. Details who should be involved in the process.

For

Would need some background knowledge, but would generally be able to set you on the way to carrying out an evidence assessment.

Social Science Methods Series - Guide 5: Systematic Reviews

Scottish Government

Synopsis

Systematic reviews are one of a number of tools to help make the best use of existing evidence. A systematic review addresses a specific question and aims to reduce bias by being both systematic in, and explicit about, the step-wise approach that is used. Previously, systematic reviews focussed on measuring the effectiveness of policy interventions to produce statistical summaries of the combined results of primary studies. However, a wider range of synthesis techniques are increasingly being applied to answer different types of research questions, drawing on different types of primary study. The key characteristics, stages and tasks of a systematic review are outlined.

For

Anyone unfamiliar with systematic reviews, looking for a basic introduction to the approach, when and when not to use it, and links to resources.

BACK TO TOP

S: Sampling

Know How No 9: Guidance for Research and Evaluation in Fife: Survey Sampling

Fife Council (2005)

Synopsis

Know How guides are produced on behalf of Fife Research Co-ordination Group to provide advice and guidance on planning, carrying out, delivering and disseminating research and evaluation in Fife. This document provides simple explanations of the key issues involved in sampling. It consists of basic descriptions of sample size, confidence intervals and the pros and cons of various sampling methods.

For

It is designed for public authorities in Fife, however may be a useful introductory/overview source of information for other people in other public authorities responsible for conducting or commissioning research.

Methodology Glossary - Sampling

Scottish Government

Synopsis

The Methodology Glossary is a useful quick reference guide for people who may not be familiar with statistical methodologies.

There are a number of resources which look at sampling. These outline why its necessary to take a sample and describe the different methods of sampling that can be adopted.

For

Tier 1 is useful for people who have never studied statistics before and are interested in learning about the basic elements of sampling. Tier 2 is aimed at those who wish to look at some examples in slightly more detail and considers each of the techniques in greater depth.

The Magenta Book: What is Sampling?

Government Social Research Unit (2007)

Synopsis

The Magenta Book is organised around a number of questions frequently asked about policy evaluation and analysis. This chapter provides useful guidance about how to select an appropriate sample for a survey and explains in detail concepts like clustering, stratification, standard errors and weighting.

For

Policy evaluators, analysts, and people who use and commission policy evaluation. Especially useful for people who are new to survey work for comprehensive guidance on key aspects of selecting a sample.

BACK TO TOP

S: Statistical Disclosure Control

 


GSS/GSR Disclosure Control Policy for Tables Produced from Surveys

ONS

Synopsis

This guidance is based on the ONS statistical disclosure standards. It explains why and when disclosure control is needed, and the different methods that are used to protect tabular data. It also explains how disclosure control relates to the freedom of information act; the exceptions that mean disclosive information can not be released.

For

This guidance will be useful for statisticians and researchers who need to protect tabular data produced from surveys. It will also be useful for anyone wishing to find out more about what statistical disclosure control is and how analysts protect the confidentiallity of data collected from surveys.

Methodology Glossary - Statistical Disclosure Control

Scottish Government

Synopsis

The Methodology Glossary is a useful quick reference guide for people who may not be familiar with statistical methodologies.

There are two sections which look at statistical disclosure control. These discuss why it is necessary to ensure that individuals cannot be identified from publicly available data and how to go about protecting it.

For

Tier 1 is for people who have never studied statistics before and wish to gain an understanding of the general ideas involved in statistical disclosure control. Tier 2 is more advanced and goes into greater detail about the different approaches that can be taken to protecting data and things to look out for.

BACK TO TOP

 

S: Statistical Process Control (SPC)

 
Improving NHS Scotland - Statistical Process Control

NHS Scotland

Synopsis

This site provides a brief introduction to SPC and the history of the technique. The site also provides references to further information, access to an Excel SPC tool and other useful quality improvment tools.

For

Those involved in quality improvement primarily in health care, but also useful for anyone with an interest in or wishing to know more about SPC.

 
 
 
 
Statistical Process Control

NHS Information Services Division (ISD)

Synopsis

This page provides a brief introduction to statistical process control (SPC), a tutorial guide and a downloadable Excel SPC tool. the site also provides details of the training ISD offer on SPC and the projects that currently involve the use of SPC.

For

Anyone involved in healthcare quality improvment, or generally interested in SPC techniques.

 
 
 
 
 
 
Statistical Process Control Methods in Public Health Intelligence

Association of Public Health Observatories (APHO)

Synopsis

This document discusses the uses that can be made of statistical process control methods and illustrates this in the context of public health. It contains several examples and includes a glossary which provides useful contextual information.

For

This will be useful for anyone interested in statistical process control, regardless of whether they work in the health sector, as lots of general information is also included. The material is clearly set out hence will be useful for people of all statistical expertise levels.

BACK TO TOP

 

S: Statistical Significance

Scottish Government

Synopsis

The Methodology Glossary is a useful quick reference guide for people who may not be familiar with statistical methodologies.

This section briefly explains the ideas behind the concept of statistical significance which considers how likely it is that certain observations occur by chance.

For

People who have never studied statistics before and are interested in learning some basic statistical concepts.

BACK TO TOP

S: Survey Design

 

A Guide to ... Questionnaire Design

Local Government Data Unit - Wales

Synopsis

This guide provides advice and guidance on designing questionnaires. It focuses mainly on points to consider when writing questions and how to format self completion questionnaires.

For

Aimed at Public Body staff who design and use questionnaires to evidence service improvements. May also be useful to anyone with an interest in designing questionnaires.

 

Know How No 8: Guidance for Research and Evaluation in Fife: Survey and Questionnaire Design

Fife council (2005)

Synopsis

Know How guides are produced on behalf of Fife Research Co-ordination Group to provide advice and guidance on planning, carrying out, delivering and disseminating research and evaluation in Fife. This is a succinct summary of things to think about when developing a survey. It covers things like the advantages/disadvantages of different types of surveys, different ways of asking questions and things to consider when piloting surveys. It contains useful illustrative examples.

For

It is designed for public authorities in Fife, however may be a useful introductory/overview source of information for other people in other public authorities responsible for conducting or commissioning research.

 

Methodology Glassary - Harmonisation

Scottish Government

Synopsis

The Methodology Glossary is a useful quick reference guide for people who may not be familiar with statistical methodologies.

This section looks at standardising survey questions and outputs

For

People who have never studied statistics before and wish to gain an understanding of what is involved in standardising questions across surveys.

Methodology Glossary - Social Survey Design

Scottish Government

Synopsis

The Methodology Glossary is a useful quick reference guide for people who may not be familiar with statistical methodologies.

This section looks at the basic elements that need to be considered when conducting a survey, such as sample sizes, response rates, administration and resources.

For

People who have never studied statistics before and wish to gain an understanding of what is involved in carrying out a survey.

Methods in Survey Design to Improve Response Rates: A Review of the Empirical Evidence

Scottish Government (2010)

Synopsis

This paper addresses what can be done at the survey design stage and data collection stage to improve response rates. The focus is on methods that affect response rates, and particularly the empirical findings of research that test various practical ways to improve response.

For

Anyone involved in survey design or interested in improving response rates and what the research findings are for a number of the most commonly used methods.

Question Wording

American Association for Public Opinion Research

Synopsis

A brief overview of the main issues to consider when developing survey questions concerning opinions, rather than factual questions. Explains how to avoid leading or confusing questions to help ensure that data will be reliable.

For

People involved in developing opinion based questions without much time, or just needing a quick refresher of the key issues.

Questionnaire Design Guidelines

Market Research Society (2006)

Synopsis

This document is a set of guidelines used by the Market Research Society (MRS) to interpret the MRS Code of Conduct. It contains a lot of advice on practical issues that should be considered when planning a survey and also covers things like ensuring respondents are suitably informed of the purpose of the survey and how their results will be used.

For

Members of the Market Research Society but will be useful to anyone who is about to commission research for the first time. It may also be interesting background reading for people who are new to this field of work even if they are not planning to carry out a survey themselves.

Social Science Methods Series - Guide 7: Cognitive Testing in Survey Questionnaire Design

Scottish Government

Synopsis

'Cognitive testing' describes a range of qualitative methods used in survey design to capture people's thought processes and understanding when they are responding to questions. Used with pilots, cognitive testing helps improve the reliability and validity or survey questions and, so, improve data quality. Findings from cognitive testing also help in the interpretation of survey data. This outlines the main reasons for using cognitive testing, and its pros and cons.

For

Analysts involved in designing or commissioning surveys, especially studies involving complex, sensitive or intrusive questions, or high-cost surveys.

The Magenta Book: How are the data collected?

Government Social Research Unit (2007)

Synopsis

The Magenta Book is organised around a number of questions frequently asked about policy evaluation and analysis. This chapter provides extremely useful information about data collection and how to go about it. It presents a variety of different methods, both qualitative and quantitative, and outlines their advantages and disadvantages. It provides the reader with many useful things to consider when embarking on a data collection exercise.

For

Policy evaluators, analysts, and people who use and commission policy evaluation. Especially useful for people who are about to design and develop a data collection process.

BACK TO TOP

 

S: Surveys

 
About Statistics in Scotland - Major Statistical Surveys

Scottish Government

Synopsis

This website provides information about major statistical surveys in Scotland and the different groups who coordinate and manage them. The website links to each of the surveys and provides information about harmonisation of survey questions and the Long Term Strategy for Population Surveys in Scotland.

For

Anyone wishing to find out more about Scottish Population Surveys. The section on survey harmonisation and the standard socio-economic core questions will be particularly useful to someone developing a new survey.

Social Science Methods Series - Guide 2: Omnibus Surveys

Scottish Government

Synopsis

This briefly introduces the characteristics of omnibus surveys, which are carried out at regular intervals, with a large sample of the population. In addition to standard questions that asked of the whole sample, at every wave, subscribers 'buy' additional questions for their own needs.

For

Anyone unfamiliar with the different options available for designing and commissioning surveys.

BACK TO TOP

T: Time Series Analysis

Scottish Government

Synopsis

The Methodology Glossary is a useful quick reference guide for people who may not be familiar with statistical methodologies.

This section considers what time series are and the different types that occur. It also looks at the components of time series and how they impact on the pattern of the data.

For

People who have never studied statistics before and wish to develop an understanding of time series data.

BACK TO TOP

ScotStat Analysts Network Working Group On Analytical Guidance

The working group on ANALYTICAL GUIDANCE met just once in May 2009 and the rest of the work was taken forward through electronic communication. The group a) assessed the availability of guidance material on analytical methods and the suitability of that guidance for analysts in local government and other public bodies and b) established the electronic Analytical Guidance Library.

Details of the meeting can be seen here:

Agenda / Discussion Document

List of guidance documents

Note of meeting