No One Left Behind: user experience research

Annex A – Sampling and weighting

The sample for No One Left Behind service users survey was initially selected and collated by the Scottish Government, working closely with local authorities who hold contact details for No One Left Behind service users. The Scottish Government used the monitoring data submitted by local authorities, which includes a service user identification code, alongside details of demographic characteristics and employment outcomes, to select an initial sample, based on:

• Initial engagement with No One Left Behind services recorded roughly 6-18 months prior to the planned start date of the survey (comprising service users whose first date of contact was recorded as July 2024 to June 2025).
• Stratifying the sample by sex, age group (under and over 25), parental status, disability, and local authority, to ensure cases were selected roughly proportionally on these criteria
• Oversampling participants from minority ethnic backgrounds, as far as possible, to boost the size of this sub-group to enable more robust sub-group analysis.

Once this initial sample was drawn, each local authority was sent, via a Secure Transfer Method, an Excel file of selected user identification numbers and asked to add contact details and preferences. The Scottish Government’s No One Left Behind privacy notice, which is intended to be used by all local authorities, states that service user details may be shared with third party contractors acting on behalf of the Scottish Government for evaluating the effectiveness of services operating under No One Left Behind.

Local authorities returned completed spreadsheets to the Scottish Government, via a Secure Transfer Method, who checked them before securely transferring them to Ipsos.

The final sample for inviting people to take part in the survey was then drawn in two stages. A first wave of sample was issued in January 2026. At this stage, 17 local authorities had returned contact details for the survey. The nature of the sample – specifically the fact that different service users had different contact preferences, requiring different invitation routes, and the fact it included service users who may no longer be in contact with support – meant it was very difficult to predict what the response rate might be. An initial sample of 3,520 service users was selected for issue, stratified by the criteria above. Once sample for the remaining participating local authorities (all except Aberdeen City, Moray and Perth and Kinross, who did not return sample within the timeframe) had been received in February 2026, a further sample of 7,303 invites was issued. This was informed by analysis of response to the initial wave of invites, aiming to ensure around 1,000 responses overall.

As the survey was online, and email invitations can include more information than text invitations, where sampled respondents had either a preference for email contact, or no preference but had provided an email address, invitations were issued by email. Where they had a preference for phone contact, or had no preference but had only provided a valid mobile number and no email, they were invited by text. Where they had only agreed to postal contact, they were invited to take part by letter, to ensure these service users were not excluded from taking part if they wished.

Two reminder emails or text messages were sent to those invited by these modes (those invited by letter were sent a single letter, as they were primarily invited to ensure they were not excluded if they wished to take part, rather than to increase response).

Table A.1, below, shows the breakdown of sample issued by different modes. This is based on valid issued sample, excluding undeliverable messages (which indicate the contact details provided were invalid). Response rate was, as expected, higher to email invitations. While the overall response rate was low (9.6%), this was higher than our initial expectations.

1 – 471 of contacts included in the issued sample were invalid and are excluded from this figure. These were largely undeliverable text messages, indicating the phone number was incorrect or not in use.

Tables A.2 shows the number of responses by local authority. As noted above, three local authorities did not provide sample in time to be included in the survey. However, there were respondents from each of the 29 local authorities that were included. As the response rate by local authority varied, these are not necessarily completely proportionate to the distribution of No One Left Behind participants by local authority. As the aim of the survey was not to provide data at local authority level, but to provide a broad picture of service user experience across Scotland, the data was not weighted by local authority, as this would have created some large weights for individual respondents and distorted the effective sample size.

Table A.3 shows the breakdown by different demographic groups. There were some differences between the unweighted sample and the overall profile of No One Left Behind service users, including:

• Women were relatively overrepresented and men underrepresented in the sample
• Younger service users (under 25) were underrepresented relative to service users aged 25 or older
• The three priority groups – disabled service users, parents, and people from minority ethnic backgrounds – were all slightly overrepresented in the achieved sample compared with the profile of No One Left Behind service users.

In order to ensure the overall findings were more representative of No One Left Behind service users overall, the data was weighted back to match the profile of eligible No One Left Behind service users (i.e. No One Left Behind service users recorded as having first been in contact between July 2024 and June 2025) on key demographics. The weighting targets were based on data provided to Ipsos by the Scottish Government,^[12] and adjusted to take account of missing data (for respondents who were preferred not to give a response at demographic questions).

1 – Respondents were categorised as disabled if they said they had a long-term health condition or illness, expected to last 12 months or more, and this condition reduced their ability to carry out day to day activities either a little or a lot.

2 – Including those who identified as Asian, Scottish Asian or British Asian; African, Scottish African or British African; Caribbean or Black; Mixed or multiple ethnic groups; and other ethnic group.