Framing a Minimum Income Guarantee

Appendix 3: Technical appendix

Method: quantitative

• The data was collected by online survey.
• The target group for this research study was a representative sample of the Scottish population. The variables used to define the sample as representative of the target group were age, gender and SEG.
• The sample type was non-probability/quota sampling.
• The sample source was access panels.
• The target sample size was 1,000 and the final achieved sample size was 1,053. The reason for the difference between these two samples was standard sampling procedures allowing for slight overage.
• Fieldwork was undertaken between 7th and 13th October 2024.
• Respondents to self-completion studies are self-selecting and complete the survey without the assistance of a trained interviewer. This means that Progressive cannot strictly control sampling and, in some cases, this can lead to findings skewed towards the views of those motivated to respond to the survey.
• The sample is judged to represent the target population well.
• Data gathered using self-completion methodologies was validated using the following techniques:
  • Where the data is collected via an internet survey using an access panel, all respondents can only submit one response due to a system of unique IDs used by panel providers. Panellist IDs are also checked for duplication as part of Progressive’s fieldwork checks. Where more than one panel provider is used, we use cookies to ensure the same individual cannot complete the same survey via two different sample providers.
• The sampling for this project was sub-contracted to Norstat and Cint.

Data processing and analysis

• The final data set was weighted to reflect Scottish population Census data. The sample base before weighting is 1,053 and the weighted sample base is 1,053.
• Quota controls were used to guide sample selection for this study. This means that we cannot provide statistically precise margins of error or significance testing as the sampling type is non-probability. The margins of error outlined below should therefore be treated as indicative, based on an equivalent probability sample. The overall sample size of 1,053 provides a dataset with an approximate margin of error of between ±0.60% and ±3.02%, calculated at the 95% confidence level (market research industry standard).
• The following methods of statistical analysis were used: Z tests and t-tests.
• The data processing department undertakes a number of quality checks on the data to ensure its validity and integrity.
• For online questionnaires, these checks include:
  • Responses checked for duplicates where unidentified responses permitted. Cookies are also used and open-ended response are checked to avoid duplicate responses.
  • The raw data is monitored throughout fieldwork to check for flatlining responses, quality of open-ended responses and speed of completion. Rules will be agreed with the DP team at the start to determine when to exclude data based on these checks. Where the sample source was an online panel, the IDs of all respondents removed from analysis are referred back to the panel provider to assist with on-going quality improvements.
• Other data checks include:
  • Every project has a live pilot stage, covering the first few days/shifts of fieldwork. The raw data and data holecount are checked after the pilot to ensure questionnaire routing is working correctly and there are no unexpected responses or patterns in the data.
  • A computer edit is carried out prior to analysis, involving both range (checking for outliers) and inter-variable checks.
  • Where an ‘other – specify’ codes is used, open-ended responses are checked against the parent question for possible up-coding.
  • Responses to open-ended questions will be spell and sense checked. Where required these responses may be grouped using a coding frame, which can be used in analysis. The code frame will be developed by the executive or operations team and will be based on the analysis of minimum 50 responses.
  • Open-ended coding is validated using a dependent approach, whereby a second person has access to the original coding and checks a minimum of 5% of cases coded. Once responses are fully coded and validated, the completed code frame is given a final check by the Executive responsible for the project, and any queries or amends are passed back to the Data Project manager.
• A SNAP programme was set up with the aim of providing the client with useable and comprehensive data, with cross-breaks discussed with the client to ensure that all information needs were met.

Method: qualitative

• The data was collected by in-depth interview.
• The target group for this research study was members of the Scottish general public.
• In total, 24 depth interviews were undertaken.
• Fieldwork was undertaken between 27th August and 4th September 2024.
• Respondents were recruited were recruited by Progressive’s professional recruiters, using their local knowledge and existing contacts. Recruiters work to predetermined quota controls to ensure that the final sample reflects the requirements of the project. All respondents are screened to ensure that they have not participated in a group discussion or depth interview relating to a similar subject in the last 6 months prior to recruitment.
• Interviews lasted an average of around 50-60 minutes.
• Respondents received an incentive of £40 to thank them for their time.
• In total, 5 moderators were involved in the fieldwork for this project.
• Each recruiter’s work is validated as per the requirements of the international standard ISO 20252. Therefore, all respondents were subject to validation, either between recruitment and the date of the group discussion/depth interview, or on the day of the group discussion/depth interview. Validation involves respondents completing a short questionnaire asking pertinent profiling questions and checking that they have not participated in similar research in the past 6 months.
• All research projects undertaken by Progressive comply fully with the requirements of ISO 20252, the GDPR and the MRS Code of Conduct.