Scotland's Migration Service: early insights evaluation

7. Evaluation Conclusions and Recommendations

This early insights evaluation has considered three main evaluation questions, focusing on assessing the process, impact, and value for money of Scotland’s Migration Service:

• To what extent has SMS been delivered as intended?
• To what extent is SMS contributing to its intended outcomes?
• To what extent is SMS delivering value for money?

This chapter summarises the key findings and conclusions discussed throughout this report, and makes recommendations for service improvement and future evaluation. All findings should be considered cautiously and in light of the data limitations highlighted in the Limitations section of this report. Further research is needed to draw robust conclusions in relation to the service performance and impact. This is explored in further depth below.

Implementation and process

This early insights evaluation assessed the extent to which SMS has been delivered as intended, using the SMS theory of change as a framework. The evaluation found that core components of the service, including the digital platforms and tailored advice appointments to migrants, employers, and investors, were implemented largely as intended. The digital platform delivery featured an implementation deviation based on user research which indicated two tailored digital platforms rather than one universal platform would better meet diverging user needs. The Scotland’s Migration Service websites have had 1.4 million views over the space of 17 months (March 2024 and August 2025). The migration-related pages are particularly popular with a million views of the “Study in Scotland” page alone. Evidence suggests that the Study in Scotland marketing campaign run by Brand Scotland substantially boosted international awareness of SMS. Both the employer and migrant focused sites have directed users to book appointments with the relevant service.

Opportunities for strengthening delivery were also identified, including limited adviser capacity for complex cases, ensuring up-to-date immigration knowledge, and the need for improved resources and data collection. Stakeholder and adviser feedback pointed to opportunities for strengthening service delivery, such as expanding appointment availability, enhancing training, and developing a centralised knowledge hub. Overall, the evaluation found that SMS had demonstrated implementation fidelity and responsiveness to user needs, and identified suggestions for further improvements to ensure continued consistent quality and effectiveness across all service components.

Impact on migrants

804 one-to-one appointments were held with migrants over the first 17 months of the service with a wide range of advice provided, and the majority of appointments were with migrants who were already living in Scotland (79%). 65% of appointments were with migrants aged between 25 and 44, and slightly more women attended appointments (45%) than men (41%). Appointments were attended by a wide range of nationalities, representing 92 countries, with a notable portion of users being Nigerian (20%), American (13%), or Indian (13%). We were not able to establish whether different groups of migrants are experiencing or being impacted by the service in different ways, as there was insufficient data to do so. Future evaluations should therefore seek to investigate how experiences may differ according to socio-demographic characteristics, including any differences between the post- and pre-mover groups. The evaluation was also not able to robustly establish whether SMS makes it easier for migrants to move to Scotland and settle into their communities, one of the service’s key desired outcomes. This is due to the lack of available data and the fact the full service had been operational for less than a year at the time of evaluation. There was early evidence of at least five migrants reporting SMS had supported them to relocate or settle in Scotland from feedback data. Interview evidence also pointed to users’ needs being met, and most migrant service users responding to the feedback survey indicated that the service met their needs.^[44] Further evidence also highlighted the positive impact attending an appointment had on migrant service users’ wellbeing, confidence, and knowledge of the immigration system. The service itself was noted to support migrants’ positive feelings towards Scotland, with one participant in particular noting the fact it was a Scottish Government service made them feel supported in their decision settle in Scotland.

There is therefore some evidence that SMS has contributed to its desired outcomes, meeting migrants’ needs and helping migrants to move to Scotland and settle into communities.

There were also examples of negative experiences with the service, including one appointment where the adviser was felt to not be adequately knowledgeable. Other negative experiences were linked to ways in which external factors are constraining the service’s contribution to its intended outcomes. Notably, elements of UK immigration legislation were referenced by a number of participants as barriers to long-term settlement and international recruitment and had influenced how positively they experienced the support provided by SMS. Recent policy changes outlined in the Home Office’s Immigration White Paper and subsequent changes to immigration rules have served to make the immigration system more restrictive by making fewer people eligible to use immigration routes.^[6] This may risk making the potential choice of migrating to Scotland less accessible or attractive to a prospective migrant, as well as deterring use of the system by employers and investors.

SMS operates within a complex and challenging policy environment. The service has demonstrably responded to these challenges, and the frequent and substantial proposed and enacted changes to the immigration system which have taken place during the period of the service’s delivery. Given the sensitivities and impact surrounding changes to the UK immigration system, Scottish Government recognises that SMS will need to continue to navigate these changes in as close to real time as possible, as it has sought to during the current delivery phase.

Impact on employers and investors

In total, 749 employers and investors attended SMS webinars from June 2024 to August 2025, with 82 employers and 29 investors attending one-to-one appointments between April 2024 and August 2025. Of the employers who attended appointments, most had fewer than 20 employees (60%), with almost a quarter (24%) having between 20 and 200 employees, and 15% had over 200 employees.

There is some incomplete early evidence to demonstrate that SMS has helped employers to use the immigration system more effectively and efficiently. There is also some evidence to suggest that the service assisted some employers in meeting their skills and labour needs. More employers and investors are engaging with the service as it develops, suggesting that an increasing number of employers are supported to use the immigration system to meet their needs. At least one appointment has led to a positive outcome with an investor reporting that they successfully incorporated their business in Scotland. Feedback from the user survey was generally positive with respondents noting that the service provided them with the information they needed to deal with similar situations in the future and use the immigration system to meet their needs. Only two employers and no investors were interviewed as part of this evaluation. Both employers were NHS Scotland Health Boards who spoke positively about their experiences with the service. They indicated they felt confident recruiting internationally and complying with both sponsorship requirements and immigration law following their engagement with SMS. One employer reported that the service helped them make informed decisions linked to the recruitment of two individuals.

Indicative Value for Money

This early insights evaluation indicates that Scotland’s Migration Service has generated economic benefits, though the full scale of these benefits cannot be robustly quantified with the current data. These findings should be understood in the context of limitations in measuring impact, particularly the absence of systematic data on employment and investment outcomes, and the role of the service in delivering these outcomes, which constrain the conclusions of the value for money assessment.

The value for money assessment presented in Chapter 6 estimates an illustrative benefit-cost ratio of 0.51–0.85, based solely on a proxy measure for successful employment outcome from a small number of returned follow-up surveys and applying minimum visa salary thresholds. This suggests that the monetised employment benefits captured in the available data are lower than the costs.

However, this indicative estimate should be interpreted as illustrative and a lower bound. Follow-up survey response rates, particularly among migrants, were very low, and sensitivity analysis indicates that under our modelling framework relatively small numbers of additional unobserved employment outcomes would be sufficient for the service to achieve a BCR of one under conservative salary assumptions. We also do not monetise wider economic spillovers from employment-related activity. It is therefore plausible that the economic contribution of the service from employment income covers the cost of the service to date, although this cannot be confirmed with the data gathered.

In addition to employment impacts, early qualitative evidence suggests that SMS may also be supporting investment activity, with one investor reporting the registration of a business in Scotland following engagement with the service. Due to limited data, the scale and economic impact of any investment supported by SMS cannot be quantified and are not included in the monetised indicative value for money assessment.

Wider social benefits, including improved wellbeing, reduced stress, and increased confidence navigating the immigration system, were also reported by service users. These outcomes align with the service’s strategic objectives and contribute to benefits but are not monetised in this assessment.

Overall, while the available monetised evidence does not demonstrate that the monetised benefits of SMS outweigh the costs to date, sensitivity analysis indicates that under our modelling framework, only a small number of additional unobserved employment outcomes would be required for the service to break even. Taken together with non-monetised potential investment impacts, economic spillovers and wider social benefits, the balance of available evidence provides tentative conclusions that SMS may be delivering value for money.

Looking ahead, the value of SMS may be expected to grow as awareness and uptake increase. Strengthened outcome tracking, particularly on employment, earnings, investment, and the role of SMS in facilitating these outcomes, will be essential to enable a fuller assessment of value for money in future evaluations.

Recommendations

Digital content improvements

Feedback from one stakeholder suggests that it may be beneficial to more clearly signpost that SMS’ online content is consistently aligned with the latest changes in UK Government immigration policy, to increase confidence among users that the content is up to date. Advisers also recommended adding case studies, such as international students securing employment in Scotland or employers addressing skills gaps through global talent, to enhance relevance and engagement.

Marketing and visibility

Advisers suggested branding SMS more strongly as a Scottish Government service could help expand its reach, and both advisers and stakeholders commented on the importance of increasing the social media presence of the service. Seraphus advisers suggested using more digestible content formats including video reels, short how-to-guides, and short clips to widen the reach of SMS.

Training and resources for advisers

Insights from migrant users highlighted instances where advisers were felt to have knowledge gaps and would have benefitted from further training in relevant immigration matters. Similarly, CAS advisers who participated in the evaluation focus groups noted the importance of providing more level 1 and level 2 immigration training to respond to anticipated increased demand from users. As part of the continuous improvement of the SMS service, a further 18 CAS advisers are currently undertaking training to be certified as Level 2 advisers. This training is expected to take between 6 and 12 months to complete.

CAS advisers also suggested the following could improve delivery efficiency and output quality:

• creation of a centralised knowledge hub with up-to-date guidance;
• creation of template materials on common visa routes to minimise time spent on follow-up emails;
• and accessing more information about migrants ahead of appointments (although some CAS advisers disagreed with how useful this would be in practice, as information provided in advance of appointments as not always accurate or relevant).

Availability of appointments and advice

Both migrant and employer users provided suggestions for improving availability of appointments and advice. For migrants, this included expanding the number of appointments offered to individual users as the maximum of three per year was felt to be insufficient for more complex cases (although it should be noted that only 1% of migrant users had three appointments, with 93% only requiring one appointment). It may be beneficial to consider how migrants who approach CAS in person or via a central CAS email address or phone number for migration advice could be supported more quickly, particularly as ‘walk-in’ users are likely to be more vulnerable or have more complex needs than SMS users who access the service through the online booking tool. One employer expressed a desire for an increase in the available appointments per year due to the complexity of the immigration scenarios they face. Another employer suggested SMS should consider developing a ‘helpline’ employers and investors could phone for advice, in addition to the available phone and online appointments. This employer did acknowledge this may be an unrealistic ask of a free service, and similarly noted that advisers would not have any time to prepare for these helpline calls which could limit advice. Another employer suggested that Seraphus appointments could be arranged as six 30-minute slots rather than three 1-hour appointments.

External factors

This recommendation relates to recognising and communicating how external factors, such as market complexity and the UK Government legal and policy environment, shape the service’s ability to succeed and achieve its intended outcomes. SMS is viewed as a relatively low-budget intervention within both a sensitive domestic political context and a highly complex global migration market. While it has been possible to gather some evidence regarding the practical and personal value of the service to its users, it is more difficult to understand the macroeconomic impact of the service while isolating the effect of external factors. It has also not been possible for the service to provide tailored service offerings for subgroups of users or provide informative data related to subgroup service usage to stakeholders. These limitations, related to both the service’s scale and the context the service exists within, should be clearly articulated to decision-makers and stakeholders to manage expectations and inform decision-making, including the ministerial decision on whether the service should continue beyond March 2027. It is important that decision-makers and stakeholders fully understand the limitations and constraints of what a service of this kind can deliver, particularly when relatively small in scale.

Address service gap in relation to job opportunities for international graduates

Evidence from the evaluation suggests that one of the challenges for SMS in meeting users’ needs is supporting international graduates to find skilled work in Scotland with an organisation holding a sponsor licence. Stakeholder feedback flagged that demand from international students for jobs far exceeds the supply of relevant roles. The service should consider if further marketing and promotion of the service to Scotland-based employers, with an emphasis on the potential benefits of obtaining a sponsor licence, could mitigate this barrier.

Enhanced data collection

This evaluation drew primarily on monitoring and performance data, which was largely incomplete and consisted of small sample sizes. The service should consider how to improve data collection both through the appointment data and the feedback survey. The limited availability and quality of such data have impacted the ability to understand the impact of the service and the experience of users. Improved data collection is crucial to ensure the continuous improvement of the service, enabling it to adapt to meet the needs of migrant and employer users and deliver its outcomes.

The small number of responses to the user feedback surveys has impacted the reliability of KPI figures, largely rendering these unrepresentative of the performance of the service, and further limited the ability to draw robust conclusions from the open text responses. Service providers should consider how to maximise response rates, which may involve consideration of reminder frequency, survey length, and incentives. If the questionnaire is updated, this should remain consistent over time to enable meaningful analysis. Similarly, consideration should be given to how user characteristics can be more consistently captured during the collection of appointment data. This is particularly important for the migrant service users where missing data has impacted the ability of this evaluation to draw conclusions on the impact of the service on different groups.

To improve future Value for Money assessments, SMS should prioritise more comprehensive data collection on employment and investment outcomes. Increasing feedback survey response rates as above and refining questions to focus more directly on economic outcomes would significantly improve the robustness of benefit-cost estimates. These enhancements would reduce uncertainty in modelling and support more accurate evaluations of the service’s contribution to Scotland’s economy.

Further analysis and use of evidence could support policy development and resultant strategic service delivery for SMS. This could include filling evidence gaps in relation to which sectors and regions of Scotland have unfilled vacancies. Similarly, a better understanding of which parts of the world have an excess number of workers in relevant fields and who might stand a realistic prospect of moving to Scotland would also be desirable to inform targeting of promotional activities.

Follow-up Evaluation

While this early insights evaluation has generated valuable insights into service delivery and user experiences, it has not been able to robustly assess the impact of SMS, particularly on its longer-term outcomes. A follow-up evaluation is therefore required to assess impact of the service and the achievement of longer-term outcomes, as articulated in the SMS theory of change. This should be conducted after at least two to three years of full service operation, allowing sufficient time for early evidence of intended long-term outcomes to emerge. This follow-up evaluation should examine the extent to which the service has supported migrants to relocate to Scotland and settle into communities, as well as assess whether it has contributed to reducing skills shortages in Scotland’s economy. A mixed methods approach is recommended, incorporating longitudinal quantitative analysis of administrative data alongside in-depth qualitative research with migrants and employers and investors, including the development of case studies. A follow-up evaluation should also include an economic assessment, such as a cost-benefit analysis that considers both financial and social benefits. This approach would enable a proportionate and robust assessment of impact and inform future policy decisions on the service design and investment. In line with this recommendation, a future evaluation of SMS is planned.