Techscaler Programme 2022-2024: early evaluation - main report

2 Methodology

2.1 Introduction

This chapter provides an overview of the evaluation methodology which draws on a mixed methods quantitative and qualitative approach to capture the full picture of the mobilisation and delivery of the Techscaler Programme to December 2024.

Please also see:

• Appendix B — Study methodology.
• Appendix C — Stakeholder organisations interviewed.
• Appendix D — Lessons learned from undertaking the early evaluation.
• Appendix F — Current theory of change and critical review.

2.2 Study method

The evaluation was undertaken in six main stages, Figure 2.1, and each stage has been described in more detail on the following pages.

Stage 1: Inception

The research started with an inception and set-up meeting on the 8^th January 2025 between the evaluators and the steering group for the evaluation and subsequent agreement with CodeBase on a Data Sharing Agreement for the purpose of this evaluation.

Stage 2: Theory of change session

Scottish Government, together with CodeBase, have developed and refined a theory of change (and logic model) for the Techscaler Programme. The evaluation team facilitated a high-level critical review session with Scottish Government and CodeBase representatives on 14^th January 2025 to develop an understanding of the development work that has been done to develop the current theory of change and to stress test whether the right evidence is being collected. The session was also undertaken to identify possible gaps in data collection, and to inform questionnaire design for the primary research for this early evaluation of the Techscaler Programme.

Stage 3: Desk review

The secondary desk research element included a wide-ranging review of existing information and data. This included a review of Techscaler Programme documentation and data provided by both the Scottish Government and CodeBase (for example, specification, bid and contract documents, quarterly reports, etc.), and a review of published strategy and research documents.

Stage 4: Stakeholder and beneficiary engagement

Scoping interviews

Fourteen scoping interviews^[3] were undertaken with representatives from the Scottish Government, CodeBase, and Enterprise Agencies during February 2025. A semi-structured topic guide was used to inform these discussions, and these were undertaken prior to the wider primary research starting to help the evaluators:

• understand the context for the development, mobilisation, and initial delivery of the Techscaler Programme.
• understand how the programme works in practice and how Techscaler Programme support is evolving.
• inform primary research design.

Techscaler company member (beneficiary) surveys

Techscaler members who have engaged with the programme’s support — a telephone survey, which was undertaken by our study partner Research Resource, started in late March 2025, and was completed in mid-May 2025 with email reminders issued by CodeBase during the survey period.

CodeBase internal categorisation of the 978 key company contact members who have joined the Techscaler Programme as members since its launch (Techscaler Annual Report 2024) identified 330 members who have not accessed formal programme support or recorded at events. This reduced the telephone sample from 978 to 648, see Table 2.1.

The 330 key company contacts may, however, have had a touchpoint(s) which has not been recorded by CodeBase such partnership engagement, referrals, and ad hoc support and engagement with Community and Engagement Managers in the regional hubs. They may also have formed their own networks following attendance at Techscaler Programme partner event(s).

The sample then reduced from 648 to 587 when undelivered emails and those who declined to take part are accounted for.

Research Resource conducted a telephone survey using a randomised sample of Techscaler members. All selected contacts were emailed and invited to take part. While the sampling method gave each member an equal chance of being chosen, participation was voluntary and only those who agreed were interviewed. This approach helped reduce selection bias and increased the fairness of the sampling process. However, randomisation alone does not guarantee the final sample will reflect the full population, it only gives the selected contacts an equal opportunity to be included. Because of this self-selection, the final sample is not wholly representative, and the results cannot be generalised to all Techscaler members.

A total of 140 telephone interviews were undertaken with mainly founders and co-founders of companies who have engaged with Techscaler Programme formal activities and support. The proportion of completed interviews by engagement level are considered acceptable given the size of the sampling frame.

This survey sought feedback on companies’ experience of Techscaler Programme engagement, including on:

• initial engagement with CodeBase.
• the type of Techscaler Programme support accessed and satisfaction with the support.
• personal and company benefits and impacts achieved and forecast as a direct result of the support.
• the extent to which the Techscaler Programme support has met their needs.
• suggestions for how Techscaler Programme support could be further improved.

The telephone survey was supplemented with an abbreviated online survey aimed at an agreed sample — those company founders and co-founders who did not take part in the telephone survey as well as some other individual members of Techscaler (that is, member company employees and individuals at the ideation stage). This online survey went live in late April 2025 and closed mid-May 2025. This resulted in another 34 responses, all bar one was from Techscaler members who have formally engaged with the programme in some way^[4].

The surveys aimed at engaged Techscaler members resulted in a total of 173 interviews, against a target of 200 interviews (86% achieved).

Techscaler members who have not formally engaged with the programme’s support — CodeBase issued an online survey to the 330 company contacts^[5] who are registered Techscaler members but who have not yet accessed Techscaler Programme formal support. As noted above, they may have had a touchpoint(s) which has not been recorded by CodeBase or may have formed their own networks following attendance at Techscaler Programme partner event(s).

This short online survey asked questions related to:

• reasons for joining Techscaler.
• views on the application process.
• the support companies were interested in accessing from the programme.
• why they have not yet accessed formal support from the programme.
• barriers faced in accessing Techscaler Programme support.
• likelihood of accessing Techscaler Programme formal support in the future.
• suggestions for how the programme could be further improved.

This survey went live on 21^st March 2025, an email reminder was issued on 7^th April 2025, and the survey closed on 18^th April 2025.

A total of nine responses were received. This increased to ten responses once the other online survey (as described above) closed.

Partner and stakeholder interviews

In addition to the 14 scoping interviews, a further 33 interviews have been undertaken with representatives from organisations across the ecosystem. These interviews were largely undertaken during June and July 2025, and a semi-structured topic guide was used to capture informed views on startups, investment, the Techscaler Programme, and the programme’s fit within the tech ecosystem. When the initial scoping interviews are included, 47 interviews have been undertaken with Techscaler Programme partners and stakeholders as part of the early evaluation. Table 2.2 provides a further breakdown of these interviews.

Note: The total number of interviews includes three CodeBase Board members who have been included within the broader categories listed in the table (that is, they have multiple roles).

Techscaler mentors

A two-pronged approach was undertaken to capture the views of Techscaler mentors, including: an online survey which received 43 responses (29% response rate); and 12 follow-up interviews.

The purpose of this engagement was to capture the perspectives of mentors on:

• their experience of becoming a Techscaler mentor.
• how the matching and relationship management process worked.
• benefits of mentorship for Techscaler mentees.
• benefits of being a Techscaler mentor.
• the effectiveness of the Techscaler Programme support for tech startups and scaleups.
• suggestions for how the mentorship/Techscaler Programme could be further improved.

Stage 5: Early indicative impact assessment

The Techscaler member telephone survey asked questions that sought to understand the economic impacts and benefits that have been generated to date and anticipated in the future as a direct result of members engagement with the programme. It also provided an early indicative assessment of the net additionality delivered by the Techscaler Programme including the Benefit Cost Ratio (BCR) and Value for Money (VfM).

The questions were segmented based on the stage of the company (ideation, early-stage, growth, scaling) and gathered impact data relating to jobs, turnover, and levels of investment secured.

The impact assessment will also be used to set a baseline position to help monitor progress (for example, have forecast impacts been achieved) as part of future evaluations of the Techscaler Programme.

Stage 6: Analysis, reporting and dissemination

The information and data captured during the preceding stages helped to inform the interim, draft, and final reports. On submission of the draft report a presentation of emerging findings was undertaken with the client team and the Scottish Government’s new Chief Entrepreneur on the 30^th July 2025.

A further presentation of findings was held with the Scottish Government and CodeBase on 11^th December 2025. Following this meeting and further feedback the main report and executive summary were finalised.