Techscaler Programme 2022-2024: early evaluation - main report

7 Early indicative impact assessment

7.1 Introduction

In line with the evaluation objectives, this chapter provides an initial overview of the impact of the Techscaler Programme. This includes consideration of softer outcomes, with some early hard impacts, to help indicate whether the programme is on track to deliver longer-term results.

An independent early indicative Economic Impact Assessment (EIA) was undertaken as part of the evaluation which sought to quantify the economic benefits and impacts achieved as a direct result of supported members’ engagement with the Techscaler Programme. In line with the stated objective of this evaluation, the EIA presented in this chapter focuses on impacts that have been achieved to date. An additional EIA including forecasted impacts and benefits as well as a detailed technical note are included in Appendix H.

The Techscaler Programme contract was awarded in the financial year 2022-2023 with much of this first (financial) year of operation dedicated to set up and mobilisation. It was then in the following two financial years when member support activity commenced. Therefore, the EIA considers programme costs from 2022-2023 to 2024-2025 and benefits from the start of 2023-2024 to the end of 2024-2025.

7.2 Softer impacts

As reported earlier, the feedback captured from engaged members of the Techscaler Programme highlighted high levels of satisfaction with the support accessed. This is further reflected in the wide range of softer impacts achieved as a direct result of engaging with the programme and accessing Techscaler support.

Confidence and motivation

Survey participants reported increased confidence and motivation through Techscaler Programme engagement.

Mentorship was also highlighted as valued programme support, providing reassurance, constructive challenge, and a trusted sounding board. While international programmes and peer interactions were seen as confidence-building, broadening ambition and mindset.

Skills development

Survey participants reported that Techscaler’s education programmes (Startup Basics, First Steps, Next Steps) and mentorship helped founders improve knowledge of startup fundamentals, business models, and growth strategies.

Founders reported that skills development extended beyond technical know-how to include leadership, investor readiness, and organisational design.

Further, Techscaler Programme mentors who responded to the mentor survey confirmed mentees gained sharper focus on priorities and improved strategic thinking from the mentoring support they received.

Building networks and community and cultural and mindset shifts

The Techscaler members survey also found early signs of a shift toward a more entrepreneurial mindset, with founders reporting improved resilience and openness to learning. As an example, satisfaction with international programmes was high and the primary research intimates that exposure to global ecosystems (for example, Silicon Valley, Singapore) can encourage founders to think bigger and adopt best practices.

Partnership and ecosystem connectivity

The Techscaler members survey also found that engagement in the programme had helped founders to develop new working relationships, contacts, and networks. Being part of a ‘community’ was also valued (for members and mentors alike). Other impacts reported by members includes improved understanding of, and access to, ecosystem support. The primary research found that mentors play an important role in helping to connect mentees into other Techscaler Programme and wider ecosystem support.

7.3 Note on EIA methodological limitations

The analysis presented in this chapter and in Appendix H is based on a standard economic impacts assessment model using a mix of data collected directly from supported beneficiaries and economic coefficient and metrics sourced from official government databases.

We note that while this method is well suited to assess impacts and benefits over a short to medium term, an assessment over a longer time horizon relying only on this methodology may fail to capture or not accurately reflect the specific nature of tech ecosystem growth.

Specially, power-law dynamics of typical of start-up ecosystems, agglomeration, and spillover effects such as talent recycling, investment attraction and secondary entrepreneurship.

This has the following implications for the results presented in this chapter and in the appendix:

• “to date” analysis: over a two-year period (start 2023-2024 to end 2024-2025), it is unlikely that these innovation ecosystem specific dynamics and spillover impacts would have already begun to occur.
• “forecast analysis” (Appendix H) projecting up to 10-years from the first instance of support, it becomes more likely that innovation ecosystem dynamics and spillover effects begin to accrue. As the model does not fully capture these potential effects, it is possible that the forecast analysis may underestimate economic impacts.

We would note that these types of impacts are difficult to quantify, and a mixed methods approach supplemented by case studies may be an appropriate way to capture agglomeration and spillover impacts.

Please note that we were unable to specify the analysis to uniquely assess ideation/early-stage companies and growth/scaling stage supported members due to the sample size. This would have resulted in a confidence interval above 20% for growth and scaling members — degrading the accuracy of the analysis. Therefore, the sample of supported Techscaler members is analysed as a single cohort.

The economic impacts and benefits are considered as turnover, full-time equivalent (FTE) jobs and GVA.

7.4 Gross economic impacts

The analysis relies on primary data collected through telephone interviews of supported Techscaler members. In total, 69 of 140 of those surveyed provided responses to the economic impact specific questions within the survey. The survey was open to all supported members and as such the sample has been effectively randomised through voluntary participation.

Therefore, for the purposes of this analysis the sample is considered as 69. As shown in Table 7.1, this represents a 10% response rate and allows for a 11% confidence interval at a 95% confidence level. Good practice suggests that a confidence interval of plus or minus 10% should be achieved. A confidence interval of plus/minus 11% is therefore, just beyond the ideal range, though still represents a level of precision which support reliable and meaningful analysis. Overall, given the randomisation of the sample and the sample size, statistically, the sample is representative of the population of supported members.

A summary of the population and sample of supported members is provided in Table 7.1.

Note that we have also removed one outlier from the sample based on reported impacts which were more than two standard deviations away from the mean average reported impact. Removing the outlier from the sample has a negligible impact on the grossing up factor and confidence interval. The impacts associated with the outlier are then added back after the “grossing up” process.

The gross impacts generated to date (2023-2024 to the end of 2024-2025) represent the overall change in economic metrics reported by supported companies since their initial engagement with the Techscaler Programme and are presented in Table 7.2.

These sample impacts that have been “grossed up” to the population. They are also the “midpoint estimate” in that no margin of error has been applied.

Source: Survey Data

Note: Jobs rounded to nearest 10. Turnover and GVA rounded to the nearest £1 million.

Within the sample 44 of 69 supported members (64%) reported gross impacts within the first two years of the programme. Though, gross impacts are heavily concentrated, with three supported companies accounting for nearly three-quarters of the total reported impacts to date. The remaining companies reported significantly lower levels of turnover during 2023-2024 and 2024-2025.

Given the early stage of the Techscaler Programme and the diverse nature of the companies supported, it is not unusual for the benefits stream at this point to be driven primarily by a small number of higher-performing businesses.

7.5 Gross Impacts — Supplemental Analysis

While is it good practice to remove outliers from samples when “grossing up” as to remove any distortions in our population level estimates, given the nature of the innovation ecosystem and power-law dynamics where a single company can often outgrow the rest of the field by a wide margin (for example, tech unicorns) a supplemental analysis where the outlier is not removed has been included. These findings are presented in Appendix H.

7.6 Net additional economic impacts

The net additional economic impacts consider the additionality factors of deadweight/attribution, leakage, displacement, and economic multipliers. These are described below. Appendix H describes how these factors are applied to move from gross to net additional economic impacts.

Attribution and additionality

The following section provides further details on adjusting the gross impact to generate a net additional impact.

The assessment of additionality has been informed directly by the feedback obtained through the supported member survey and assessed on a case-by-case basis.

Deadweight

The survey sought to gather views on the level of impacts and benefit that could be attributed to the Techscaler Programme. Specifically, awardees were asked to estimate the level of attribution at year N+1 (the year following initial engagement), N+3 and N+5.

In terms of the role that the Techscaler Programme has had in generating economic impact, Table 7.3 provides a summary showing the average level of attribution reported by surveyed companies.

Note that this analysis includes data provided by responders who were not able to fully quantify or failed to report quantitative impacts but could comment on the relative impact that the Techscaler Programme has had.

Source: Survey Data. N=90

As illustrated above, most of the surveyed members reported some attribution of economic impact to the Techscaler Programme (that is, attribution 1% or greater).

However, the overall level of attribution varies, with 54 of the 90 (or 60%) respondents reporting between 10%-25% attribution and far fewer reporting higher levels of attribution, with 28 of 90 respondents (or 20%) reporting attribution 50% or above.

There is some variation between company stage with ideation and early-stage companies more incline to report lower attribution levels than growth and scaling companies. The average reported attribution for ideation and early-stage was 29% compared to 36% for growth and scaling members.

There are several methodological challenges in assessing accurate attribution of support. This includes the respondent’s own ability to assign attribution correctly, disentangle the additionality of Techscaler Programme support from support from other programmes, or asses the importance of this support compared to any financial/investment support they may have received over this same time period.

To strengthen our assessment of attribution, the EIA also considers two additional data points from the survey feedback:

• previous support received from other organisations or programmes. High levels of prior support may dilute the extent to which outcomes can be attributed to the Techscaler Programme.

Most supported members have received little support from other programmes with just under 50% reporting no other support at all and a further 40% receiving support from only one other programme or organisation.

• intensity of engagement with the Techscaler Programme. Higher levels of engagement are more likely to result in impacts that can be attributed directly to the programme.

In terms of intensity of support from the Techscaler Programme, the majority of supported members were classified as level 2 (some engagement) or 3 (a lot of engagement), accounting for 110 of the 140 respondents (79%).

Displacement

Displacement considers both product market^[18] and labour market^[19] displacement and is based on analysis of primary data collected through the beneficiary survey. Table 7.4 summarises the location of product markets showing the number of supported members who have indicated what percentage of their sales occur in each geographic region.

Source: Survey Data. N=102

Note: Survey respondents were able to select more than one geographic region and therefore the totals do not sum to 102.

Supported members were also asked to identify the geographic location of competitors.

Table 7.5 summarises the results showing the number of supported members who have indicated what percentage of their competitors are based in each geographic region.

Source: Survey Data. N=92

Note: Survey respondents were able to select more than one geographic region and therefore the totals do not sum to 92.

The responses suggest that companies expect some but limited competition with other businesses within Scotland. In general, competition is based in the rest of the UK, Europe and the rest of the world.

Further the location of sales suggests a varied product market spread across Scotland, the UK, Europe, and the rest of the world with the rest of the UK being the biggest market.

Taken together, the overall displacement effect at the Scotland level is estimated to be low at 25%.

Leakage

The level of leakage (that is, economic activity that is anticipated to be generated outside of the UK (the target region) is based on reported employment increases that are expected to be located outside of Scotland.

Leakage is assessed as the percentage of all reported employment increases that will be located outside of Scotland.

Of the 69 respondents who reported impacts, 19 indicated that some of the new jobs generated would be located outside of Scotland. However, of those reporting impacts to date this falls to just 8 respondents. Further, the scale of this employment activity is small with the overall number of jobs being 24 FTEs meaning that those employing staff outside of Scotland are employing just 1 or 2 FTEs while maintaining most activity within Scotland. Therefore, leakage has been assessed as very low at 12%.

Economic multipliers

Multipliers — the additional rounds of spend/impact from supplier purchases and wages/salaries — have been assessed using Type 2 multipliers sourced from the Scotland Input/Output Tables.

Turnover, GVA and employment multipliers have been applied on a case-by-case basis, mapping against 2-digit Standard Industrial Classification (SIC) for each supported member based on Companies House filings.

Net additional economic impacts

Table 7.6 reports on the net additional impacts generated by the Techscaler Programme at the Scotland level and are presented as:

• net direct impacts are the immediate economic activity generated by the Techscaler Programme — the outturn of adjusting the gross impacts for displacement and leakage.
• indirect impacts are the economic activity generated in the supply chain as a result of the direct activity.
• induced impacts are the additional economic activity resulting from the spending of wages earned through the direct and indirect effects.

Source: Survey Data

Note: Jobs rounded to nearest 10 and GVA and turnover to nearest £1million.

It is important to highlight the context in which these economic impacts have been generated. Firstly, this economic analysis represents impacts generated only over a two-year period and represents early findings. Particularly, within an innovation ecosystem, where companies are developing new technologies, processes, products, are engaged in research and development activities the impact time horizon is expected to be in the medium to long term. Therefore, these early-stage impacts are a positive indication of the scale of impacts that could be achieved as the programme continues.

To add further sensitivity and robustness to the analysis, we present two scenarios: representing the lower and upper bounds of the sample margin of error (plus/minus 11%), See Table 7.7.

Source: Survey Data

Note: Jobs rounded to nearest 10 and GVA and turnover to nearest £1million.

To date, the Techscaler Programme is estimated to support net additional economic activity, estimated at between 100 and 130 FTEs per annum on average (210 to 260 annual job years), company turnover between £27m and £33m and GVA between £16 million and £20 million.

The overall additionality delivered by the Techscaler Programme to date (considered as the net additional impact set against the gross impacts) is 23%. This is considered a low level, with the main driver being a relatively low level of reported impacts which can be directly attributed to the Techscaler Programme support at this stage.

It is also important to note that we would not typically expect a programme of this nature to demonstrate a high level of direct attribution. Given its longer-term ambitions in driving market adjustment and supporting growth and scaling companies, it is expected that these businesses will engage with a range of support providers and funding sources over time. This is a natural feature of the growth and scaling journey.

Therefore, while attribution to the Techscaler Programme alone may be modest, the programme is designed to be an important and contributory element in the wider success of the ecosystem.

7.7 Scaling companies supported

The EIA also sought to identify the number of scaling companies supported by the Techscaler Programme. A scaling company is identified using the Scottish National Investment Bank (SNIB) definition which defines scaleups as companies that are achieving average annual growth of 20% or more in turnover and/or employees over three years, with at least 10 employees at the beginning of the observation period.

In order to assess the number of scaleups, we rely on both to date and forecast (up to 10 years after initial engagement with the Techscaler Programme) impacts in order to capture the longer-term nature of scaling up.

After accounting for optimism bias in forecasts, there are 15 supported Techscaler members within the sample of 69 who are forecasted to employ at least 10 FTEs at some point. Of these 15, 12 meet the average annual growth requirement and thus can be classified as scaleup companies. This represents 17% of the sample.

Grossing up to the population of supported Techscaler members and applying the plus or minus 11% margin of error, it is estimated that between 100 and 125 companies could meet the criteria for a scaleup company within 10-years of receiving Techscaler Programme support.

7.8 Forecast impacts

Given the nature of the Techscaler Programme and the profile of the companies it supports, impacts are expected to be realised over a long-term lead in time.

Participating companies in the survey were asked to estimate the economic impact they anticipate over a ten-year period from the date of their first engagement with the programme (n+10). Data is presented in Table 7.8.

The estimates presented below have been adjusted for sensitivity, considering factors such as company growth stage, the level of other support received, investment secured, and industry-standard failure rates. These adjustments help account for potential optimism bias in projections and ensure a robust analysis. Further detail on the sensitivity applied and technical adjustments are presented in Appendix H.

Source: Survey Data

Note: Jobs rounded to nearest 10 and GVA and turnover to nearest £1million.