Town Centre Regeneration: How Does it Work & What can be Achieved?

6 USING A THEORIES OF CHANGE APPROACH

Introduction

6.1 The purpose of this research was to develop a clearer understanding of the activities taking place as part of town centre regeneration and assess the scope and nature of the outputs and outcomes that follow on from this. A main aim was also to develop and populate a theory of change for town centre regeneration ( TCR) drawing on existing data and evidence clearly identifying the gaps in the evidence that should be explored. This chapter summarises the limitations in town centre regeneration project planning, monitoring and evaluation highlighted previously. It illustrates how the Theories of Change approach might offer solutions to these limitations using the TCRF case studies in the short-term and in future for town centre regeneration more generally. It addresses:

• What is Theories of Change and why use it;
• How Theories of Change helps in overcoming limitations in planning;
• Using Theories of Change to further specify TCRF plans;
• Using Theories of Change to address limitations in monitoring and to identify the need for project specific questions and measures;
• Using Theories of Change to enhance monitoring and evaluation plans; and;
• The potential of Theories of Change to enhance wider town centre regeneration.

6.2Chapter 5 illustrated that many of the problems highlighted in the literature review with regard to TCR in general were also evident within the TCRF projects. The research has identified limitations in planning, the use of routine/town centre health check data for monitoring and in the robustness of monitoring and evaluation plans.

6.3 The limitations in planning identified included: a too narrow focus on physical place and its anticipated impact on retail as a means of regenerating town centres of varied sizes and types; poorly detailed activities and intended target groups; limited specificity with regard to the levels, types and timelines of expected changes (outcomes); and, a lack of results chains that highlight the anticipated casual links (the how and why) between activities and short, interim and longer-term outcomes.

6.4 The limitations in routine/health check data included: a lack of consistent approaches which prevent within and across area comparisons of need or progress; no agreed methods for conducting health checks; lack of robust routine data available frequently and at small area levels; and, limited resources to commission primary data collection to fill these gaps.

6.5 The final set of limitations relate to there being a lack of robust monitoring and evaluation plans. Two key reasons for this are the earlier limitations in planning and routine monitoring of town centre outcomes. Impact evaluation relies on projects having: a clear anticipated results chain; well specified outcomes; robust monitoring data on both outcomes ( e.g. routine sources such as health checks) and project implementation (activities and outputs delivered); and, integrating process and outcome data to enhance attribution (confidence that changes in outcomes result from the project activities and not other interventions). In addition, there is a lack of people skilled in evaluation methodology within TCR that can identify and fill gaps via primary research and integrate both implementation and outcome data to provide lessons for future programmes.

6.6 These limitations were, to some extent, already known to commissioners and informed the aims of this research which included:

• To develop a clearer understanding of the activities taking place as part of town centre regeneration and assess the scope and nature of the outputs and outcomes that follow on from this; and,
• To develop and populate a theory of change for town centre regeneration drawing on existing data and evidence clearly identifying the gaps in the evidence that should be explored.

6.7 As a means of further highlighting and beginning to tackle these issues the research team adopted aspects of the Theories of Change evaluation approach as requested in the second aim. The rest of this chapter discusses what this approach involved and the extent to which it offered solutions to these problems within the TCRF, as well as its potential contribution for wider TCR.

What is Theories of Change and Why Use It?

6.8 Theories of Change ^[5] is an evaluation approach that is increasingly used in complex interventions to enhance planning, support the development of an evaluation framework and to aid attribution (being able to attribute changes found to the project activities). The approach involves encouraging project stakeholders to prospectively describe and link their long-term, interim and short-term outcomes back to the activities and outputs they intend to deliver. It also promotes consideration of why such changes might occur as a result of these activities. This process therefore uncovers a project's results chain (theory of change) and some of the underlying theories that underpin it. The resultant theory is often presented as a logic model (See Figure 6.1).

Figure 6.1: Generic Modelling Approach

6.9 This approach when used prospectively therefore seeks to encourage the development of plans which provide greater specificity in relation to the activities and the outcomes and the linkages between these. This means providing information about who the activities are targeted to, which outcomes the activities hope to change, in what way and by how much and why they might do so. Logic models are tools used in order to develop theories of change as they provide a framework to align activities, outputs and outcomes. Ideally, a Theories of Change approach is used at the start of programme or project design so that the plans are developed as described above. Often, however, it is applied retrospectively as part of an evaluation. Its role is then to unpick and understand what is/was actually delivered and then use the learning that results to inform the evaluation design, methods and tools. In the TCRF case studies the approach was in a sense 'tagged on' to an already developed project and evaluation design. In the case studies the Research Team were presented with finalised projects and existing monitoring and evaluation arrangements rather than using Theories of Change as part of the early project development process.

6.10 For this study, the research team were asked to develop and populate a theory of change/logic model for town centre regeneration based on the literature and evidence-base. The literature review was conducted long after TCRF project plans had been funded (in fact projects were in the process of implementation). Given this there were limitations in the extent to which the approach/models could influence ongoing TCRF plans/implementation. Despite these difficulties it was thought that drafting hypothetical theories of change may have utility for future programmes and policy development.

6.11 The literature review highlighted that there was a lack of specificity generally in town centre regeneration plans and a lack of evidence about the postulated or actual links between activities and their anticipated outcomes. (See Sections 3.23 & 4.32.) Given this the draft models were based on the original TCRF submissions and the typologies.

6.12 The research team developed three draft theory of change models that illustrated the key activities that many of the TCRF projects were intending to deliver for 1) public realm and streetscape; 2) retail and business; and 3) housing and leisure. It was also an attempt to map, or where such information was not specified by the TCRF plans, hypothesise the steps and mechanisms by which these activities would feasibly lead onto their associated expected project outcomes. Finally, it was hoped that there might still be time available to use the models to encourage greater specificity in the types and thresholds of anticipated outcomes amongst the case studies. This in turn would make future monitoring and evaluation more feasible.

6.13 A fourth model was also developed that attempted to integrate the initial three models into an overall strategic model for town centre regeneration generally. The sub model for public realm is detailed in Section 6.16 and all the models are included in the Appendices report which is available as a separate document.

How the Draft ToC Models Help in Overcoming Project Planning Limitations

6.14 The previous chapter illustrated that there were limitations in the way the TCRF plans were presented. There were inconsistencies in the descriptions of activities and outcomes, outcomes were not ordered in terms of expected timescales of delivery and were not clearly linked back to particular activities. In other words, the plans did not provide a results chain.

6.15 Figure 6.2 below illustrates a draft theory of change/logic model. This model focuses on the intended activities and outcomes in projects focusing on public realm interventions in particular. It attempts to place the types of activities and outcomes described in the TCRF plans in the order they will be delivered /achieved and illustrate some of the interim steps (short or interim outcomes) along the way. Many of these steps were missing from the TCRF plans.

Figure 6.2: Draft Theory of Change/Logic Model

6.16 Representing plans as a theory of change/logic model clarifies activities, outputs and outcomes, the timelines for delivery of outcomes and the earlier steps that are necessary to achieve longer-term outcomes. For example, the logic model makes it more explicit (see shaded boxes) that improved business performance (particularly for retail businesses) is likely to be dependent on increased footfall and spend from locals and visitors which in turn are likely to be dependent on achieving enhanced public spaces, with improved facilities.

6.17 Representing the plans in this way also highlights where key activities may have been overlooked. For example, is a 'shop local' marketing campaign necessary to inform locals and visitors about the independent retail offer and new/upgraded facilities?

6.18 The development of the four logic models/theories of change illustrated that there are substantial evidence gaps within town centre regeneration. For example, if regeneration specialists are claiming that by enhancing public realm ( e.g. via paving, street furniture, enhancing shop fronts and Wi-Fi) they will improve the economy and community, then the routes leading to these changes need to be better articulated and tested over time via monitoring and evaluation.

Using the Draft Theory of Change Models to Further Specify Existing TCRF Plans

6.19 By the time the early phases of the research were conducted (literature review, survey and typologies) many TCRF projects had moved into a delivery phase. This meant that there were limitations in the extent to which there was scope to change plans. The draft theories of change were, however, shared with the TCRF case study area contacts in the first stage of the field work. They were received positively and were used as part of the case study discussions to encourage further specificity in terms of both plans and establishing a more robust monitoring and evaluation framework where this was feasible. This resulted in some case study contacts completing tables with the evaluators which defined outcomes as short, interim and longer term and tied the emerging results chains more specifically to activities. (See TCRF Case Studies report). As an example, Figure 6.3 shows the table completed for the Stromness Case Study Project: Outcomes Table.

Figure 6.3: Stromness Case Study Project: Outcomes Table

Activities						Outputs	Short Term Outcomes: 2011	Interim Outcomes: 2013	Longer Term Outcomes: 2015+	Measure In Place
Community & business engagement in design						Number of people attending consultation events	Changed decision; went for local sourcing resulted in 2/3 local jobs	Increased membership of Stromness Business Group ( SBG) Local quarry continues in operation	Strong resilient SBG Quarry business established & sustainable	Membership of SBG pre-2010 & post TCRF project completion Monitor quarry business
Improved setting for refurbished vacant/ significant buildings e.g. Commercial Hotel						Remove blight	Hotel converted for small business space targeted at companies in the 'renewables sector'	More Attractive Environment Improved Sense of Place Local residents & visitors acknowledge improved town centre environment Improved Sense of Pride	Stronger Economy More footfall (residents & visitors) More investment Increase business relocations/start ups/business growth Increase jobs	Perception surveys to be repeated Repeat number of business/jobs/vacancies survey
Improved setting for the Pierhead Regeneration Project						Area improved	More OIC jobs based in Stromness Local residents will make fewer trips to Kirkwall Improved sense of pride in perception surveys		Reduced Carbon Footprint	Perception surveys to be repeated Repeat number of business/jobs/ vacancies survey
Improved events space at Pierhead						Area improved Increased number of events organised & promoted	More visitors, staying longer, spending more	Increase number of events & attendance Establish spin off impacts for local shops/services	Stronger Economy	Number/type of events Footfall survey: residents & visitors
Improved walking & cycling connectivity						Increased use of public /routes	Improved health (increase take-up of healthy lifestyles within the community)		Footfall & cycle use survey
Installation of a fibre optic ducting network						Length of fibre optic improved: number of properties served by improved broadband	Improved IT Infrastructure Identified speed & capacity improvements Increase in business start up & business growth in town centre to take advantage of improved broadband	Inward Investment: new businesses OIC decentralisation of services & staff	Stronger Economy:	Need details on existing and proposed speed/ capacity/ constraints Take up by OIC & business Regular business surveys: take up of improved broadband, start up, growth, jobs, confidence Monitor number of OIC jobs relocated to Stromness
Renew underground services						Length of pipes/wires renewed			Ensure services are fit for purpose in future
Install improved signs						Number of improved signs	More visitors, staying longer, spending more		Stronger Economy	Footfall survey: residents & visitors

Notes
OIC identified outcomes in bold

6.20 Although all four models were made available, the public realm and streetscape model was the one used most frequently and added to by the case study contacts. This reinforces the predominance of public realm and place making in the TCRF. This might also be due to the funding levels and process encouraging ready to go projects as explained in Chapter 5. Few of the case studies used the retail and business model or housing and leisure models.

6.21 Anecdotal feedback from the case studies contacts was that the models were useful in encouraging them to further specify their activities and improve alignment of activities with particular outcomes.

6.22 One of the case study areas used the model with their project team to articulate more detailed interim steps linking key activities to particular outcomes. Figure 6.4 below shows how the Barrhead case study team used the public realm template model and identified missing activities and inserted additional outputs and outcomes. The additions made by the Barrhead contact are shaded.

Figure 6.4: Barrhead Public Realm Model

How the Models Help to Explain Limitations In Monitoring and Need for Project Specific Questions and Measures

6.23 The draft models were useful in highlighting limitations in the monitoring and evaluation plans. The research team then used the emerging outcome/theory of change tables (see Paragraph 6.19 above) to plot the existing baseline measures (routinely available and project specific /primary data) being used by projects. It became evident that many of these measures addressed longer or interim project outcomes leaving a gap in relation to the short term (in the main project specific measures).

6.24 The TCRF funding was intended for spending in one financial year (2009/2010) and many of the projects have been delayed in terms of their implementation. Given this, the absence of short or interim measures means that there may be limited evidence available to assure Scottish Government and other funders that projects are 'on route' to achieving their interim or longer term outcomes.

6.25 This reinforces the learning point from above and Chapter 5 that routine data alone may be too late to show changes within funding periods. Project specific measures, if developed for baselines and then followed up, would show change within, rather than beyond, the three year timescale of the project funding. Specific project measures (of process and outcomes) may also be vital in attributing any long term change that is measured to the contribution made by the TCRF projects.

6.26 The process described above helped both the research team and TCRF contacts to identify gaps in existing baseline measures and establish whether these could be filled. It also highlighted where follow-up measures were available and if they were sensitive or timely enough to measure the likely levels of change anticipated in outcomes within or beyond the TCRF funding period.

Example

In Stromness gaps in project specific baseline and follow-up measures were identified in relation to the improved events space, the number and type of events that could be held, the absence of a regular footfall or business surveys. These gaps should addressed by commissioning short sharp surveys. In addition, the installation of ducting will enable the future provision of a fibre optic network but the details of delivering the actual network have still to be established.

6.27 Figure 6.5 uses a refined version of the draft public realm theory of change to further illustrate how such models can highlight the limitations in routine monitoring data and the need for project specific/primary data. It illustrates that the more routinely collected data (such as rental levels, capital values, vacancy levels, footfall, employment rates), even when available at the appropriate town centre level, tend to relate to interim or longer term change.

6.28 Considering the available measures directly against the outcomes and anticipated timelines in a logic model highlights the importance of more timely and sensitive measures for projects outputs and short term outcomes. Without such measures, judging project impact (effectiveness) and attributing this to TCR in contexts where many other activities are being simultaneously delivered will be almost impossible.

Figure 6.5: Public Realm Theory of Change: Routinely Collected Data

Using Theories of Change/Logic Models to Inform Monitoring and Evaluation Frameworks

6.29 Section 6.19 above highlighted that the Theories of Change approach can aid the development of monitoring and evaluation plans. This is partly because it can be used as described in 6.23 - 6.25 above to illustrate where there are limitations in monitoring data and missing measures (routine or primary). When Theories of Change is applied earlier in programme and project development and evaluation it can be used in a more influential fashion.

6.30 Theories of Change should ideally be used to support the development of logic models (also called results chains). These should be developed collaboratively between evaluators and implementers and other stakeholders. Again, evaluators should work with stakeholders to critique and strengthen these models. This is done by asking questions about the following criteria in relation to the models: is the model plausible ( e.g. is it based on evidence and logic); do-able ( e.g. can it be delivered with the available resources and timescales); and testable ( e.g. is it expressed in a way that allows measurement)?

6.31 Learning from the above process is used to strengthen the plan and adapt it according to the available evidence-base or improve the clarity and specificity on activities and outcomes. Tools such as the Re-aim Framework (Re-Aim.org) look to encourage programme and project planners, evaluators, funders, and policy-makers to pay more attention to essential programme and project elements, including external validity, so as to improve the implementation and effectiveness of evidence-based interventions. The Re-aim Framework can be used alongside the models and criteria above to assess the likely impact by considering:

• Who the activities reach;
• Whether the planned interventions are known to be effective;
• What organisations or settings might adopt the activities;
• Whether the activities have been implemented as planned; and
• How long the programme will be maintained for.

6.32 These questions help to highlight potential weaknesses in plans and also begin to identify other important potential evaluation questions.

6.33 The numerous issues that flow out of this critique of the logic models become potential evaluation questions and need to be prioritised in terms of the importance of their learning for project improvement or in judging efficacy. Similarly, the models are used to agree the most important outcomes that should and can be measured. The prioritised questions and outcomes then inform the evaluation design and influence the selection of methods and tools. Consideration needs to be given to whether existing data sources or new primary data needs to be collected to address these key questions and establish achievement of the prioritised outcomes. The next step is to develop an evaluation framework/data collection plan.

6.34 Figure 6.6 below highlights how this might happen focusing on one strand of the draft public realm model. It focuses on the provision of Wi-Fi and its associated outcomes which are shaded in Figure 6.6.

Figure 6.6: Provision of Town Centre Wi-Fi & Associated Outcomes

Examples of Issues Raised as a Result of the Critique of the Model

Reach - who is the improved Wi-Fi targeted at? Is it local business, visitor or young people using cafes?

Efficacy - Has provision of Wi-Fi in other areas been linked to uptake in these groups and been shown to have been a reason for increased visits/use of refurbish business venues?

Adoption - How many local cafes will offer Wi-Fi access/businesses will update their IT infrastructure or practices?

Implementation - will Wi-Fi installation be to highest speeds, reach key localities, be timely for new start up businesses, done to appropriate quality?

Maintenance - Will funding be available to maintain and upgrade? Will cafe businesses sustain their provision/access?

How a Prioritised Issue/Outcome can be Developed into an Evaluation Question and so Inform Evaluation Design

6.35 In this case, if the critique process confirmed that Wi-Fi was intended to encourage increased use of enabled offices and local business centres for work or meetings then the following questions might be prioritised.

Figure 6.7: Prioritise Questions to Inform Evaluation Design: Example

Outcome of interest
Increased number of existing business locating or new businesses starting in the town centre Locals and visitors meet more in the town centre and use space
Logic
Existing business or new businesses will be encouraged into the town centre to use Wi-Fi in enabled offices and business centres and cafes.
Question (s)
Has there been an increased use/occupation of local business space and meeting venues as a result of Wi-Fi
	Indicators		Measure/method
	Usage of offices/business centre pre and post Wi-Fi installation Reason for increased usage of business centre		Pre/post survey of local business centre use Survey of users to see extent to which Wi-Fi access influences use?
Have new businesses been influenced in their decision to locate in Town Centre due to enhanced Wi-Fi
	Indicators		Measure /methods
	No. of new businesses located in pre and post Wi-Fi No. citing improved Wi-Fi as influential in decision		Routine data on new business start ups/relocations Survey of above re reasons for decisions

6.36 Clearly, Figure 6.7 can be further developed to identify which partners will conduct the research and when it will be completed, so establishing the basics of a monitoring and evaluation plan.

Conclusions: Potential of Theories of Change to Enhance Town Centre Regeneration

6.37 There is scope to encourage further use or training in how to apply these tools or this approach within wider town centre regeneration programmes and among practitioners. This would build evaluative thinking into programmes at the planning stage which should enhance programme planning, delivery and monitoring and evaluation.

6.38 Significant change in the quality of planning and monitoring and will only occur if funders demand that plans and monitoring and evaluation frameworks are informed by results chains. Funders will also need to support the development of skills and capabilities amongst practitioners to respond to these demands.

6.39 A final way in which Theories of Change could be used to improve future TCR programmes is for funders to further consider what theories of change they are trying to test as part of future programme and project commissioning. Funders could consider the specific contexts, likely mechanisms and intended outcomes that they most want to test and commission programmes and projects that specifically develop and test these via robust monitoring and evaluation. For example, if funders think that in rural towns the mechanism of change that leads to regeneration is most likely to be via increased tourists/passing trade they may commission projects that establish specific visitor attractions and thoroughly evaluate that postulated results chain. In large programmes where multiple projects are funded, as in TCRF, projects and evaluations could be grouped and used to test out several promising theories.

6.40 In conclusion, this chapter has illustrated how a Theories of Change approach was applied in this research to enhance existing plans and to strengthen previously developed monitoring and evaluation plans. It also suggests ways in which it might be used to increase learning in future town centre regeneration programmes by enhancing planning, supporting the developing of robust monitoring and evaluating plans and subsequently refining promising town centre regeneration theories of change.