The 5 Step Approach to Evaluation: Designing and Evaluating Interventions to Reduce Reoffending
Updated guidance on how to use the 5 Step approach to design and evaluate criminal justice interventions.
Judging the worth of an intervention
How can the 5-Step Approach help funders to make their decisions?
Assessing an evaluation report
The extent to which an organisation has undertaken these 5-steps could be judged using an objective scoring scale which would standardise the way interventions are judged
A standardised, objective and transparent scoring system could be developed to assess the extent to which these 4 elements have been addressed in the report, namely,
- To what extent is the intervention based on strong and consistent evidence drawn from the results of sound research studies?
- Is there is logic model that shows clear, evidence-based and logical links between each activity and the outcomes?
- Has an independent and robust evaluation been carried out?
- To what extent did the evaluation show a) that the resources (inputs) and been spent on evidence-based activities, that b) the target group were obtained c) that most completed the intervention and d) that the anticipated outcomes for users were achieved?
| Example judging criteria matrix for reducing reoffending interventions | Yes, No or to some extent (Comments) |
Weighting | Score (1-3) |
|---|---|---|---|
| Is there a clear rationale for this intervention? Why was it needed? |
|||
| Did the target group pose a medium to high risk of reoffending? |
|||
| Is intervention content (what they are going to do) described in detail? |
|||
| Is there a comprehensive assessment of published robust studies? |
|||
| Is this evidence clearly embedded into the design of the programme? For example, does the intervention develop individual motivation, skills and competencies? | |||
| Are there also evidence-based or at least logical links between inputs (costs), content/activities and short-, medium- and long-term outcomes? |
|||
| Has an independent evaluation been carried out? |
|||
| Does the evaluation collect appropriate data to test the logic model as described? |
|||
| Impact - was there a control or a comparison group e.g. matched pairs? If so, what was the sample size? |
|||
| Did the evaluation show that resources were spent appropriately on activities with users? |
|||
| Is there evidence that activities were carried out and to a high standard? |
|||
| How many were eligible? What was the throughput? |
|||
| Were outcomes achieved? Was impact measured and has the intervention made a real difference? |
Features and Advantages of a scoring system
- A scoring system could be developed in collaboration with Justice Analytical Services, funders and interventions.
- Judging criteria could be weighted according to the importance of each criteria
- A total score could be worked out for each intervention and assessed - it even provides a basis for making objective and transparent comparisons between interventions.
- There is a precedent for this type of scoring system - 'formal' criminal justice programmes seeking accreditation are assessed using a similar scoring system and Analytical Services use a similar system of criteria to assess bids for research projects
- Advantages and disadvantages of the 5-step approach
Advantages and disadvantages of the 5-step approach
| Advantages | Disadvantages |
|---|---|
|
|