2. The operation of the Caledonian System
- Overall, the Caledonian System is being delivered in line with its core principles and design. There were, however, some variations in team structure and delivery across local areas.
- Staff expressed a range of views on the impact of variations in team structure. While one view was that the ideal was for staff to be focused solely on the Caledonian System, another was that being able to draw on experience of delivering other related programmes brought a 'broader perspective'.
- The Children's Worker role was described as still developing. One view was that there is a need for greater consistency across areas in what Children's Workers offer, including whether or not they work directly with children and families.
- Delivery of the Men's Programme largely followed the structure provided by the manual. However, there were some examples of deviations from the manual reflecting either local resourcing issues or deliberate decisions by staff to vary content or delivery, primarily to try and better match it to men's perceived needs.
- Delivery of the Women's Service also largely appeared to reflect the aims and design of the System. However, there were again a few examples of local variations, particularly in relation to how Men's Workers worked with both Women's Workers and women themselves, and the service provided to new partners of men on the programme.
- Partnership working with other services was generally described in positive terms by both Caledonian staff and stakeholders from Children and Families and Women's Aid. However, staff did identify issues around differences in understandings of domestic abuse, particularly among Sheriffs, and information-sharing by police in some areas.
This chapter summarises findings on how the Caledonian System has operated in practice across the five Scottish 'hubs', in order to assess whether it has been delivered as planned. It also explores wider issues around operational delivery which may have implications both for the future development of the Caledonian System and for other services that work with families at risk of domestic abuse, including:
- Variations in delivery across Hubs, and any potential learning from this, and
- How the Caledonian System works with other partners (including managing 'system-generated' risk). 
The findings are based primarily on interviews with staff across the five hubs. As discussed in Chapter 1, it was not possible within the evaluation timescale and resources to speak to staff in every local authority area involved in delivering Caledonian. As such, while interviews reflected a range of experiences, it is possible that there are further variations in practice or opinion that are not fully captured here.
2.2 How is the Caledonian System being delivered in Scotland?
2.2.1 Staff roles and responsibilities
In order to deliver the Caledonian System, a number of specific roles are required, including:
- A System Manager (responsible for overall managerial oversight of staffing, communication, practical arrangements for running the services and monitoring and evaluation)
- A Delivery Manager (who may in practice also be the System Manager), who is responsible for day-to-day management of delivery, including monitoring the integrity of delivery, providing supervision and support to staff, and implementing any nationally agreed changes
- Programme assessors, who assess whether men are suitable for the programme, and who may also be Case Managers and/or Group Workers
- Men's Case Managers, who deliver the Pre-group and Maintenance sessions to men as well as supervising their order or licence throughout
- Men's Group Workers, who deliver the Group stage
- Women's Service Workers, who are the main point of contact for women partners as well as playing a key role in risk assessment and management
- Children's Service Workers, whose role is to ensure that children's rights are upheld and their needs met. 
All the teams interviewed for the evaluation had staff in each of the roles above. However, there were some variations both within and across Hubs in terms of the precise team structure. In particular, teams varied in terms of:
- Whether or not they were only working on the Caledonian System or whether staff also had other roles. The City of Edinburgh has a dedicated Caledonian Team, working exclusively on the Caledonian System (although the Edinburgh team also work with additional case managers based in local criminal justice teams, who work on Caledonian alongside other work). In other areas, at least some members of the team had multiple roles - for example, Men's Workers either worked on general criminal justice social work cases, or delivered other offender intervention schemes with some similarities to Caledonian, such as the Moving Forward, Making Changes programme for adult male sexual offenders and the Constructs programme for adult male persistent offenders. The Women's and Children's Workers we interviewed also varied in whether they only worked on Caledonian, or whether they also did domestic abuse work with women whose partners were not on the Caledonian Men's Programme.
- Whether the Men's Workers carried out assessment, case management and group work, or were only involved in one or two of these roles. In some areas, Men's Workers did all three, but in others case management and group work were delivered by different members of staff.
- What tasks the Children's Worker undertakes and whether or not this role was combined with a Women's Worker role.
While all these variations in staff structure are allowed for in the programme manuals, it is nonetheless worth reflecting on the potential impacts of these differences in order to inform future delivery. In this context, it is important to note that staff views on the impact of differences in team structure varied. One view - expressed both by staff in teams where this was the case and by staff with experience of other models - was that the ideal was for the whole team to be focused solely on the Caledonian System. Staff who were able to focus exclusively on Caledonian System work felt that this strengthened their expertise in delivering the programme and helped to support stronger relationships with key partner agencies (since they were in more regular contact with them). However, where staff teams were working on other offender interventions in addition to Caledonian, this was also seen as having benefits in terms of being able to draw on other programmes to bring a 'broader perspective' to their practice.
Being able to work on both case management and group work was similarly seen as building greater expertise in comparison with working on different elements separately - delivering all stages of the programme was described as giving Men's Workers ' a much better understanding of the overall picture of the Caledonian System' so that ' you know where your work is going to'.
A related issue was whether or not teams were co-located (something that varied both between and within Hubs), with Delivery Managers, Men's, Women's and Children's Workers all based together. Co-location was seen as fostering a 'strong team identity', providing support for (particularly new) team members, making it easier to implement regular client liaison meetings between Men's, Women's and Children's Workers, and facilitating more effective information sharing between Men's, Women's and Children's Workers in general. Staff spoke about the ' symbiotic relationship' between Men's and Women's Workers and emphasised the benefits of co-location in supporting this joint-working. Discussions with Women's Workers provided a continual check on men's accounts of their own behaviour and a better awareness of the whole family dynamic.
The role of the Children's Worker within the Caledonian System was described as ' probably the one (role) that is developing the most'. However, one view was that there is currently too much variation in what children are being offered by Caledonian across areas, reflecting differences in both local resourcing of this role and local practice decisions. In particular, there is variation between areas in whether or not Children's Workers ever work directly with children and families themselves to support them in addressing the consequences of domestic abuse, or whether they always refer children to other services to meet their needs. Although Children's Workers all referred children to other services on occasion, in some areas they also had a small caseload of families with whom they worked directly. There is also variation in whether Children's Workers work with men in the programme, either one-to-one or through delivering elements of the 'Children and fathering' module of the group work sessions.
These differences in the tasks Children's Workers engage in between areas have resource implications. Where Children's Workers were not working directly with families, staff did not report any particular workload challenges, or issues around combining women's and Children's Worker roles. However, where Children's Workers were supporting families directly, they reported that they were not always able to accept every case they were asked to take on, either because they did not have sufficient time, or because of conflicts around the same Children's Worker working with multiple members of the same family (for example, a child and a father). Staff in areas where the children's work and women's work roles were combined also commented that they would not currently be able to resource this kind of direct support to families. The implications of variations in the Children's Worker role are discussed further in section 2.6 and Chapter 6.
2.2.1 Consistency of delivery and use of the manuals
Delivery of the Men's Programme
As described in Chapter 1, the Caledonian System is supported by a series of detailed manuals (see ' References'). However, although there are manuals for each service within Caledonian, the Men's Programme is more highly structured than either the Women's or Children's Service. The Men's Programme manual includes detailed plans for each of 14 pre-group activities and for 26 group work sessions covering themed six modules (lifelong change, responsibility for and to self, relationships, sexual respect, men and women, and children and fathering).
The Caledonian Men's Programme manual is almost 400 pages long. As such, it was not possible for the evaluation to assess whether every element of the programme was being delivered as specified. However, overall the accounts of both staff and men who had participated indicated that the main components (the general principles of the programme; the structure, length and duration of Pre-group, Group and Maintenance sessions; the topics covered; and the core exercises and techniques included in the manual) were being delivered in line with the design of the System.
Men's Workers interviewed for the evaluation were broadly very positive about the structure provided by the manual. Although some reported using the manual more flexibly over time as they became more familiar with it, it remained a regular point of reference throughout delivery of the Men's Programme
The programme is so intense and there is so much to it, I don't think you could remember every single aspect without having the manual.
Where staff identified deviations from the manual, these fell into two categories: those that reflected staffing and resourcing issues; and those that reflected deliberate decisions by staff to adapt the content or delivery with the aim of better meeting participants' needs. Examples of variations related to resourcing issues were:
- Cases where Group Work sessions had to be delivered by two women staff (instead of reflecting the gender-balanced co-facilitation recommended in the manual). This was attributed to difficulties recruiting suitable male staff.
- Periods of time where men did not appear to be progressing through Pre-group preparation as quickly as they should, as a result of staffing issues (this had been resolved by the time of the evaluation), and
- Periods of time where meetings between Men's and Women's Workers were not happening regularly (which in part appeared to reflect issues around co-location, discussed above).
Examples of deliberate decisions by staff to vary content or delivery included:
- Dropping or amending exercises that were viewed as unhelpful or potentially risky. Examples included:
- 'Abuse accounts' - the manual states that all men must present an account of their abusive behaviour to the rest of the group. This exercise is presented as a core means of holding men to account for their abuse. However, the value of abuse accounts was contested among staff. While some viewed them as useful, one area reported that they had stopped including them in the group work as they felt they were 'shame inducing' and unnecessary, since the whole programme was geared towards men talking about and accounting for their behaviour. Men themselves also expressed mixed views on the abuse accounts where they discussed them. One view was that it could be very daunting, particularly for men that are new to the group. However, in spite of this there was evidence that it could help men to acknowledge their behaviour and move on from it.
It was very emotional at times, very tough actually…. In a way I'm actually kind of glad I did that because it couldn't be any worse, I don't have anything worse than that, that I've ever done… I can look at it and say "actually I'm a much better person than that, I will never do that again".
(Men's Programme participant D)
- Elements of role playing - Staff highlighted that where role plays involved specific (fictional) scenarios between couples, this could increase risk to women, as men might mistakenly think it was based on discussions with their partner. This concern was reflected in comments from a woman participant that her partner had in fact believed that fictional examples discussed in a group had been reported by her, resulting in 'intense' conversations. One area also reported dropping a section where men were asked to role play getting angry, as they found this counter-productive.
- Reordering sessions or content. For example, it was suggested that one of the exercises that occurs in the first session of the 'children and fathering' module could be quite distressing and might be more appropriate to introduce later in the module. One area also reported moving some of the paperwork from the start of the Pre-group to later in the Pre-group stage, since they felt that the volume of paperwork could hamper building an effective 'therapeutic relationship' with men - something central to the Caledonian Men's Programme. 
- Adding to exercises to get more out of them. For example, the manual recommends using the metaphor of an 'iceberg' to identify and discuss the behaviour, feelings, thoughts, and beliefs that have led them to be on the programme. However, workers felt this exercise was improved by using a 'split iceberg' that encouraged men to think not only about their abusive behaviour, but also about where they want to get to in the future.
- Adapting exercises to meet the specific needs of men in terms of cognitive ability, personality or specific circumstances - for example, by modifying the language or examples used.
Drawing on their experiences of delivering the manual, Men's Workers had various suggestions for improvement, which are outlined at the end of this Chapter.
Delivery of the Women's Service
Although there are detailed manuals and materials for the Women's Service (again totalling around 400 pages), delivery is much more loosely structured than the Men's Programme (necessarily so, given that participation is voluntary). While Women's Workers reported finding the Women's Service manual useful, it was suggested that they used them less often once they were experienced in delivering the programme, perhaps just dipping in to refresh or re-focus particular activities.
Given that a flexible, 'woman-centred' approach is built in to the Women's Service, assessing consistency of delivery is arguably less straightforward than for the Men's Programme. However, again the accounts of staff and women interviewed for the evaluation confirmed the delivery of key elements, including: regular, long-term support, mirroring the two-year time frame of the Men's Programme; a strong focus on safety planning and management of risk (both via direct work with women and liaison with the Men's Programme); and a commitment to empowering women, improving both their general confidence and emotional wellbeing and their specific understanding of domestic abuse and how it affects them. Staff interviews identified three exceptions to this, however.
1. Whether or not men's Case Managers were always attending regular review visits with women. The Women's Service manual states that men's Case Managers should meet with women partners/ex-partners at three key reviews to inform their assessment of the men's risk. However, Women's Workers indicated some differences in local practice and attitudes to case managers having contact with women directly (or in some cases even working closely with Women's Workers), reporting their belief that some Men's Workers felt "there was something sneaky about going to see the woman" given the Men's Workers' relationship with the man. Given the central importance of women's views to informing assessments of men's risk, this may be something that needs reinforcing across all Caledonian teams.
2. Whether or not the Women's Service was being consistently offered to new partners as well as those women who were victims of the 'index offence'. The manual states that the Women's Service should be offered to both those women who experienced the offence which is the basis of the man's referral to the Men's Programme, and any new partners associated with that man. However, staff reported that this had not happened consistently across all Hub areas - something they attributed to resourcing issues (insufficient Women's Workers).
There were mixed views among staff on how the Caledonian System should approach working with new partners. On the one hand, the current process, whereby workers are meant to visit all potential new partners, was felt to be too broad, in that it could involve offering support to someone who has only been on a couple of dates with the man and is no longer seeing them. On the other, offering no support to new partners was seen as potentially failing women who may be at greater risk in some cases than the partner who was the victim of the index offence:
Because quite often the index offence person … after he's been lifted for that offence, they may never be together again, so you're offering this service for a two-year period. This woman's like "I've not set eyes on him for six months and I don't intend to." So there's only so much that you can do. But if he's in a new relationship and he's living in the same home, that woman's more at risk.
3. Variations in the length of support offered to women whose partner is not given an order. It was suggested that the manual's advice around the time-limited support to be offered to women whose partner is not in the end given an order to attend Caledonian may be too narrow - one view was that if Women's Workers had identified a woman whose safety was at risk at the assessment stage, they would continue to see her rather than leave her without support:
Women's Worker 1: Sometimes we're the only person that woman has ever had contact with, you know …
Women's Worker 2: Yes, that's what I mean and you can't then just go "he never got the order so, bye."
Other variations in delivery that may be worth reviewing included:
- Assessment tools used with women - Women's Workers questioned whether the assessment tools were always appropriate and reported that they were not always able to complete them at all the points recommended by the manual. The System manuals include a 37 item 'women's behaviour checklist' and a 55 item 'partner checklist', both of which focus on detailed aspects of abusive behaviour by men. Women's Workers acknowledged that these checklists could be useful in highlighting the multiple dimensions of abuse to women. However, they also felt that asking women detailed questions about men's abusive behaviour immediately after they have experienced abuse could be unhelpful, while asking the same questions at the end of the programme, when they may have moved on from the relationship altogether, could be irrelevant at best or re-traumatising at worst.
I know that we all struggle with that type of paperwork … If you're meeting a woman and she's extremely emotional, she's really upset, there is kids running around, we're maybe even in a public place because sometimes it's not safe to go to her house. The last thing we need to do is whip out a piece of paper and start asking her has he ever sexually assaulted you? Has he ever physically restrained you?
In addition, it was suggested that the assessment tools were too focused on men's behaviour rather than women's needs. In one area, Women's Workers had responded by incorporating the CAADA-Risk Identification Checklist ( RIC)  , which they felt was better able to identify women's needs. These issues around the perceived appropriateness of the Caledonian women's checklists are reflected in gaps in the monitoring data, discussed in Chapters 1 and 4.
- Running women's groups alongside one-to-one support - Aberdeen Women's Workers ran both informal and more structured groups for the women they were supporting (a weekly drop-in café and a 12-week domestic abuse group programme for women). Interviews with women for this evaluation suggest this is something they found useful in terms of peer support.
Delivery of the Children's Service
Variations in delivery of the Children's Service between areas - and differences in staff views on this - have already been discussed. The revised Children's manual puts the focus on all staff (not just Children's Workers) sharing responsibility for ensuring children's needs are recognised and addressed, and on accessing universal services where possible. However, there are clearly still differences between areas in how the Caledonian's support for children is implemented - in particular, whether Children's Workers work with children (and their families) directly.
2.4 How does the Caledonian System work with other services?
Effective information sharing and joint working is vital to managing men's risk and maximising women and children's safety. Interviews with staff and stakeholders explored perceived facilitators and barriers to effective interagency working, with a specific focus on how system-generated risks are managed.
Caledonian staff report working with a very wide range of services. Children and Families Social Work, Police Scotland, and the Court service were mentioned as particularly key, but staff also worked with housing, health services, drug and alcohol support services, Victim Support, Women's Aid and a whole range of other voluntary and statutory services. Ease of inter-agency working varied across both agencies and areas. However, building strong relationships and having clear information sharing protocols in place were, unsurprisingly, viewed as key. Additional factors influencing partnership working included:
- Partner agency awareness and views of Caledonian - low awareness was linked by Caledonian staff with experiencing more challenges around inter-agency working, while high awareness and credibility was felt to be associated with easier access to information from other agencies. Staff noted that ' resistance' from other agencies - who may initially be unsure about the Men's Programme in particular - was usually overcome with time and joint working. This was supported by the accounts of stakeholders from Women's Aid and Children and Families Social Work, who regarded the Caledonian System very highly. Stakeholders interviewed for this evaluation acknowledged that information sharing between themselves and the Caledonian System had in some cases been difficult at the start, but felt that this had improved over time as a clearer sense of roles emerged. Working together on multi-agency forums, like the Multi Agency Risk Assessment Conference ( MARAC), had also helped cement inter-agency relationships.
- Shared understandings of domestic abuse - Differences in understandings of, or approaches to dealing with domestic abuse across services could pose a challenge in joint working - for example, there was a perception among Caledonian staff that Children and Families social workers sometimes held women wholly responsible for keeping their children safe, ignoring the man's role. However, staff also felt the Caledonian System was playing an important role in disseminating best practice among wider services around how to support women and children and how to hold men to account.
This view was confirmed by the accounts of stakeholders from Women's Aid, who reported the role of training from Caledonian staff in helping them understand the Caledonian System and the benefits of working with both men and women. Stakeholders also commented on the fact that, in working with men to address their problems, the Caledonian System is doing something different to other services, since other services may place more onus on women to protect themselves and their children. The Caledonian approach to safety planning was also reported by stakeholders to have influenced the delivery of safety planning by other services.
- Sharing systems and paperwork - there was variation between areas in whether or not Caledonian staff had access to social work databases. Where they did have access, it was commented that this supported both safer working for Caledonian staff (who could assess risks before going out on visits) and helped avoid re-traumatising women by reducing the need to ask for information already provided to other professionals. The area that was using the CAADA RIC assessment form with women reported that this had also facilitated partnership working, since other partner organisations working with women used the same form.
- Organisational change - some (though not all) areas reported some difficulties around joint working with the police, which they attributed to the transition to Police Scotland. In one area, it was reported that the quality and quantity of information sharing had decreased - for example, their joint protocols stated that Caledonian staff should always be notified if there had been a police call-out involving a participant, but this was not always happening. In other areas, however, previously good relationships with individual police officers or sections with a domestic abuse remit had been maintained and did not appear to have been impacted by the transition to Police Scotland.
- The role of courts - Caledonian staff expressed considerable frustration with the role courts - particularly Sheriffs - play in domestic abuse cases.
First, staff reported that there had been men who they believed would have benefited from the Men's Programme, and who had been assessed by them as suitable, but whom Sheriffs had not in the end referred as part of their Order.
Second, the sentences courts hand down were sometimes seen by Caledonian staff as too lenient. Where this was believed to be the case, it was viewed as having a major impact, not only for women themselves but for the ability of the Caledonian System and others to work with women effectively. Staff commented that where they had supported women through the court process but then both staff and women felt the sentence had only been 'a slap on the wrists', 'It makes them (women) not want to report again … if she is failed from the top it all sort of falls apart again.'
Third, Caledonian staff felt that Sheriffs' interpretations of domestic abuse varied. This was seen by staff as a key factor explaining why sentences (in staff's view) were sometimes too lenient or were inconsistent between Sheriffs. They felt that it was difficult to influence Sheriff's understanding of domestic abuse, as Caledonian staff reported they had very limited access to Sheriffs in comparison with other groups of professionals they worked with, who they more often came into contact with through interagency forums and training. 
It is important to note (as discussed in 1.5.6, above) that we were not able to speak to any Sheriffs for this evaluation: they may have had a different perspective on court processes for domestic abuse cases.
2.4.1 Managing system generated risk
Caledonian staff were clear that system-generated risk (ways in which a service might increase rather than reduce risk to participants - see footnote 9, above) was a continual challenge in domestic abuse work, but reported that they took active steps to manage this. Examples included:
- Asking women for consent when they want to share information, while also being up front about when they might need to do so even without consent
- Considering carefully what source of information to quote to men on the programme when discussing their abusive behaviour - for example, favouring police or court reports over accounts of abuse given directly by women or children
- Being very careful about what information is stored on men's, women's and children's files (and keeping these separate).
There were, nonetheless, examples where staff believed 'system-generated risk' had resulted in adverse outcomes for women - for example, cases where information had been disclosed to men, either at their partner's request, or as a result of discussions between men in a group, and staff believed these disclosures had triggered subsequent assaults. However, in general staff believed that system generated risks were as well-managed as they could be within Caledonian. At the same time, they felt there were still some problems with the 'bigger system' of services involved in domestic abuse cases - for example, courts disclosing a woman's address for bail conditions, or other professionals inappropriately sharing Caledonian reports about a male participant with their partner.
2.5 Suggestions for development or improvement
Participants' suggestions for improvement to the general design and operation of the Caledonian System itself related to: the manual and supporting materials; the timing of the Women's Service; training; and staff roles. Staff interviews also raise a number of issues around changes that might be required to the wider 'system' of services within which Caledonian operates to maximise its ability to secure positive outcomes.
- Changes to the manuals: While seen as comprehensive and well-thought through, there was a consensus that the manuals - particularly the Men's Programme manual - needed shortening, simplifying, and modernising to reflect both changes in the world and in how domestic abuse manifests itself in relationships, particularly around the growth of social media. Staff also suggested the Men's Programme manual should cover men's own experience of trauma, alcohol, and cultural differences in attitudes to domestic abuse in more detail.
One staff view was that rather than a one-off update, supporting materials in particular should be updated or added to on a rolling basis. There was a strong desire from staff to be involved in updating the manuals - indeed, it was commented that staff had already identified numerous specific exercises that could be improved.
- Changes to assessments and psychometric tests: in addition to questions about the appropriateness of assessment materials built into the System for use with women (discussed above), Men's Workers also debated the value of the some of the psychometric tests used with men during the programme. The Caledonian System includes a number of psychometric tests and assessments. While one view was that some of these - particularly MCMI III  - were helpful in enabling staff to identify men with particular personality disorders or traits (and in informing how they subsequently work with them), staff commented that some of the psychometrics "are not really used in practice to shape our thinking about the men". This issue is discussed further in Chapter 6, in the context of future evaluation.
- Changes to the length of time the Women's Service is held open for: Although the length of the Women's Service - the fact it offers a full two years of support - was viewed as a key strength, there was also a view that in some cases it was too long. Although the Women's Service is voluntary, Women's Workers do attempt to keep in touch periodically throughout the period the men is on the programme, unless a woman specifically requests them not to. It was suggested that where women move on from a relationship earlier on in that period, it may not always be appropriate (and could even be re-traumatising) to continue trying to initiate contact.
- Training: Staff and stakeholders identified various potential improvements to training:
- Increasing the length - One staff view was that the training for Caledonian was not long enough, given the length and complexity of the manuals.
- Adding refresher training - Men's and Women's Workers suggested that there may be a need for more refresher training, particularly where there is a big gap between Case Managers receiving their initial training and actually starting delivery (an 18-month gap was reported for one area).
- Greater involvement of practitioners in delivering training - There was a desire among staff for current practitioners to be more involved in the delivery of training, so that they can more effectively share learning from their experiences of delivering Caledonian.
- Staff forums - Staff suggested that local practitioners' forums should be reintroduced in areas where these were no longer happening, to support continual reflection and sharing of learning.
- More training for stakeholders - Stakeholders from outside the Caledonian System commented on how useful they had found training delivered to them by Caledonian staff, and indicated that they would welcome more.
- Staff roles: Differences in local practice and opinion on the role of the Children's Worker, discussed above, suggest there may be a need to consider whether further clarity or guidance is needed - particularly in relation to whether or not Children's Workers should be working directly with children and families affected by abuse. As noted above, any changes to what is required from this role may also have resource implications that need to be considered.
- Changes to the wider 'system': While not a specific focus of this evaluation, comments around different understandings of, and levels of training around, domestic abuse among other services highlight the need to consider the ways in which the wider 'system' could respond to domestic abuse more effectively. As well as issues around variations in understandings of domestic abuse, particularly among those involved in the Court system, Caledonian workers also felt that the whole system of 'plea bargaining' works against 'holding men to account' for their behaviour, since they often arrive on the Men's Programme already having had several charges struck from their record. As discussed, unfortunately we were not able to ascertain the views of the judiciary on these issues as part of this evaluation.
There is a problem
Thanks for your feedback