contractSCOTLAND: Review of the service extension pilot

Annex 2: Methods and Evaluation framework

Stakeholder consultation

Fourteen face-to-face meetings with 166 participants were held in a range of locations across Scotland (including meetings that were held for other purposes and incorporated questions on the contactSCOTLAND service). The vast majority of people attending these meetings had not heard of contactSCOTLAND suggesting the need for on-going awareness raising work.

Four online survey questionnaires were prepared for; Deaf BSL users; Deaf networks and organisations; interpreters; and public services. These surveys were developed in consultation with members of the Deaf Sector Partnership and the contactSCOTLAND implementation group. A BSL version of each survey was also available online. The surveys were available for completion during the following periods:

• Surveys for Deaf BSL users, Deaf networks and public services: 11 September 2015 - 14^th October
• Interpreter Survey: 29^th September - 14^th October.

The surveys were issued by the DSP who were responsible for promoting the surveys and issuing reminders. The links to the surveys were made available on the DSP and contactSCOTLAND websites. Letters were sent to all public authorities making them aware of the survey and links were also sent to Deaf networks and organisations asking them both to complete the survey on their own behalf and to make their members and clients aware of the opportunity to provide feedback on the contactSCOTLAND service. The contactSCOTLAND service providers also undertook some promotion of the survey with users of the service.

The surveys were tailored for each stakeholder group, and based on the following themes:

• Background information on the respondent
• Awareness and understanding of contactSCOTLAND
• Experience of using contactSCOTLAND (or why not used)
• Views on different aspects of the service
• What works and what could be improved about contactSCOTLAND.

Who responded:

Deaf community survey

Forty-seven responses were received to the Deaf community survey. A further 9 responses to the section on the experience of using contactSCOTLAND were collected through random call backs to service users and these have been combined with the main survey making a total of 56 responses. The degree of completeness of responses varied. All discussion in the report refers only to those who answered that question.

The vast majority of those who responded were Deaf BSL users. There was a fairly even split between male and female respondents. Of those who provided their gender 25 were female and 21 were male. Responses were received from at least 14 different local authority areas with the most responses (9) from Glasgow City. No responses were received from the Highlands and Islands areas. Just over half of respondents were aged between 30-49. Only six were under 30 or 65 or older.

Public services survey

Twenty six responses were received to the public services survey. These included responses from staff working in local authorities, the health sector, legal and criminal justice bodies, the Public Services Ombudsman, Student Awards Agency Scotland, and Skills Development Scotland. Respondents tended to answer based on their own role and experience and therefore these cannot be considered as organisational responses. The roles that respondents worked in covered policy, equalities, corporate functions, social work, disability support, speech and language therapy, sensory impairment and participation.

Interpreter survey

Thirty-six responses were received to the interpreter survey. Two-thirds of those who responded work on a freelance basis. Those who responded worked in a wide range of geographical areas covering all of Scotland (and in some cases also working UK wide) although a majority worked across the central belt.

Deaf organisations survey

Fourteen responses were received to the Deaf organisations survey. However, not all of these were from Deaf organisations as two were from local authorities, one from an academic and one from a working group member. The coverage of those who responded was predominantly in the central belt with some working across Scotland or further afield.

The original proposal from the DSP had included discussion groups, however, for the reasons discussed above only limited information was generated, usually through existing meetings, with most of those attending not familiar with the contactSCOTLAND service.

Management Information

The management information and additional material gathered for evaluation purposes have provided important contextual information for the review. Information is provided in the form of monthly call reports generated by the system and additional forms filled by interpreters on call completion. Full details of the call information are given at Annex 1.

Other evidence sources

The review is focused on the views and experiences of service users and potential users. It has not included an in-depth review of existing evidence but has taken account of several existing evidence sources in order to understand the context for the review. These include the preliminary evaluation of the NHS 24 pilot, and international evidence and comparisons^[14].

The research questions for the review were informed by a logic model based evaluation framework. The framework sets out the anticipated short, medium and long term outcomes from contactSCOTLAND alongside the inputs, activities and assumptions that underpin the service. It was intended that this could guide not only the review of the service extension pilot but also contribute to future monitoring and evaluation activity.