We are testing a new beta website for gov.scot go to new site

The Evaluation of Children's Hearings in Scotland: Volume 3 - Children in Focus

Annex 1

The Children's Hearings System

METHOD

Postal questionnaires

At Stage 1 all Reporters completed one of 3 main questionnaires adapted according to the age of the child. Three age groups were chosen: under 5 years; 5 to 15 years; and 16 years or older. All 3 questionnaires consisted of a core set of questions on biographical information, including the child's family background, schooling and living circumstances. Reporters rely on information from a range of sources including, for example, teachers and social workers. The range of information requested by and provided to Reporters varied considerably from child to child. Reporters when responding to referrals do not always request information from other key professionals.

Current concerns in relation to each child, and positive factors as well as adversities in each child's life were explored. This was by way of a series of mainly open-ended questions asking Reporters to indicate their view of any concerns and positives in the lives of the children. Some specific questions examining factors that appear to offer protection to children in adverse circumstances were asked of Reporters, including direct questions in relation to health and disability.

Reporters provided a narrative answer outlining their views of concerns and adversities. Frequently the concerns and adversities recorded were similar and therefore a decision was taken to analyse these questions together. The first 4 concerns and adversities cited by Reporters were coded for the purposes of analysis with categories created on a post-hoc basis after sufficient returns had been made to the research team. The coding of 4 concerns was generally sufficient to address all the concerns and adversities expressed by Reporters in relation to each of the cohort children.

As with the question in relation to concerns, provision was made for Reporters to give a narrative answer in relation to positive factors in the children's lives. The first 3 positive features cited by Reporters were coded for the purposes of analysis with categories created on a post-hoc basis. This was generally sufficient to address all the positives expressed by Reporters in relation to each of the children.

Information on the profile of the child in the Children's Hearings system, to include sources of referral and the position of any siblings in the system, followed. A further section asked for information on decisions taken on the child, including those of any hearing where appropriate. The remainder of the questionnaire sought to capture the quality of childhood experience across family and educational dimensions as understood by the Reporter when reaching his/her decisions. The legal grounds of referral where relevant and final official decisions after 1 February 1995 were also recorded for each child.

If, however, a Reporter in the normal course of carrying out his/her duties at Stage 1 requested information from either a SWD or school, a brief questionnaire designed by the research team specifically for social workers and teachers was enclosed. The Reporter sent this to the SWD/school on behalf of the research team although completed forms were returned directly to Edinburgh University.

Further information was gathered from Reporters in February 1996 and in February 1997. This allowed assessment of changes and developments in the children's lives over time and of their status in the hearings system.

Throughout the study a coded numerical system was used to ensure the confidentiality of all cases. If the Reporter dealt with siblings during the sampling period a questionnaire was completed separately for each sibling.

Consent

Parents and young people aged 16 years or older were given information about the study through a letter sent out on behalf of the research team by Reporters. They were asked to contact the research team if they did not wish their child, or in the case of 16 year olds themselves, to be involved. Opting-out was by means of a simple tick-box reply form with a pre-paid reply envelope included to aid this process.

There were some 353 parents and/or young people who did not wish to participate in the study, representing just under one-fifth (18%) of the initial sample of children identified. A small number of parents and young people were contacted by letter to explore their decision and other parents or young people made direct contact with the research team to explain their reasons for non-participation.

The most common reason appeared to be a desire to put the past behind them and not re-open old wounds, earlier experiences or concerns. A minority of parents feared their involvement in the study would influence the outcomes for their children, others regarding the referral circumstances as minor and no longer a difficulty. Several parents thought the project would be an invasion of their privacy, creating more stress and pressure on them as a family.

Regional Reporter response rates

After 1 February 1995, Reporters completed 1,155 returns. The majority of returns were from the Strathclyde Regional Reporter's Department (61%), followed by Grampian (12%) and Tayside (11%). This was not unexpected, as 48% of all children referred to the Reporter in 1995 were in Strathclyde Region. Highland Regional Reporter's Department did not take part and Lothian Regional Reporter's Department, despite having 13% of all children referred in 1995, was only able to provide information on 22 children (0.02%).

At 1 February 1996, 619 returns were completed by Reporters, reflecting a broadly equal distribution of returns across all 3 age ranges in proportion to the original age distribution.

At 1 February 1997, 510 returns were completed by Reporters with a higher percentage of returns for children in the 5 to 15 years age grouping (45%), compared to those under 5 years (39%) or 16-plus years (38%).

Table A1.1 below summarises the numbers of returns at all 3 stages of the study, expressed as a percentage of the original sample.

Table A1.1
Numbers of case details returned at all stages of the study

Stage 1
February 1995
Returns

Stage 2
February 1996
Returns

Stage 3
February 1997
Returns

Age Group

No.

No.

%

No.

%
< 5 years1146254%4741%
5-15 years96551954%43345%
16+ years763850%3039%
Total115561954%51044%
  • The overall return rate by Reporters of the questionnaire at February 1995 approached three-quarters (71%) 3, which is a high rate for a postal survey.
  • The overall return rate at February 1996 was 54%.
  • The overall return rate by Reporters of the questionnaire at February 1997 was 44%.

Information was not available on all children over all 3 time spans, as is summarised below:

  • There are 418 cases which are present at all 3 stages.
  • There are 619 cases in Stage 2 which are present in Stage 1.
  • There are 510 cases present in Stage 3 and Stage 1.

Social work and education responses were disappointing, although in some cases seeking a social work or education report would have been deemed inappropriate by the Reporter. Furthermore, not all social work and education departments in Scotland agreed to participate in the study. In total, 235 social work questionnaires and 298 school questionnaires were returned with both sets of information available on 87 children.

The attrition rate increased with the passage of time. This was partly related to the impact of major organisational change within the Reporters' service affecting returns in 1996 and 1997. A number of Reporters advised the research team of lost or missing files that precluded a return being made, and in one case a Reporter directly informed us of an inability to complete any of their allocated returns due to workload demands.

Reporters were more able to provide information on children who were subject to a supervision requirement and for whom there had been some contact between the Reporters service and the child during the study.

Government statistics

Since the inception of the Children's Hearings system in 1971, The Scottish Office has collected annual statistical information on referrals based on SWS21 returns from the offices of all Regional Reporters. The SWS21 return is a pre-coded form completed by the Reporter or administrator. This system for the collection of information is based on referrals.

Each time a child is referred to the Reporter a statistical return (SWS21 return) is completed and sent to Social Work Services Group at The Scottish Office/Scottish Executive, after a final decision has been taken in relation to that referral. A child may be referred to the Reporter on multiple occasions in a single year, by separate sources and on different grounds with offence and non-offence grounds always reported on separate returns.

The general criteria for gathering information are highly specific. If offence and non-offence grounds are reported simultaneously to the Reporter, separate returns are completed for each type of referral even when they relate to the same child. Some children may be referred by several different sources, for example the police and the school. In these cases once again separate returns are completed. Furthermore a child may be referred to a Reporter on more than one occasion during any year, resulting in further SWS21 returns on the same child. SWS21 statistics do not allow analysis of multiple-agency referral.

The SWS21 returns offer detailed information on an individual referral and the related outcome. They do not explain the relationship between a series of referrals and an individual child. Thus the same child may have been referred on 20 different occasions in one year, each of which will have been recorded on a separate SWS21 form. As a result, there is no one to one correspondence between referrals, children and disposals.

Every child who is referred to the Reporter is allocated a unique reference number that s/he retains throughout his/her childhood. SWS21 returns on all the children in the sample were requested. These were used to establish which children had a history in the Children's Hearings system prior to the commencement of the study, and their progress in the system.

It is important to understand SWS21 returns do not provide biographical information on the children; instead they record why a child is in the system and the legal decisions which have been reached. It is also important to note the SWS21 return was revised in 1987, altering slightly the type of information required from Reporters. This brought no appreciable consequences for our analysis.

A small number of children were excluded from certain SWS21 analyses. Why? Some surprising errors in the information have come to light, for example one child who was recorded as both a male and a female and another who was aged 83 years! A small minority of children were completely missing from official statistics although clearly had ongoing and, in some cases prior, involvement in the hearings system.

The Scottish Office issue annually a statistical bulletin based on SWS21 returns ( Statistical Bulletin on Referrals of Children to Reporters and Children's Hearings) reporting on general patterns of referrals to Reporters, decisions of hearings, numbers and types of supervision requirements providing comparisons over ten year time periods. As the official statistics are based on referrals, analysis in relation to individual children is complex.

The responsibility for gathering statistical information has been transferred to the Scottish Children's Reporter Administration. This may mean changes to the way information is gathered and presented in the future.

Interviews

Interviews were carried out at home with 26 parents or young people included in the study. We would have liked to meet more families, but this number represents, out of over 300 requests, the parents or young people who gave their written consent to being interviewed. The interview asked about the child's current position as well as the parents' or young person's experience of the Children's Hearings system.

We were grateful to the parents and young people who agreed to participate in this part of the study. As a group, however, they are atypical because they agreed to be seen. Yet their views are still valid, each in their own right, and raise important questions about the philosophy, policies, and practices of the Children's Hearings system. Whether some of the confusions, complaints and benefits of the system they identify would be raised by parents and young people who did not agree to an interview is a question that cannot be answered. While extreme caution is needed about generalising from what these parents and young people tell us, their voice remains important.

Observations

Finding out about the children and their families might seem simple enough. Like any attempt to gather, classify and analyse information, this study faced 2 main sets of difficulties. The first set of difficulties was technical.

The decision to gather information during 2 specific weeks in February 1995 meant that a representative sample of children involved in the hearings system could not be guaranteed. This census approach produced a national snapshot of a sample of children and families involved at a particular point in time. Two weeks in the year 1995 were selected as most likely to produce a good cross-section of cases. Where appropriate comparisons have been made throughout the Report between the children in the sample and national statistics for all children in Scotland referred in the same year, 1995. The sample proved mainly representative when compared with national statistics.

Second, who provides information can be critical in ensuring reliability. In this study Reporters were asked to provide information on the children and their families on the grounds that Reporters could be expected to know about the children in exercising their professional responsibility to determine whether a child may be in need of compulsory measures of care. When Reporters lacked information, this in turn was reflected in their answers to our questionnaire. The quality and type of information provided by Reporters could only ever be as good and as accurate as their information systems and record keeping permitted.

Third, the budget set for the study ruled out personal interviews with Reporters or in any volume parents and children. Reporters acted as professional respondents in this study. They brought an expertise that was highly valuable. It must be faced, however, that with growing economic pressures in public institutions, professionals have finite resources at their disposal to use for meeting research objectives. Initially Reporters responded exceptionally well (a return rate of 71%), better than most studies involving professional respondents. Over time, the demands of other work and the need to respond over successive years proved a taxing requirement on Reporters and the return rate dropped to 44% over 3 years, still reasonable by many standards.

Fourth, a high number of families opted out of the study, nearly one-fifth.

This study was concerned with a sensitive subject, namely the backgrounds and personal circumstances of children in need and sometimes also in trouble. The population for study represented a specific group of children where difficulties had already been identified and brought to public and official attention. Willingness for involvement in a study of this kind is likely to be affected by the fears and possible stigma that referral may bring to parents and young people 16 and over. It needs also to be remembered that parents made the decisions on behalf of their children, sometimes based on an apparent wish to close the door on events which marked a difficult time in their and their children's lives.

When families were asked to give positive written consent, rather than simply opting out, the loss rate was dramatic. It took over 300 requests to produce 30 written consents. It makes sense that parents may not want to be interviewed because of circumstances that they may feel cast a mark against them or the child. Indeed, some of the parents who did agree to see us said as much. The difficulty arises in interpreting the results of interviews when it is the exception rather than the norm to agree to be seen. Who do these families represent? Why did they agree when so many refused? Written positive consent, so important for ethical reasons, will in studies like this one, dealing with sensitive subjects, result in unrepresentative samples.

In future, studies might seek to find ways of identifying such specific groups within normative populations to improve the rate of consent.

Fifth, SWS21 returns give an official position of the child in the Children's Hearings system. They are only as reliable a source of information as the accuracy and completeness of the information recorded.

The second set of difficulties was conceptual and dynamic. The Children's Hearings is a most complex system for all its apparent simplicity. Cases follow different routes against different time-scales and may be 'open' for more than one reason or because of several referrals for the same reason. In some cases a decision about the course that a Reporter intends to follow may be resolved in a week or may take months. In the course of an investigation the Reporter may obtain information from the SWD, the child's school, the police and other professionals depending on whether this is considered material to any future course of action. In practice the Reporter uses his/her discretion in deciding what information is likely to be relevant in an individual case. This means the type and amount of information available on different children may vary greatly.

When Reporters were asked to describe the circumstances of the children one and 2 years later, they could only tell us information as it applied to the last point of contact between the system and the child. There was no way of knowing whether their information was a complete account of the child's circumstances.

It is also possible that a child's position in the system and her/his circumstances may be the same at the outset of the study and one year later. Yet between these 2 snapshots many changes may have occurred, if only to come full circle. Mapping all changes between follow-ups rather than at the time of follow-up was beyond the scope of this study.

Referrals in the system figure, at least in a technical sense, as discrete events. There was a problem in determining in some cases which of several referrals (perhaps all applying to the same event) to use for the purpose of gathering information on the child. Reporters used their judgement much as they would do in carrying out their official function.

One of the limitations of the cohort study in relation to outcomes was the lack of information on the nature of any interventions with the child and her/his family; the content of any supervision requirements (beyond knowing if this involved a child living away from home in a foster care or residential placement); or the availability of specific resources to a child and her/his family. These factors were beyond the remit and resources of the study, although remain important for any future enquiry.

This study depended on professional respondents, Reporters. The findings are based on what the Reporter knew and acted upon. To maximise information the individuals themselves remain the single most important source of comprehensive information. Official statistical data sets, however, proved a good foundation for retrospective longitudinal analysis.

Longitudinal studies are expensive. Policy makers, practitioners and researchers need to make the investment that such studies demand if longer-term outcomes and patterns are to be compared. There is a need for some studies of this kind in Scotland. This study has produced good baseline data using the methods adopted, and reasonable information one year on. Two years on, the system knows surprisingly little about the children, mainly because numbers of them have left the system. This finding may represent an achievement in policy as the intention of the Children's Hearings system has always been to bring children before it as little as possible. For research purposes, the lack of follow up of the children in the system means researchers must look outside the system itself for information.