We are testing a new beta website for gov.scot go to new site

Technical Note: Follow up Survey of Leavers from Scottish Schools: Leavers from School Year 2007/08

DescriptionMethodology used for Follow up Survey of School Leavers 2007/08
ISBN
Official Print Publication Date
Website Publication DateJune 30, 2009

Technical Note:

Follow Up Survey Of Leavers From Scottish Schools: Leavers From School Year 2007/08

Background

Information on the latest known destination of school leavers in September each year is supplied to the Scottish Government by Skills Development Scotland (SDS). This is referred to as the school leaver's 'initial destination' roughly 3 months after they left school (although some may have been Christmas leavers thus representing a longer follow up period). In 2006/07 this data was supplied at individual pupil level for the first time, and since then it has been possible to match it to information from the Pupil Census for the relevant year. This means that we were able to provide much more detailed analysis of the characteristics influencing different destinations. For example, pupils from more deprived areas (based on the Scottish Index of Multiple Deprivation 2006) are less likely to go on to Higher and Further Education than those from less deprived areas.

During March and April 2009, Careers Scotland contacted the 2007/08 cohort of leavers from publicly funded schools to update their destination from the initial survey. The results will be used to monitor the national indicator on increasing the proportion of school leavers in sustained positive destinations. This paper sets out the process through which the final estimate for the national indicator was derived.

Data matching

In 2006/07, matches were based on using the Scottish Candidate Number (SCN) first, and if that was missing, combinations of school code, date of birth, postcode and gender, since these four variables were common to both datasets. Full details of the methodology can be found in 'Methodological changes to the 2006/07 Destinations of School Leavers Survey' ( http://www.scotland.gov.uk/Publications/2007/12/07093501/26).

For 2007/08, SDS also supplied the Student ID (where known), which is not unique across schools. The methodology used in 2006/07 was applied to the 2007/08 data for the initial destinations publication ('Destinations of Leavers from Scottish Schools: 2007/08' http://www.scotland.gov.uk/Publications/2008/12/08090751/0). Prior to analysing follow up destinations, we revisited the matching process for 2007/08 data as part of a continuous programme of data improvement.

One consideration was that the original matching had excluded status 04 pupils (those on the roll of a school but attending a course of Further Education elsewhere) from the exercise, and SDS had since confirmed to us that their data included such pupils. Furthermore, the follow up survey of leaver destinations which is the true focus of National Indicator 10 uses pupil characteristics in the process of imputing destinations for leavers who could not be contacted. This consideration lent weight to the need to improve the matching of leavers to pupil census records.

For the rematching, status 04 pupils were included and tighter constraints on what was considered a good match were applied. Rematching was performed on the follow up data provided by SDS at the beginning of June 2009.

The following table shows the results of the matching.

Table 1: Match type, 2007/08

Number

%

scn,student id,centre,gender,postcode,dob

49,268

83.7

scn,centre,gender,postcode,dob

740

1.3

scn,centre,gender,postcode

42

0.1

scn ,centre,gender,dob

5,522

9.4

scn ,centre,gender

21

0.0

scn ,gender,dob

372

0.6

student id,centre,gender,postcode,dob

458

0.8

student id,centre,gender,postcode

5

0.0

student id,centre,gender,dob

104

0.2

student id,centre,gender

2

0.0

student id,gender,dob

18

0.0

centre,gender,postcode,dob

319

0.5

centre,gender,postcode

707

1.2

No match found

1,266

2.2

All

58,844

100.0

  • The total number shown (58,844) differs from that published in December 2008 (58,791). The reason for this is that leavers are only included in the published figures if they leave from a publicly funded secondary school, so changing the match may result in changing the school concerned.
  • For pupils where no match was found on the census (1,266), the data reported by SDS on school, gender and postcode was accepted and no attempt was made to assign additional characteristics. A non-match may result from a lack of information from SDS (not all fields were supplied for all leavers) or because the leavers were not on that year's pupil census (pupils may be omitted if they are not in the school on census day, for instance if they had left school but subsequently returned).
  • Of the 58,844 leavers:

o 56,657 are status 01 (on roll and based in school)

o 921 are status 04 (on roll but attending full time Further Education elsewhere

o 1,266 are unknown (no match on pupil census, due to limited information or absence of the leaver from that year's census as noted above)

  • The percentage in positive destinations compared to the figure published in December 2008 decreases slightly from 86.5 to 86.4 This would have made no difference to the arrow direction for National Indicator 10 - unchanged.
  • The percentages in the destinations categories (HE, FE etc) does not change when taken to one decimal point.
  • Total numbers and percentages in destinations categories have changed slightly by local authority. This is a result of matching to different schools in the rematched dataset compared to the original dataset.

Rematching was also carried out on 2006/07 data to test comparability. Again, the number of leavers in publicly funded secondary schools that would have been reported changed slightly, from 57,364 to 57,387. Headline figures for positive initial destinations would remain unchanged at 86.6%. The 2006/07 figures reported in 'Follow Up Survey Of Leavers From Scottish Schools: Leavers From School Year 2007/08' ( http://www.scotland.gov.uk/stats/bulletins/0000738) are based on the original published totals, while the 2007/08 figures are based on the rematched data.

We are confident that this rematched data is an improvement on the original. The matching methodology will be kept under review, and the possibility of using specialised data linking software is being explored.

National Indicator Estimate

Creating the dataset

The following table outlines the steps taken to arrive at the final dataset for deriving the national indicator estimate. Firstly, the matched data gives the number of pupils in valid schools - that is, in publicly funded and grant-maintained secondaries. Leavers whose original destinations was Moved outwith Scotland were excluded, as problems with following up such leavers would be anticipated.

Table 2: Numbers for Follow up Analysis

No. of leavers on data set supplied by SDS

60,165

No. of leavers removed as duplicates

14

No. of leavers not from publicly funded secondaries

856

Leaving:

59,295

No. of leavers whose initial destination was 'moved outwith Scotland'

451

Leaving (number included for initial destinations):

58,844

No. of leavers whose follow up destination was moved outwith Scotland

151

No. of leavers whose follow up destination was deceased

11

No. of leavers whose follow up destination was school pupil

79

No. of leavers in the follow up survey

58,603

In the follow up survey, those leavers whose destination was recorded as school pupil have been excluded from the results. The surveys are intended to focus on young people of school leaving age who left school during or at the end of the school year (which is taken to run from 1 August to 31 July). In practice many young people do not decide whether or not to leave school until after they have received the results of external examinations, have tried to get a job or achieve entry to further or higher education or training. For many pupils, therefore, it is only possible to identify at a later date whether they have effectively left school by 31 July. Where the follow up has established that pupils originally thought to be school leavers are now back in school, such pupils are excluded from the analysis. Other leavers have been excluded from the original data because they are deceased or have moved outwith Scotland prior to the follow up survey.

Imputation for non-follow up

In the initial survey SDS was able to establish the destination of 98.9% of school leavers, and those results were published with the remaining 1.1% stated as 'Unknown'. For the follow up survey, destinations could be established for 94.5% of eligible cases, an improvement on the 91.1% sucessfully followed up in 2006/07. However, the higher rate of non-response compared to the original destinations survey suggested that follow up destinations should be imputed if analysis suggested that certain groups would be under-represented by only considering cases where a follow up destination is available.

The cases without a valid follow up destination comprised those not followed up: 2,913 leavers with a known initial destination and and unknown sustained destination, and 337 with both intitial and follow up destinations being unknown. These latter cases are treated as unknowns for sustained destinations.

Analysis of the follow up survey showed that several variables might be related to the likelihood of a successful follow up. The variables considered for use in the imputation were i) match type - i.e. whether a match to the pupil cenus was possible or not; ii) original detailed destination data; iii) deprivation as measured by the Scottish Index of Multiple Deprivation 2006 (SIMD); and iv) stage of leaving school.

Several other variables were examined, and either did not appear to significantly affect the likelihood of being followed up (e.g. gender) or were not regarded as being suitably robust indicators, due to small numbers (e.g. ethnicity) or doubts about the suitability of the data(e.g. looked after status). Free school meal registration, despite being used in the imputation exercise in the previous year, was excluded from consideration due to concerns about the quality of the data available in the relevant year's census, 'Pupils in Scotland 2007' ( http://www.scotland.gov.uk/Publications/2008/07/28100032/0) (see background note 5.9 in the publication). Although the SIMD variable did not appear to be as strongly related to the likelihood of being sucessfully followed up as the other chosen variables, it was felt that having another socioeconomic indicator in place of free school meals was desirable.

These variables were chosen after examing the percentages and numbers followed up by classes of the variable, followed up by chi-square tests of association. Logistic regression was then used to assess models including the chosen variables. At this point the variable related to stage of leaving school was dropped, as its influence was subsumed into the effects of original destination and SIMD indicators once all were included in the model.The following tables shows the rate of follow up for each initial (detailed) destination, SIMD category and match type.

Table 3: follow up rates

Followed up?

Yes

Yes

No

No

All

Original status

Leavers

%

Leavers

%

Leavers

Higher Education

17,424

95.2

876

4.8

18,300

Further Education

14,115

97.1

426

2.9

14,541

Training

515

98.3

9

1.7

524

Get Ready For Work

2,280

97.0

70

3.0

2,350

Employed - with Training

7,293

95.8

318

4.2

7,611

Employed - without Training

5,825

91.5

542

8.5

6,367

Part-time Employment

804

92.1

69

7.9

873

Voluntary Work

121

93.8

8

6.2

129

Unemployed

5,910

92.7

468

7.3

6,378

Economically Inactive

627

87.3

91

12.7

718

Unavailable - Ill Health

123

77.4

36

22.6

159

Unknown

316

48.4

337

51.6

653

Total

55,353

94.5

3,250

5.5

58,603

SIMD

Most deprived 20%

11,653

93.2

844

6.8

12,497

Other 80%

43,700

94.8

2,406

5.2

46,106

Total

55,353

94.5

3,250

5.5

58,603

Matched to census

Match to census

54,275

94.6

3,073

5.4

54,275

No match

1,078

85.9

177

14.1

1,078

Total

55,353

94.5

3,250

5.5

55,353



The final set of characteristics was thus:

  • Initial destination
  • Match type - those not matched to a record in the pupils census as opposed to those matched
  • SIMD ranking recoded to 20% most deprived datazones and other 80% datazones

As the percentage of known destinations was very high, it was decided that a simple imputation method would provide a sufficient adjustment to improve the representativeness of the results.

Where the initial destination was known, respondents and non-respondents were split into groups by destination, and then again by whether or not matched to the census and by SIMD indicator. This made it possible to determine the distribution of follow up destinations of respondents within each group and randomly allocate follow-up destinations to non-respondents with the same characteristics in the same proportions.

Effects of imputation

The effects of this imputation are shown in the following tables which provide the final published data together with the comparable tables if imputation had not been carried out and only the known follow up destinations had been analysed.

Table 1 (as published - with imputation)

%

Positive Destinations

84.0

Other Destinations

16.0

Number of leavers

58,603

Table 1 (known destinations only)

%

Positive Destinations

84.8

Other Destinations

15.2

Number of leavers

55,353

Table 2 (as published - with imputation)

Destination

%

Higher Education

29.7

Further Education

21.3

Training

4.2

Employment

28.5

Voluntary Work

0.3

Unemployed Seeking

13.4

Unemployed Not Seeking

2.0

Unknown

0.6

Number of leavers

58,603

Table 2 (known destinations only)

Destination

%

Higher Education

30.0

Further Education

21.9

Training

4.2

Employment

28.4

Voluntary Work

0.3

Unemployed Seeking

13.3

Unemployed Not Seeking

1.9

Number of leavers

55,353

Table 3 (as published - with imputation)

Follow up (March/April) Destinations

Positive

Other

Not applicable

Total

Initial (September) Destinations

%

%

%

Positive

78.4

7.7

0.3

50,862

Other

5.3

8.2

0.1

7,996

Total

83.7

15.9

0.4

58,858

Table 3 (known follow up destinations only)

Follow up (March/April) Destinations

Positive

Other

Total

Initial (September) Destinations

%

%

Positive

79.6

7.8

48,377

Other

5.2

7.4

6,976

Total

84.8

15.2

55,353

This comparison indicates that the imputation of follow up destinations has had a relatively small effect on the results of the survey. It can be seen from Table 2 that the imputation has produced a small increase in the destination categories which the analysis of follow up rates suggested may have been under-represented in the data, i.e. the various employed and unemployed categories.

Tables of the distributions of follow up destinations by intital destinations for imputed cases and non-imputed cases are given below.

We are confident that by adjusting the data to be more representative of the overall cohort, this imputation has improved the quality of our estimate of the proportion of school leavers in positive and sustained destinations. If you have any comments or questions about the imputation method used in this survey, please contact school.stats@scotland.gsi.gov.uk.

Imputed follow up destination

Higher Education

Further Education

Training

Employment

Voluntary Work

Unemployed Seeking

Unemployed Not Seeking

Unknown

Total

Initial Destination

%

%

%

%

%

%

%

%

No.

Higher Education

93.7

0.5

-

3.9

-

1.8

0.1

-

876

Further Education

0.7

80.0

1.6

7.5

-

9.6

0.5

-

426

Training

1.3

3.8

35.4

22.8

-

36.7

-

-

79

Employment

0.4

1.8

1.7

83.0

-

12.6

0.4

-

929

Voluntary Work

-

-

-

25.0

62.5

12.5

-

-

8

Unemployed Seeking

0.4

5.3

12.6

24.4

0.4

52.4

4.5

-

468

Unemployed Not Seeking

-

0.8

0.8

10.2

-

22.0

66.1

-

127

Unknown

0.6

0.3

-

1.2

-

-

0.9

97.0

337

Total

25.6

12.1

3.4

30.4

0.2

14.7

3.5

10.1

3,250

Non-imputed follow up destination

Higher Education

Further Education

Training

Employment

Voluntary Work

Unemployed Seeking

Unemployed Not Seeking

Total

Initial Destination

%

%

%

%

%

%

%

No.

Higher Education

93.3

0.5

0.1

4.0

0.0

1.8

0.3

17,424

Further Education

1.2

79.5

1.9

7.6

0.1

9.0

0.7

14,115

Training

0.1

5.7

35.2

23.3

0.9

33.0

1.9

2,795

Employment

0.7

1.8

1.9

84.0

0.1

10.8

0.6

13,922

Voluntary Work

1.7

4.1

6.6

16.5

47.9

17.4

5.8

121

Unemployed Seeking

0.6

5.6

12.6

23.9

0.8

51.7

4.9

5,910

Unemployed Not Seeking

0.9

2.1

2.4

11.1

0.7

19.7

63.1

750

Unknown

5.1

12.7

6.3

29.1

1.6

42.1

3.2

316

Total

30.0

21.9

4.2

28.4

0.3

13.3

1.9

55,353