We are testing a new beta website for gov.scot go to new site

Scottish Crime Survey 2003

Listen

SCOTTISH CRIME SURVEY 2003

APPENDIX C: SAMPLING ERROR AND DESIGN EFFECTS

Data collected in surveys always provide estimates of the true proportions in the population. The accuracy of these estimates - the sampling error - can be calculated for any estimate in the survey using information about the proportion of people giving a response and the number of people in the sample (or sub-sample). The sample error can be expressed as a 'confidence interval', which can be added to and subtracted from the survey estimate to give a range within which it is fairly certain that the true value lies.

The precision of estimates derived from a sample is normally measured using standard errors. Essentially standard errors are calculations of the standard deviation of the sampling distribution. They differ from standard deviations, though, in that while standard deviations are a measure of variability derived from actual observations of a sample, standard errors refer to the variability of possible values that could be obtained from a series of samples.

Usually, the formula used for calculating standard errors assumes a simple random sample (SRS). However, as the SCS sample was both stratified and clustered, the standard errors must take into account the sampling methodology in order to calculate confidence intervals.

Tukey's Jack-knife technique was used to calculate the complex standard errors for a series of results in the study. This is a well-established technique for working through the effects of stratification and clustering. Basically, Tukey's Jack-knife creates a series of sub-samples of data, recreating the original methodology of sampling the "clusters" of areas. The sub-samples preserve the stratification and clustering techniques used in the original survey. The technique then goes on to simulate a further series of samples based on evidence observed from the sub-samples and the estimate of the error - a complex standard error - is based on the degree of variation of the data from simulated sample to simulated sample.

Finally, the design effect was calculated as the ratio between the complex and the SRS standard errors. This indicates the effect of the sampling design (stratification and clustering) on the standard error and hence confidence intervals. A design effect of 1 indicates that the effect of the sampling methodology used in the survey is equivalent to that of a simple random sample. A design effect of 1.2 would indicate that the sampling methodology is 20% less efficient than a simple rate sample, while, conversely, a design effect of 0.8 indicates that the sampling methodology improves the sampling efficiency.

Table C.1 shows the standard errors, the design factors, and the confidence intervals for the victimisation rates. As can be seen from the table, the design factors ranged from 0.76 to 1.30. The overall average is 1.03, but that should not be taken as a 'typical' value, given the distribution of values across different variables. However, it suggests that using a value of 1.2 as a 'rule of thumb' for adjusting the standard errors of the survey data would safely account for the design factors associated with most estimates from the survey.

Table C.1: Victimisation Rates and Sampling Errors for 2003 SCS

Rate

Confidence Intervals

SRS Standard Error

Complex Standard Error

Design effect*

COMPARABLE WITH POLICE

VANDALISM

1,656

183

91.0

93.6

1.03

ACQUISITIVE 1

620

106

54.3

54.0

0.99

Housebreaking 1

397

82

41.2

41.8

1.01

Theft of motor vehicle

91

35

20.8

18.0

0.86

Bicycle theft

132

38

19.2

19.3

1.01

VIOLENCE

598

133

57.2

67.9

1.19

Assault

550

108

55.2

54.9

0.99

Robbery

48

35

13.4

17.9

1.29

OTHER SURVEY CRIMES

Theft from a motor vehicle

408

85

39.7

43.3

1.09

Attempted theft of/from m.vehicle

126

35

17.3

17.8

1.03

Other household theft *

499

95

47.1

48.4

1.03

Theft from the person

59

25

12.8

12.5

0.98

Other personal theft

261

68

28.6

34.6

1.22

Motor vehicle vandalism

826

107

55.0

54.6

0.99

Property vandalism

831

221

68.6

112.5

1.64

Petty assault

435

93

49.6

47.5

0.96

Serious assault

115

37

22.5

19.0

0.84

ALL HOUSEHOLD CRIMES

3,309

340

137.0

173.4

1.27

ALL PERSONAL CRIMES

919

118

67.3

60.2

0.89

Note:

1. For violence, theft from the person, assault, robbery, other personal theft and all personal offences, rates are quoted per 10,000 adults. For acquisitive crime, vandalism, housebreaking, vehicle offences, bicycle theft, other household theft and all household offences, rates are quoted per 10,000 households.
2. For the distinction between crimes which are 'comparable with police' and 'other survey crimes', see Appendix D.
3. Source: 2003 Scottish Crime Survey, unweighted n=5,041.