We are testing a new beta website for gov.scot

## Scottish Crime Survey 2003

Listen

#### SCOTTISH CRIME SURVEY 2003

##### APPENDIX C: SAMPLING ERROR AND DESIGN EFFECTS

Data collected in surveys always provide estimates of the true proportions in the population. The accuracy of these estimates - the sampling error - can be calculated for any estimate in the survey using information about the proportion of people giving a response and the number of people in the sample (or sub-sample). The sample error can be expressed as a 'confidence interval', which can be added to and subtracted from the survey estimate to give a range within which it is fairly certain that the true value lies.

The precision of estimates derived from a sample is normally measured using standard errors. Essentially standard errors are calculations of the standard deviation of the sampling distribution. They differ from standard deviations, though, in that while standard deviations are a measure of variability derived from actual observations of a sample, standard errors refer to the variability of possible values that could be obtained from a series of samples.

Usually, the formula used for calculating standard errors assumes a simple random sample (SRS). However, as the SCS sample was both stratified and clustered, the standard errors must take into account the sampling methodology in order to calculate confidence intervals.

Tukey's Jack-knife technique was used to calculate the complex standard errors for a series of results in the study. This is a well-established technique for working through the effects of stratification and clustering. Basically, Tukey's Jack-knife creates a series of sub-samples of data, recreating the original methodology of sampling the "clusters" of areas. The sub-samples preserve the stratification and clustering techniques used in the original survey. The technique then goes on to simulate a further series of samples based on evidence observed from the sub-samples and the estimate of the error - a complex standard error - is based on the degree of variation of the data from simulated sample to simulated sample.

Finally, the design effect was calculated as the ratio between the complex and the SRS standard errors. This indicates the effect of the sampling design (stratification and clustering) on the standard error and hence confidence intervals. A design effect of 1 indicates that the effect of the sampling methodology used in the survey is equivalent to that of a simple random sample. A design effect of 1.2 would indicate that the sampling methodology is 20% less efficient than a simple rate sample, while, conversely, a design effect of 0.8 indicates that the sampling methodology improves the sampling efficiency.

Table C.1 shows the standard errors, the design factors, and the confidence intervals for the victimisation rates. As can be seen from the table, the design factors ranged from 0.76 to 1.30. The overall average is 1.03, but that should not be taken as a 'typical' value, given the distribution of values across different variables. However, it suggests that using a value of 1.2 as a 'rule of thumb' for adjusting the standard errors of the survey data would safely account for the design factors associated with most estimates from the survey.

Table C.1: Victimisation Rates and Sampling Errors for 2003 SCS

 Rate Confidence Intervals SRS Standard Error Complex Standard Error Design effect* COMPARABLE WITH POLICE VANDALISM 1,656 183 91.0 93.6 1.03 ACQUISITIVE 1 620 106 54.3 54.0 0.99 Housebreaking 1 397 82 41.2 41.8 1.01 Theft of motor vehicle 91 35 20.8 18.0 0.86 Bicycle theft 132 38 19.2 19.3 1.01 VIOLENCE 598 133 57.2 67.9 1.19 Assault 550 108 55.2 54.9 0.99 Robbery 48 35 13.4 17.9 1.29 OTHER SURVEY CRIMES Theft from a motor vehicle 408 85 39.7 43.3 1.09 Attempted theft of/from m.vehicle 126 35 17.3 17.8 1.03 Other household theft * 499 95 47.1 48.4 1.03 Theft from the person 59 25 12.8 12.5 0.98 Other personal theft 261 68 28.6 34.6 1.22 Motor vehicle vandalism 826 107 55.0 54.6 0.99 Property vandalism 831 221 68.6 112.5 1.64 Petty assault 435 93 49.6 47.5 0.96 Serious assault 115 37 22.5 19.0 0.84 ALL HOUSEHOLD CRIMES 3,309 340 137.0 173.4 1.27 ALL PERSONAL CRIMES 919 118 67.3 60.2 0.89

Note:

1. For violence, theft from the person, assault, robbery, other personal theft and all personal offences, rates are quoted per 10,000 adults. For acquisitive crime, vandalism, housebreaking, vehicle offences, bicycle theft, other household theft and all household offences, rates are quoted per 10,000 households.
2. For the distinction between crimes which are 'comparable with police' and 'other survey crimes', see Appendix D.
3. Source: 2003 Scottish Crime Survey, unweighted n=5,041.