Local Area Labour Markets in Scotland - Statistics from the Annual Population Survey 2011

Annex D: Confidence intervals

One of the benefits of the boosted data is more reliable estimates for local authority areas. Prior to the boost the reliability threshold in all areas was 6,000. This was to prevent unreliable data being used. Thresholds are calculated so that they are approximately equivalent to suppressing if the standard error of an estimate is greater than 20% of the estimate itself. With the boost, different areas have different thresholds as some areas have larger samples and more variability in results than others (see Table D1).

Table D1: Local authority area reliability thresholds

As survey results, these are subject to a degree of error and implied changes over the years may not be significant and instead be within a given error range. Confidence limits can be used to assess the range of values that the true value lies between. The web tables include 95% confidence limits for each indicator.

What does the 95% confidence limit mean?

If the procedure for determining an estimate is used over and over (by repeatedly sampling from the population), the 95% confidence interval for these estimates will contain the true parameter value 95% of the time, i.e. 19 out of the 20 of the calculated confidence intervals will contain the true value.

If, for example, we have an APS estimate and confidence interval of 63%+/-0.27 percentage points, this means that we can be 95% confident that the true value lies between 62.73% and 63.27%. Furthermore, note that the smaller the computed confidence interval, the more reliable the estimate.

The confidence limits use a design factor of 1, which may not be likely in some cases but given the lack of further information an average design factor of 1 is assumed to be reasonable. Further information on estimating confidence intervals can be found in the LFS manuals^[7].

Using confidence intervals to assess change.

Confidence intervals can be used to assess whether there has been a significant change between two estimates over time. The methodology for determining if a change is statistically significant is detailed in the Methodology Glossary on the Scottish Government web-site within the Tier 2 - Confidence Intervals document, available at:
http://www.scotland.gov.uk/Topics/Statistics/About/Methodology/Glossary

If the difference between two estimates is said to be statistically significant, it means that only in exceptional circumstances (1 in 20 times) would we expect the true difference to be not significant. It should be noted that statistical significance is a tool used to help detect real change in estimates; it does not say anything about the importance of the change, which needs to be assessed by the user of the statistics in question.