Development Department Research Programme Research Findings No. 90Research into the Basis for Local and National Estimates of the Number of Below Tolerable
Standard (BTS) Houses in Scotland
DTZ Pieda Consulting in association with Dr J I Ansell
This study was commissioned to evaluate the various methods used at local and national level to estimate the number of houses in Scotland which are below the Tolerable Standard (BTS) - a set of minimum criteria for housing as specified in the Housing (Scotland) Act 1987. The report provides a detailed account of how local and national estimates are prepared, assesses the reliability of the methods used and makes recommendations about which methodologies should be used in the future.
There are two sources of information on the number of below tolerable standard houses in Scotland - annual estimates prepared by local authorities for their own areas, and the figures produced for the whole of Scotland by the Scottish House Condition Survey (SHCS), generally every five years.
Local authorities do not adopt a consistent approach to providing estimates of the number of BTS houses, and there are many aspects of local authority practice which suggest that estimates are unlikely to be accurate. These aspects are detailed fully in the report.
The Scottish House Condition Survey (SHCS) has attempted to achieve consistency throughout Scotland but it too has been bedevilled by what the research team believe to be the key difficulty in securing accuracy and consistency - the nature of the Tolerable Standard, and the fact that many elements of the standard require a judgement to be made as to whether a particular house should pass or fail.
It seems likely that surveying practices changed between the 1991 and 1996 SHCS surveys, and that by 1996 the SHCS practices and conventions for interpreting the Standard were being much more rigorously followed than in 1991. The 1991 survey was the first national house condition survey and, as such, it involved a new approach for many of the surveyors involved. There were also a number of minor changes in the interpretation of the Standard between 1991 and 1996
The principal conclusion of the research is that obtaining reliable and consistent estimates is entirely dependent on reaching an agreed interpretation of the Standard and using that interpretation consistently in the field.
The prevalence of BTS housing has fallen and this raises a number of statistical issues for estimates which are based on sample surveys such as the SHCS and local authority surveys. The question of altering the design of the survey and boosting sample size needs to be addressed, given the extremely uneven distribution of the problem.
Each year, local authorities provide the Scottish Executive with estimates of the number of houses which are below the Tolerable Standard (BTS) in their area. In addition, the Scottish House Condition Survey (SHCS) provides a national estimate (every five years) of the number of BTS houses in Scotland based on a physical survey of a sample of properties (18,000 in 1996). The data is used by the Scottish Executive to assist in developing housing policy and in monitoring the outcomes of housing investment.
There has been growing concern that the available data are not suitable for being used in this way, as the BTS estimates are not consistent between sources or over time. This research was carried out to investigate the way local and national estimates are produced and recommend a more reliable and consistent approach.
Interpretation of the Tolerable Standard
There is considerable diversity in local authority practice in the interpretation of the Tolerable Standard. Most of the Standard's nine elements include words or phrases that are open to a degree of subjectivity in assessment, for example ' substantially free of rising and penetrating damp', ' satisfactory facilities for the cooking of food'.
The Standard involves a degree of interpretation and, in the preparation of local authority surveys and estimates, there has not been a consistent approach. In consequence, the results for local authorities are not strictly comparable.
There is very detailed guidance produced for SHCS surveyors and for local authorities which use the standard Local House Condition Survey (LHCS) methodology on the interpretation of the Standard. However, many authorities use their own or some earlier interpretation of the Standard and there is no evidence to suggest that there is any degree of uniformity in the interpretation used by staff in different authorities.
Local Authority Practice
All local authorities start with a baseline estimate of the number of BTS houses, but some of these baselines are now very old and there is uncertainty about how they were produced. This estimate is updated annually by adding properties becoming BTS in the period and subtracting properties that are no longer BTS because of, for example, demolition or improvement. However the sources of information and the methods used are incapable of picking up all real change. There is significant variation between authorities in the methods used to produce both baseline and update figures.
Three quarters of local authorities have used a house condition survey to produce baseline data, but many of the house condition surveys were undertaken some considerable time ago. Other authorities have used a combination of surveys and their own records, which are based to varying degrees on out of date information. In some cases, authorities rely on imputed data rather than data from a survey- i.e. they make an assumption about the proportion of certain types of stock which are likely to fail the Standard.
Local authorities use a wide range of data sources to update their BTS estimates and methods are not consistent across Scotland. Methods include grant awards, demolition orders, waiting lists, periodic surveys, tenant notification and improvement orders.
The SHCS and Local Authority Estimates
In 1996, the Scottish House Condition Survey (SHCS) produced very different results to the local authority estimates produced for that year - the SHCS estimate for Scotland was 27,000, whereas the local authority estimates for the same year were 84,000. In 1991, the estimate produced by the 1991 SHCS had been 95,000.
The SHCS methodology was designed to minimise surveyor variation in interpretation of the Standard to the greatest possible extent. As a result, its basis is substantially more systematic, up to date and consistent than the sources of the estimates produced by local authorities. The weaknesses and inconsistencies in the basis of the local authority estimates are such that they should not be regarded as a reliable standard against which other estimates such as those produced by SHCS should be judged. However, there is good reason to believe that the SHCS itself has not produced consistent estimates over time. Differences in the interpretation of the Tolerable Standard relating to dampness and cooking facilities between the 1991 and 1996 SHCS and more rigorous adherence to SHCS conventions in the later survey make it difficult to say how much of the reduction in the number of BTS houses shown was as a result of real change rather than a change in survey practice.
Difference Between SHCS and Local Authority Estimates
It is not surprising that there were major differences between SHCS and local authority estimates, given the variety of approaches used by local authorities; indeed it might be seem as more surprising that the estimates from the two sources were so similar in 1991. Some authorities use quite different approaches to the type of methodology used in the SHCS, whereas others use a similar methodology, but often with a very different survey date. A further problem is that local authorities have, in many cases, used very different interpretations of the Tolerable Standard from that which was used in the 1996 SHCS. It is this last factor which we believe to be the most significant.
Another reason for differing figures is that in local authority areas where there is a very low prevalence (1% or under) of BTS housing, the local estimates produced have a wide margin of error. The small sample size used is therefore bound to produce a difference between local authority and SHCS data.
The key issues affecting reliability and consistency are:
the nature of the Standard and its interpretation: the Standard is not being interpreted consistently by different authorities, nor has it been consistently interpreted over time.
problem elements in the Standard: dampness is the most subjective element and it was for that reason that the 1996 SHCS introduced a more tightly specified definition of the degree of dampness which would cause a house to fail the Tolerable Standard. Other areas where there is difficulty in interpretation are water supply, cooking facilities and the presence of a satisfactorily located WC. Some, but not all, authorities use information gained from the testing of private water supplies as a source of information on the wholesomeness of water. The main difficulty with the assessment of the adequacy of cooking facilities appears to be a change in briefing standard in 1996. In 1995 a Scottish Office Circular changed the definition of 'satisfactorily located' with regard to the WC in the house, but a number of authorities have not readjusted their figures to reflect this change, partly because they do not have the survey evidence to do so.
baseline survey problems: for some authorities, there was limited knowledge about the baseline data - when they were set up, whether the houses were inspected and how the Standard was interpreted.
treatment of defects in buildings with more than one dwelling: some authorities treat all units in a building as BTS if one dwelling fails the standard, while others do not.
of abandoned / derelict units: some authorities treat empty buildings separately, others include them in the total BTS figures. Case study and survey work has suggested that rural authorities may not be aware that some BTS dwellings have become derelict or may even have been demolished. One authority actively surveyed vacant houses because they were more likely to be BTS.
update information: the sources used to update BTS data vary from authority to authority; in no case however are they likely to pick up all real change
Local House Condition Survey (LHCS) confidence intervals: at the very low level of prevalence of BTS which is now beginning to be found - around 1% - using LHCS sample sizes, the confidence intervals for individual local authorities become quite wide and it is not possible to produce a statistically valid analysis of BTS housing by reason for failure at the level of the individual district.
prioritisation of BTS issues - BTS housing is now such a limited problem in some local authorities that these Councils are reluctant to commit resources to surveys and other means of obtaining more accurate information about BTS housing
Any change to the present system must be appropriate, practical and cost effective. The main recommendations of the research are that:
there should be an agreed interpretation of the Standard and agreement as to how certain situations such as derelict and abandoned units and BTS units in multiple dwellings are treated, and all parties involved in preparing estimates should commit themselves to using that interpretation. Without such commitment, it will be impossible for consistent estimates to be prepared in which both local authorities and the Scottish Executive can have confidence.
falling prevalence levels make it unlikely that the quality of data can be improved at either local or national level without either moving to a database system or, if sample surveys are to be used, increasing sample size and stratifying the sample. There are a number of options, each involving substantial additional cost.
any new approach must take into account the extremely uneven distribution of the problem and seek to avoid imposing high cost solutions on authorities which have very few problems. The research team's preferred solution would be to focus on those authorities which are recognised as having a substantial house condition problem, probably by introducing a stratified sample which oversamples the authorities with the largest numbers and percentages of BTS stock.
The research team questioned the value of the annual updates as they are currently carried out - there is no clear audit trail showing when and why the figures have been updated and thus how the current figure has been reached. Furthermore, they are sceptical about whether annual update procedures could ever capture all real change. A more standardised approach is recommended, with separate parallel reporting of those events and actions which are known with certainty to have, in the course of that year, increased or reduced the total number of BTS houses.
About the Study
The study involved the following elements:
a survey of all 32 local authorities, which involved interviews with the key personnel responsible for preparing the estimates.
an examination of the methodology used in the SHCS and also in Local House Condition Surveys (LHCS) which are the basis of many local authorities' estimates
case studies of six local authorities, which examined in more detail the aspects of local authority practice which are particularly critical to the accuracy of estimates and the costs of preparing estimates.
case studies of the nine authorities where an SHCS Local Report was provided in 1996. The data from these authorities represented a particularly important research resource for exploring the issue of why, at a national level, SHCS estimates diverged substantially from the amalgamation of local authority estimates for the same year.
an assessment of the relationship between estimates obtained from the local authorities and the national estimate provided by SHCS.
an examination of the key issues believed to be critical to the reliability and consistency of the estimates provided.
The examination of the SHCS and LHCS methodologies and the assessment for the reasons for divergence between national and local estimates was informed by a report on statistical issues by Dr Jake Ansell, a statistician at Edinburgh University.
If you wish a copy of "Research into the Basis for Local and National Estimates of the Number of Below Tolerable Standard (BTS) Houses in Scotland", the report which is summarised in this Research Findings, please send a cheque for £5.00 made payable to The Stationery Office to:
The Stationery Office Bookshop, 71 Lothian Road, Edinburgh, EH3 7AZ
Tel: 0870 606 5566; Fax: 0870 606 5588; http://www.tso.co.uk
If you wish further copies of this Research Findings or have any enquiries about the work of CRU, please contact us at:
Scottish Executive Central Research Unit, 2J, Victoria Quay, Edinburgh EH6 6QQ
Tel: 0131-244 7560, or Email: email@example.com
This document (and other CRU Research Findings and Reports) and information about the work of CRU may be viewed on the Internet at
The site carries up-to-date information about social and policy research commissioned and published by CRU on behalf of the Scottish Executive. Subjects covered include transport, housing, social inclusion, rural affairs, children and young people, education, social work, community care, local government, civil justice, crime and criminal justice, regeneration, planning and women's issues. The site also allows access to information about the Scottish Household Survey.