5. Accuracy of housing land audits
5.1 To review previous research and current Housing Land Audits (HLA) to establish how accurate they are in predicting future housing completions.
5.2 The Scottish Government has previously published two research reports considering HLAs. The current guidance for councils is set out in PAN 2/2010. These three documents were reviewed and the key conclusions were identified.
5.3 The 5 councils with the most planning appeals for the exceptional release of land were identified. These were Edinburgh, Fife, North Lanarkshire, Stirling and West Lothian. For each council the online published HLAs were reviewed in an attempt to compare predicted programming with actual completions. Where possible, the predicted programming and actual completions were compared for selected sites. Due to time constraints, only two sites for each council area was chosen. The chosen sites had to be relatively large and almost completed in 2018/2019. This information was not available for North Lanarkshire and only one site could be identified for Stirling.
5.4 In 2008 the Scottish Government published research into HLAs entitled, "An Assessment of Current Practice in Respect of Preparing and Publishing HLA in Scotland". Part of the brief was to establish the accuracy of HLAs. 21 HLAs were considered looking at the programming from 2001 - 2006 and comparing this with actual completions.
5.5 A summary of the findings was as follows:
- 10% of HLAs correctly or under estimated completions
- 90% of HLAs overestimated actual completions
- 10% of HLAs overestimated by 10% or less
- 43% of HLAs overestimated by 10%- 30%
- 20% of HLAs overestimated by more than 50%
- The average overestimate was 27%
5.6 Overall, the researchers concluded that there was an optimism bias in most HLAs in Scotland. The researchers expressed surprise at these findings because 2001 - 2006 was a period of increasing house building completions. They suggested that the likely cause was over optimistic expectations of start dates and over optimistic assumptions as to site programming.
5.7 The report made 16 recommendations for improvements to HLAs. These recommendations formed the basis for the publication of PAN 2/2010, although not all the recommendations were adopted.
5.8 PAN 2/2010 replaced PAN 38, the previous advice relating to HLAs. It set out more guidance for some of the technical aspects of HLA preparation such as small sites, windfalls, demolitions, key information to be presented etc. The aim was to create greater consistency of approach across Scotland. The guidance recognised that an HLA was only a "snapshot" in time and that they were, "widely recognised as less than scientific exercise". Nonetheless it was emphasised that it was important not to overestimate or underestimate.
5.9 In January 2019 the Scottish Government published further research entitled, "Assessment of Housing Land Audits for Consistency and Compliance, their Potential for Standardisation and their Role in Development Plan Delivery". The main focus of this research was to see if the guidance in PAN 2/2010 was being followed. The overall conclusion was that there were, "clear, numerous and significant inconsistencies with regard to the content of and outputs from current HLAs".
5.10 Due to these differences, the researchers found it difficult to establish a Scottish wide picture of completions and programming. They did however attempt to estimate one. They estimated that across Scotland from 2017/18 (date of the research), future programming assumptions would amount to a 10% compound growth in expected completions. The researchers expressed surprise at this finding.
5.11 The researchers also considered that the role of HLAs had changed. Originally they were solely used as a monitoring tool. Increasingly, however, HLAs had become the main mechanism for demonstrating the availability of sufficient effective land to meet the requirement for a 5 year supply.
5.12 Overall, the researchers found that 60% of HLAs were disputed but that the actual rate of disputes (i.e. the number of individual sites) was low. They identified significant differences and difficulties in establishing completions and future site programming.
5.13 The 5 year programming from 5 HLAs were compared with the reported actual completions. This is shown in the following table
(highlighted bold = within +/- 10%)
5.14 Thirteen of the predictions underestimated actual completions, with 2 over estimating predictions. Four predictions were within 10%. The overall degree of underestimation was 17%, the largest underestimation was 30%.
5.15 Two sites were tracked through the different HLAs. In both cases, construction took place quicker than originally predicted. It should be noted that programming was adjusted at each HLA.
|Site ref number||Capacity||15/16||16/17||17/18||18/19||19/20||20/21||21/22|
(Figures in brackets represent latest programme rather than completions)
5.16 Fife has a complicated development plan context. The southern part is within SESplan and the northern part in Tayplan. Data is reported by housing market areas, the spatial definition of housing market areas has changed. Within the time available it was not possible to clarify how small sites and windfall sites are reported in the different HLAs. However it was possible to compare the programming and completions for the 2014 HLA as shown in the table below. The programming over estimated actual completions for every year. The overall average overestimation was 26%.
|% over estimate||16%||43%||17%||19%||37%||26%|
5.17 Two sites were tracked through the different HLAs. In both cases construction took place quicker than originally predicted. It should be noted that programming was adjusted at each HLA.
|Site ref||Capacity||Pre 13||13/14||14/15||15/16||16/17||17/18||18/19||19/20||20/21||21 on|
5.18 It should be noted from the above that initially site COW6 would have been predicted to contribute 116 units to the 5 year effective supply, whereas it actually contributed 208 units to the 5 year supply. Likewise, site DAC 253 was predicted to contribute 130 units initially but actually contributed 176 units.
5.19 The only available HLA online is the latest 2019 HLA. However, Clydeplan have archived previous land monitoring reports, which includes information for North Lanarkshire. This includes a comparison with predicted completions against actual completions. In the Clydeplan area, the councils have always used a 7 year supply of effective land. The figures and percentages are not therefore comparable to the 5 year supply used by other councils. Predicting 7 years into the future is likely to be more inaccurate than 5 years. Nonetheless, the average overestimation equates to 40%.
5.20 For North Lanarkshire it was not possible to track any individual sites.
5.21 Within each HLA Stirling Council compares the programmed completions from the year before with the actual completions. This is shown in the following table
5.22 Of the 14 predictions compared, 11 were overestimated and 3 were underestimated. Six of the 14 were within 10% of the prediction. However, 2 were over estimated by more than 80%. It should be noted that Stirling has a relatively low level of completions (compared to the larger councils looked at). This means that Stirling would be relatively sensitive to changes in a few sites.
5.23 Only one site could be found to track through successive housing land audits. The actual completions were higher and there was a big increase in the overall capacity of the site.
5.24 Eight previous HLAs were able to be compared with actual completions as shown in the following table. Nineteen represented an overestimation whereas 11 represented an underestimation. The average size of overestimation was 45% and the average underestimation was 12%. Twelve of the 30 predictions were within 10%.
(figures in bold within 10%)
5.25 Two sites were tracked from individual HLAs. It turned out that one site stopped construction for a number of years because the original building company went into administration. The site was acquired by several different companies thereafter. However, the site was kept in this study to illustrate the impact of unforeseen circumstances.
|Site ref||Capacity||Post 11||11/12||12/13||13/14||14/15||15/16||16/17||17/18||18/19||Post 19|
5.26 The 2008 research generally found that at that time there was an optimism bias in HLAs and that they were not particularly accurate. The 2019 research expressed surprise that so far as a Scottish wide HLA could be developed, the future programming from 2017/18 assumed a 10% compound growth in completions.
5.27 The analysis of HLAs from 5 councils in the central belt was consistent with these findings. Of the 68 programming predictions compared with actual completion data, 41 (60%) were an overestimation.
5.28 However, there was a difference between council areas. In Edinburgh's case, 13 out of 15 programming predictions underestimated actual completions. Approximately a third of predictions in West Lothian resulted in an underestimation and 3 out of 14 in Stirling were an underestimation.
5.29 Generally, programming predictions were not accurate. Only a third of predictions were within 10% of the actual number of completions (i.e. +/- 10%). It should be noted that the 2019 research identified clear, numerous and significant inconsistencies with regard to the content and output of HLAs. In particular, concerns were expressed regarding completion information and how future programming was derived. These are fundamental to the accuracy of a HLA.
5.30 The individual sites were not selected in order to be representative of sites across central Scotland. However, all but one finished faster than originally predicted. It is clearly difficult to accurately predict how many houses will be constructed even beyond 2 years later. As a HLA is a sum of many individual programming predictions it is not surprising that inaccuracies occur.
5.31 If the role of HLA is as a monitoring tool that is annually updated, then the consequences of any inaccuracies are not serious. However, if its role is to identify the stock of effective housing land, then it is clear that it is not a reliable mechanism. In some instances, they have been shown to be very unreliable.
5.32 If generally HLAs overestimate the number of completions that actually take place, that does not indicate that there is a general shortage of housing land constraining housing unit outputs. That would more likely to be the case where actual completions exceeded expectations. This only occurred regularly in Edinburgh.
5.33 PAN 2/2010 defines effective housing land as, "the part of the established housing land supply which is free or expected to be free of development constraints in the period under consideration, and will therefore be available for the construction of housing". Councils are expected to make a judgement as to the overall effectiveness of a site based on the following criteria, ownership, physical, deficit funding, marketability, infrastructure and land use. Marketability is the criteria which is the hardest to apply because it is changeable and subjective. What is or is not an effective site is therefore a matter of judgement and not a matter of fact.
5.34 There is generally little dispute as to what constitutes the established supply of land. The effective housing land supply will be subset of the established supply but the actual scale will be impossible to determine with any precision and it will change depending on market conditions. The expected future programming of housing completions over a 5 year period will be a subset of the effective housing supply. It is likely to be smaller than the effective supply but again is impossible to define with any precision. The actual number of completions will be a subset of the likely programmed completions. Generally, this is less than the programmed number of completions but this can vary depending on local market circumstances.
5.35 The supply of effective housing land is a dynamic variable, and to some extent a matter of judgement. HLAs are unable to predict future programming with any accuracy and should therefore be considered in broad terms rather than as a precise measure of the amount of effective housing land.