Scottish Planning Policy - amendments: housing land research paper - evidence

5. Accuracy of housing land audits

Purpose

5.1 To review previous research and current Housing Land Audits (HLA) to establish how accurate they are in predicting future housing completions.

Methodology

5.2 The Scottish Government has previously published two research reports considering HLAs. The current guidance for councils is set out in PAN 2/2010. These three documents were reviewed and the key conclusions were identified.

5.3 The 5 councils with the most planning appeals for the exceptional release of land were identified. These were Edinburgh, Fife, North Lanarkshire, Stirling and West Lothian. For each council the online published HLAs were reviewed in an attempt to compare predicted programming with actual completions. Where possible, the predicted programming and actual completions were compared for selected sites. Due to time constraints, only two sites for each council area was chosen. The chosen sites had to be relatively large and almost completed in 2018/2019. This information was not available for North Lanarkshire and only one site could be identified for Stirling.

Literature Review

5.4 In 2008 the Scottish Government published research into HLAs entitled, "An Assessment of Current Practice in Respect of Preparing and Publishing HLA in Scotland". Part of the brief was to establish the accuracy of HLAs. 21 HLAs were considered looking at the programming from 2001 - 2006 and comparing this with actual completions.

5.5 A summary of the findings was as follows:

• 10% of HLAs correctly or under estimated completions
• 90% of HLAs overestimated actual completions
• 10% of HLAs overestimated by 10% or less
• 43% of HLAs overestimated by 10%- 30%
• 20% of HLAs overestimated by more than 50%
• The average overestimate was 27%

5.6 Overall, the researchers concluded that there was an optimism bias in most HLAs in Scotland. The researchers expressed surprise at these findings because 2001 - 2006 was a period of increasing house building completions. They suggested that the likely cause was over optimistic expectations of start dates and over optimistic assumptions as to site programming.

5.7 The report made 16 recommendations for improvements to HLAs. These recommendations formed the basis for the publication of PAN 2/2010, although not all the recommendations were adopted.

5.8 PAN 2/2010 replaced PAN 38, the previous advice relating to HLAs. It set out more guidance for some of the technical aspects of HLA preparation such as small sites, windfalls, demolitions, key information to be presented etc. The aim was to create greater consistency of approach across Scotland. The guidance recognised that an HLA was only a "snapshot" in time and that they were, "widely recognised as less than scientific exercise". Nonetheless it was emphasised that it was important not to overestimate or underestimate.

5.9 In January 2019 the Scottish Government published further research entitled, "Assessment of Housing Land Audits for Consistency and Compliance, their Potential for Standardisation and their Role in Development Plan Delivery". The main focus of this research was to see if the guidance in PAN 2/2010 was being followed. The overall conclusion was that there were, "clear, numerous and significant inconsistencies with regard to the content of and outputs from current HLAs".

5.10 Due to these differences, the researchers found it difficult to establish a Scottish wide picture of completions and programming. They did however attempt to estimate one. They estimated that across Scotland from 2017/18 (date of the research), future programming assumptions would amount to a 10% compound growth in expected completions. The researchers expressed surprise at this finding.

5.11 The researchers also considered that the role of HLAs had changed. Originally they were solely used as a monitoring tool. Increasingly, however, HLAs had become the main mechanism for demonstrating the availability of sufficient effective land to meet the requirement for a 5 year supply.

5.12 Overall, the researchers found that 60% of HLAs were disputed but that the actual rate of disputes (i.e. the number of individual sites) was low. They identified significant differences and difficulties in establishing completions and future site programming.

Edinburgh

5.13 The 5 year programming from 5 HLAs were compared with the reported actual completions. This is shown in the following table

	14/15	15/16	16/17	17/18	18/19
2014 HLA	1086	1944	2442	2499	2077
2015 HLA		1937	1812	1915	1944
2016 HLA			1891	1862	2642
2017 HLA				2226	2267
2018 HLA					2716
Actual completions	1525	2297	2457	2651	2399

(highlighted bold = within +/- 10%)

5.14 Thirteen of the predictions underestimated actual completions, with 2 over estimating predictions. Four predictions were within 10%. The overall degree of underestimation was 17%, the largest underestimation was 30%.

5.15 Two sites were tracked through the different HLAs. In both cases, construction took place quicker than originally predicted. It should be noted that programming was adjusted at each HLA.

Site ref number		Capacity	15/16	16/17	17/18	18/19	19/20	20/21	21/22
5253	Predicted	220	40	40	69	34	34	3
	Actual		34	46	52	47	(41)
5463A	Predicted	206		10	6	44	60	60	26
	Actual			25	54	123	(4)

(Figures in brackets represent latest programme rather than completions)

Fife

5.16 Fife has a complicated development plan context. The southern part is within SESplan and the northern part in Tayplan. Data is reported by housing market areas, the spatial definition of housing market areas has changed. Within the time available it was not possible to clarify how small sites and windfall sites are reported in the different HLAs. However it was possible to compare the programming and completions for the 2014 HLA as shown in the table below. The programming over estimated actual completions for every year. The overall average overestimation was 26%.

	14/15	15/16	16/17	17/18	18/19	Total
Programmed	922	1304	1531	1545	1535	6837
Actual	797	913	1303	1303	1118	5434
Over estimate	125	391	228	242	417	1403
% over estimate	16%	43%	17%	19%	37%	26%

5.17 Two sites were tracked through the different HLAs. In both cases construction took place quicker than originally predicted. It should be noted that programming was adjusted at each HLA.

Site ref		Capacity	Pre 13	13/14	14/15	15/16	16/17	17/18	18/19	19/20	20/21	21 on
COW6	Prog	252	31	20	24	24	24	24	25	25	25	50
	Actual	267	31	20	15	79	67	27	25	(3)
DAC253	Prog	273			40	30	30	30	30	30	30	53
	Actual	273			36	43	81	16	44	(39)	(14)

5.18 It should be noted from the above that initially site COW6 would have been predicted to contribute 116 units to the 5 year effective supply, whereas it actually contributed 208 units to the 5 year supply. Likewise, site DAC 253 was predicted to contribute 130 units initially but actually contributed 176 units.

North Lanarkshire

5.19 The only available HLA online is the latest 2019 HLA. However, Clydeplan have archived previous land monitoring reports, which includes information for North Lanarkshire. This includes a comparison with predicted completions against actual completions. In the Clydeplan area, the councils have always used a 7 year supply of effective land. The figures and percentages are not therefore comparable to the 5 year supply used by other councils. Predicting 7 years into the future is likely to be more inaccurate than 5 years. Nonetheless, the average overestimation equates to 40%.

	07/14	08/15	09/16	10/17
Programmed	8740	6722	6379	6122
Actual	5667	4845	4478	4881
Over estimated	3073	1877	1901	1241
%	54%	39%	42%	25%

5.20 For North Lanarkshire it was not possible to track any individual sites.

Stirling

5.21 Within each HLA Stirling Council compares the programmed completions from the year before with the actual completions. This is shown in the following table

	05/06	06/07	07/08	08/09	09/10	10/11	11/12	12/13	13/14	14/15	15/16	16/17	17/18	18/19
Prog	383	386	474	384	306	363	291	263	215	387	337	218	179	233
Actual	263	231	414	204	290	353	311	223	307	304	336	214	192	121
Diff	120	155	60	180	16	10	+20	40	+92	83	1	4	+13	121
%	46	67	14	88	6	3	+6	18	+30	27	0.3	2	+7	93

5.22 Of the 14 predictions compared, 11 were overestimated and 3 were underestimated. Six of the 14 were within 10% of the prediction. However, 2 were over estimated by more than 80%. It should be noted that Stirling has a relatively low level of completions (compared to the larger councils looked at). This means that Stirling would be relatively sensitive to changes in a few sites.

5.23 Only one site could be found to track through successive housing land audits. The actual completions were higher and there was a big increase in the overall capacity of the site.

Site ref		Capacity	15/16	16/17	17/18	18/19	19/20
254	Prog	167	41	34	32	32	28
	Actual	254	63	51	56	47	(37)

West Lothian

5.24 Eight previous HLAs were able to be compared with actual completions as shown in the following table. Nineteen represented an overestimation whereas 11 represented an underestimation. The average size of overestimation was 45% and the average underestimation was 12%. Twelve of the 30 predictions were within 10%.

HLA	11/12	12/13	13/14	14/15	15/16	16/17	17/18	18/19
2011	247	555	683	698	797
2012		577	554	714	796	777
2013			573	649	755	875	773
2014				722	1062	1109	1087	819
2015					654	1449	1373	995
2016						728	1169	1476
2017							639	1145
2018								1073
Actual completions	229	523	615	775	884	587	590	934

(figures in bold within 10%)

5.25 Two sites were tracked from individual HLAs. It turned out that one site stopped construction for a number of years because the original building company went into administration. The site was acquired by several different companies thereafter. However, the site was kept in this study to illustrate the impact of unforeseen circumstances.

Site ref		Capacity	Post 11	11/12	12/13	13/14	14/15	15/16	16/17	17/18	18/19	Post 19
Winch	prog	153					12	35	35	35	36
	Actual	153					12	24	40	41	27	(9)
Harthill	Prog	136	62	0	0	0	0	10	12	10	0	38
	Actual	136	62	3	12	0	0	0	1	16	22	(20)

Conclusions

5.26 The 2008 research generally found that at that time there was an optimism bias in HLAs and that they were not particularly accurate. The 2019 research expressed surprise that so far as a Scottish wide HLA could be developed, the future programming from 2017/18 assumed a 10% compound growth in completions.

5.27 The analysis of HLAs from 5 councils in the central belt was consistent with these findings. Of the 68 programming predictions compared with actual completion data, 41 (60%) were an overestimation.

5.28 However, there was a difference between council areas. In Edinburgh's case, 13 out of 15 programming predictions underestimated actual completions. Approximately a third of predictions in West Lothian resulted in an underestimation and 3 out of 14 in Stirling were an underestimation.

5.29 Generally, programming predictions were not accurate. Only a third of predictions were within 10% of the actual number of completions (i.e. +/- 10%). It should be noted that the 2019 research identified clear, numerous and significant inconsistencies with regard to the content and output of HLAs. In particular, concerns were expressed regarding completion information and how future programming was derived. These are fundamental to the accuracy of a HLA.

5.30 The individual sites were not selected in order to be representative of sites across central Scotland. However, all but one finished faster than originally predicted. It is clearly difficult to accurately predict how many houses will be constructed even beyond 2 years later. As a HLA is a sum of many individual programming predictions it is not surprising that inaccuracies occur.

5.31 If the role of HLA is as a monitoring tool that is annually updated, then the consequences of any inaccuracies are not serious. However, if its role is to identify the stock of effective housing land, then it is clear that it is not a reliable mechanism. In some instances, they have been shown to be very unreliable.

5.32 If generally HLAs overestimate the number of completions that actually take place, that does not indicate that there is a general shortage of housing land constraining housing unit outputs. That would more likely to be the case where actual completions exceeded expectations. This only occurred regularly in Edinburgh.

5.33 PAN 2/2010 defines effective housing land as, "the part of the established housing land supply which is free or expected to be free of development constraints in the period under consideration, and will therefore be available for the construction of housing". Councils are expected to make a judgement as to the overall effectiveness of a site based on the following criteria, ownership, physical, deficit funding, marketability, infrastructure and land use. Marketability is the criteria which is the hardest to apply because it is changeable and subjective. What is or is not an effective site is therefore a matter of judgement and not a matter of fact.

5.34 There is generally little dispute as to what constitutes the established supply of land. The effective housing land supply will be subset of the established supply but the actual scale will be impossible to determine with any precision and it will change depending on market conditions. The expected future programming of housing completions over a 5 year period will be a subset of the effective housing supply. It is likely to be smaller than the effective supply but again is impossible to define with any precision. The actual number of completions will be a subset of the likely programmed completions. Generally, this is less than the programmed number of completions but this can vary depending on local market circumstances.

5.35 The supply of effective housing land is a dynamic variable, and to some extent a matter of judgement. HLAs are unable to predict future programming with any accuracy and should therefore be considered in broad terms rather than as a precise measure of the amount of effective housing land.