Accuracy and Reliability

This section discusses how accurately and reliably these statistics portray reality.

Data quality

The quality of information collected from each farm is very high as data are based on fully reconciled farm accounts. Data for each farm is also validated against a comprehensive set of quality assurance checks.

The potential for processing errors is regarded as low risk. This is due to much of the collection being based on reconciled accounts, the extensive use of cross-checking validation routines and that the vast majority of farms have previous records in the survey which are used to identify inaccuracies in returns.

Results are examined alongside previous outputs and related evidence from alternative sources in order to ensure that the data and methods being used are reliable.

Potential outlying results that may have an impact on the analysis are also identified. Such outliers, when identified, may be excluded from specific analysis to ensure that the results are representative of the population being described.

Although the quality of information for each farm business in the survey is considered to be high, relatively low sample sizes do mean that the results are subject to a degree of uncertainty in terms of representing overall national averages by farm type. Estimates of the degree of uncertainty associated with average income estimates are available in Sampling error and bias.  Weighting is applied to account for representation as best as possible, more information is available in Weighting.

In some cases, accounts may not be finalised until after the deadline for submission of data. In such cases estimated records are updated and the published figures are revised in the following year. In this sense, the first release of data for a particular year may be regarded as provisional.

Basis Period Reform (BPR) methodology changes and improvements to analytical processes

Farm business income estimates for 2023-24 and 2024-25 are affected by BPR and some improvements to analytical processes. More information is included in Methodology changes and corrections.

Both BPR and improvements to data processing have no impact on the comparability of 2023-24 and 2024-25 estimates with the timeseries (data back to 2012-13). The size of the impact on estimates remains small compared with the margin of uncertainty around survey results.

Sampling effects

The sampling strategy of the FBS is based on a stratified simple random sample and is effectively designed as a panel survey with little change in the membership of the sample between years. The amount of sample change varies year on year.

In most years, sample turnover is small. In some years, a larger turnover or turnover of farms with specific characteristics will mean that changes in average farm income are more strongly influenced by sample variation.

Results for 2023-24 and 2024-25 are impacted by larger changes in sample characteristics. The survey actively recruited an increased number of smaller farms in order to improve proportional representation of the sample against the wider population. This ensures that individual farms do not end up representing unrealistically large proportions of the whole population when weighting is applied and ensures that results are as accurate and reliable as possible.

As farm income varies each year for all farms, it is difficult to determine the size of the impact of sample turnover on reported estimates. The changes in the average reported figures as a result of sample change are also likely to be smaller than these might be for a random sample drawn from the population each year. Where changes are strongly attributed to changes in the sample representation, these are reported on alongside the official statistics.

Sample coverage of general cropping farms with potatoes

Limited sample coverage of general cropping farms growing potatoes is a known weakness of the data. The survey represents 99% of the June Agricultural Census potato area for farms that meet the survey criteria. However, the number of potato farms in the sample may not reflect the population proportion of potato enterprises on general cropping farms. A breakdown of ware and seed potatoes is not released, as survey coverage limits the ability to accurately reflect these separate enterprises.

Farms with large potato enterprises can have large incomes as potatoes are a valuable crop. This means some general cropping farms in the sample will have very high incomes while others will be much lower. As a result, general cropping farms have a wide spread of incomes compared to many other farm types, which adds more uncertainty to estimates for general cropping.

We are actively seeking to recruit more small to medium potato farms to improve the quality of the results.

Sampling error and bias

A total of 402 farms were in the 2024-25 sample. This accounts for 4% of the total relevant agricultural holdings in Scotland. As the survey does not capture data from the entire population, FBI estimates are susceptible to sampling error.

Sampling error is the difference between the result from a sample and the true result for the entire population. It occurs when the sample characteristics differ from the population characteristics. This can occur randomly or be caused by bias towards or away from particular characteristics. Known sample limitations or impacts are reported. Including known limitations on potato coverage and impacts to data as a result of improving sample coverage.

As the data collected through the FBS is of a highly sensitive nature, the refusal rate of farms approached to participate is high. Non-responders (farms refusing to participate) may have different characteristics to responders (farms willing to cooperate), which could lead to biased results. Currently there has been no assessment of non-response bias in the FBS for Scotland.

To quantify sampling error and provide a measure of uncertainty, 95% confidence intervals for estimated means have been produced. Figure 1 displays average FBI for each farm type with confidence intervals displayed as error bars. These data are also available alongside the publication under Supporting Documents. 

Figure 1: Average 2023-24 and 2024-25 farm business income estimates and 95% confidence intervals as error bars by farm type. 2024-25 prices.Statistically, a confidence interval provides a plausible range for the true population mean. The 95% confidence interval indicates the range within which we expect the population mean to lie for 95% of similarly constructed samples.

Narrower confidence intervals typically indicate larger sample sizes or less variability within the sample, thereby offering more precise estimates of the population mean. Conversely, wider confidence intervals often result from smaller sample sizes or greater sample variability, signalling less precision.