Publication - Research and analysis
Agricultural tenancy data collection for Scotland: methodology report
This report explains the research methods used to develop and test improved agricultural tenancy questions, including reviews, interviews, workshops and surveys that informed the final recommendations.
Part of
Appendix D: Complete visual heatmap
The complete visual heatmap for all three question sets, summarising their relative performance across key usability criteria, is provided below. A detailed commentary on the factors influencing the usability of Set A (JAC) and Set B (SAF) can be found in the accompanying report.
| Usability considerations | Set A (JAC) | Set A Explanation summary | Set B (SAF) | Set B Explanation summary |
|---|---|---|---|---|
| Respondent motivation to complete | Red: Performs poorly | Respondents perceive little or no personal benefit from completion | Green: Performs very well | Respondents are motivated to complete due to connection with financial subsidies |
| Time or effort to complete | Amber: Performs moderately | Accurate completion may require consulting tenancy agreements or other records | Green: Performs very well | Respondents often use land agents, and tenancy records may already be accessed for other SAF questions |
| Ability to identify unfamiliar tenancy types | Amber: Performs moderately | Questions assume respondents either know their tenancy details or can access relevant records | Amber: Performs moderately | Questions assume respondents either know their tenancy details or can access relevant records |
| Response reliability | Amber: Performs moderately | There are no penalties or follow-up if questions are left blank or answered incorrectly | Green: Performs very well | Respondents have direct incentive to provide correct and consistent information |
| Data granularity | Amber: Performs moderately | Data is captured at the farm holding level | Green: Performs very well | Data can be captured at the level of individual land parcels |
| Sample coverage | Green: Performs very well | The number and type of respondents reached is consistent with the previous 2021 JAC questions | Amber: Performs moderately | Some agricultural holdings may not be captured if relying solely on the SAF (further analysis required) |
| Operational feasibility | Green: Performs very well | Questions can be reinstated following the format and approach used in the 2021 JAC | Amber: Performs moderately | Questions would need to be incorporated into the SAF as a new section |
| Usability considerations | Set C (JAC) | Set C Explanation summary |
|---|---|---|
| Respondent motivation to complete | Red: Performs poorly | Respondents perceive little or no personal benefit from completion |
| Time or effort to complete | Amber: Performs moderately | Given the complexity of tenancy legislation, the branching logic is lengthy and could take time, especially if reporting multiple tenancy types |
| Ability to identify unfamiliar tenancy types | Green: Performs very well | The branching logic asks simple, indirect questions to help respondents report unfamiliar tenancy types without prior knowledge required |
| Response reliability | Amber: Performs moderately | There are no penalties or follow-up if questions are left blank or answered incorrectly |
| Data granularity | Amber: Performs moderately | Data is captured at the farm holding level |
| Sample coverage | Green: Performs very well | The number and type of respondents reached is consistent with the previous 2021 JAC questions |
| Operational feasibility | Red: Performs poorly | Given the complexity of tenancy legislation, the branching logic is complex and highly likely to be difficult to operationalise |
Heatmap scoring was based on the following scale:
- Green: Performs very well; meets objectives with minimal risk, effort, or barriers. Strong likelihood of positive outcomes.
- Amber: Performs moderately; meets objectives with some effort, uncertainty, or minor barriers. Partial risk of issues or limitations.
- Red: Performs poorly; fails to meet objectives, with high effort, uncertainty, or significant barriers. High risk of issues or failure.
Contact
Email: agric.stats@gov.scot