Statistics.gov.scot improvement project: alpha user research report

Summary of Alpha user research and recommendations

This programme of user research is part of the Users Workstream of the Statistics.gov.scot Improvement Project – Alpha. This Alpha phase follows on directly from the Discovery phase. For the purposes of clarity and continuity, this report follows a similar structure to the Discovery report. Following this summary is an overview of the user research, including methodology, before the findings and recommendations are presented. The next sections offer further detail of the user testing sessions. Appendices A-C respectively contain the user journeys considered for testing, the scripts used for testing, and the resulting catalogue of user needs.

The overall purpose of the Alpha Users Workstream was to understand how the service could be improved to better address the needs and expectations of the service’s current and potential user base, through testing a set of redesigned Open Data Portal prototypes (Cobalt and Emerald) and data publishing platforms (CKAN Admin and a Workflow Manager prototype).

The user research primarily involved two rounds of one-on-one usability testing sessions with 29 participants from various backgrounds, and with a range of accessibility requirements. Participants included statisticians, analysts, policy advisers, data publishers, commercial users, and citizens with varying interest in data and statistics. Complementing this largely qualitative, moderated research approach was a largely quantitative, unmoderated approach, being a questionnaire including an established measure of usability (the System Usability Scale). This received 21 responses, resulting in 50 in total.

Analysis of the data indicates that the new designs are well-regarded, welcome improvements on the current systems, with Cobalt and Workflow Manager broadly seen as the more user-friendly prototypes of the four.

In summary, for an Open Data Portal, Users want:

• minimal jargon and plain-English labelling wherever possible
• efficient, intuitive, accessible search (e.g. typo-tolerant, fuzzy string searching)
• a simple, accessible preview of tabular data
• example visualisations with clear labelling (e.g. an example graph with uncoded x and y axes)
• prominent and useful metadata (including easy-read data dictionaries)
• short paths to multiple data download formats and APIs
• quick access to key statistics about their local area

In summary, for a data publishing platform, Data Publishers want:

• a reliable, efficient, accessible, and logical service
• clear guidance (in context) and help documentation
• minimal jargon and plain-English labelling where appropriate
• clear progress indicators with the option to save progress
• automated quality assurance (QA)
• preview and/or staging before publishing

As suggested in Discovery, an iterative approach to deploying and improving both the portal and publishing platforms is recommended for the Beta phase, beginning with the basics of easier publishing, updating, finding and accessing datasets, before moving onto more advanced features such as visualisations, automated QA and local statistics. Further testing of the information architecture is recommended to test labelling during Beta. Across all of the above, it is vital that both the open data portal and data publishing platform are as accessible as possible to a wide range of users, including disabled users and users of assistive technologies. A full accessibility audit to WCAG 2.2 (AA) standard should also be undertaken during Beta, with failures prioritised and addressed as required.

The following section provides an overview of the research, including details of the methodology.