We are testing a new beta website for gov.scot go to new site

Data Publication

Publication of spend over £25k: content and format

The Public Services Reform (Scotland) Act 2010 imposes duties on the Scottish Government and listed public bodies to publish information as soon as is reasonably practicable after the end of each financial year.

Administrations such as the UK, US, Canada have found that the publication of public data has benefits both in improving transparency and accountability, and in encouraging economic activity through reuse of that data. To achieve these benefits, however, it is clear that data must be published as usable data: that is, data that follows certain standards, and which is released in formats that can be automatically processed. Without that, it is difficult for anyone, and impossible for many, adequately to review or analyse the data we release, let alone reuse it effectively.

While there is no prescribed format for the publication of the information, the Scottish Government will provide advice and recommendations on good practice on these pages. This advice will develop over time: and we hope to include or link to tools which you will be able to use to help make this publication easier and more effective.

These pages represent current proposals: we are happy to have feedback on the proposals and questions relating either to the technical standards or to the principles of transparency and open data as those policy areas develop.

What format should we use?

At present, we recommend publishing data in a simple comma separated value (CSV) format. This format is easily output by, and can easily be read by, spreadsheet packages such as Microsoft Excel (and can be created by pretty well all standard reporting packages). You may wish to publish a "human readable" version in native spreadsheet format as well - but for automated harvesting, CSV is the most effective simple format.

CSV (comma separated value) format can be created and read on pretty much any computer or device. It will open by default in Microsoft Excel, or any alternative spreadsheet package installed. If none, it can be opened in any text editor or word processor. It is completely non-proprietary, requires no plug-ins, and can be read by machines as well as by people.

Isn't PDF better?

Adobe PDF files have been used for some time as a way of providing a printable format which preserves layout and style. However, because the format is designed specifically to emulate paper, little use that can be made of them except to print them out or read them onscreen. Thus, without conversion to another format, you cannot sort data in a PDF into a different order; you cannot extract the top 10 amounts; you cannot combine consecutive lists into one cumulative list. More important, as public bodies north and south of the border increasingly publish spend reports, you cannot compare spend of one organisation with that of other bodies. Thus, rather than being helpful, the PDF format is being seen as obscuring the data and making it harder (but not impossible) for interested citizens to understand, analyse, and challenge published data - and hence as opposed to the very idea of transparency.

How often should we publish?

The Act requires annual publication, but the Scottish Government has taken the decision to publish its spend figures monthly. This is good practice: it means that public scrutiny of spend can happen through the year, and that the public - and we ourselves - can learn about how spend compares and where savings might be possible in as near real-time as possible. Monthly production of an additional standardised report from your financial system should not tax resources greatly.

What data should we publish?

A standard set of fields is being worked on, and an initial proposed set is given below. This will not be a heavyweight standard: it is largely common sense. Using it - and promulgating it among other Scottish public bodies - will help application writers and analysts use data across the public sector, answering questions such as "How much do public bodies spend with IBM?", "Who has spent most on hospitality over the past 2 years?". In line with this standard, the fields we should use are:

Account code

Account code description

Narrative (free text explanation)

Supplier code

Supplier/payee name

Company registration number (and/or VAT number)

Payment value

Payment date

Cost centre code

Business unit

Licensing your data

the Scottish Government and its agencies licence information under Crown Copyright terms. Although these terms do not cover the reuse of data (database right), it should be taken that anyone has a right to reuse our data for any purpose subject only to attribution. Exact licence terms are being developed, and will be available shortly on these pages.

Other public bodies will own their own copyright, but as a matter of good practice you should ensure that your licence terms and conditions are clear and especially that they do not restrict reuse for commercial or non-commercial purpose.

Tell us about your data!

The requirement of the Act is simply that each public body publish the relevant statements at the appropriate time. However, it will assist those wishing to make use of this data if links to the web publication of these statements can be found in one place, and we propose to put them on these pages. This does not imply that the Scottish Government is taking any responsibility for or in any way approving those statements - that is a matter for each body - but simply that as a public service we will try to ensure that they can all be found from a single point. We would therefore ask you, as a matter of good practice, to notify us of the link (URL) to your reports, and we will publish them together on here.

Where does it go from here?

The Act gives powers to Ministers to modify the reporting requirements, and there is anyway every likelihood that there will be demand from citizens to see more data about the activity and performance of our public bodies. The best way to cope with these demands, and to do so without committing more and more resources to the task, will be to share tools, ideas and experiences.

To comment on or request more information regarding the information on these pages, or if you wish to join the community of those developing this guidance, please email ben.plouviez [at] scotland.gsi.gov.uk