Scottish Business Sentiment Index

2 Data

To construct the underlying dataset used for the analysis below, articles are collected from Factiva, an online database that archives content from a broad range of news sources. The reliance on Factiva as a data source aligns with recent advancements in sentiment research, where structured news data has been leveraged to create business sentiment indices (Seki, Ikuta, & Matsubayashi, 2022). The original selection focuses on retrieving articles from sixteen key leading British newspapers, including ten national newspapers—The Times, The Sunday Times, The Daily Telegraph, The Daily Mail, The Daily Express, The Guardian, The Mirror, The Sun, The Northern Echo, and The Evening Standard—alongside six major Scottish newspapers—The Independent, The Scotsman, The Herald, The National, The Courier, and The Press and Journal.

Specifically, to extract the articles from the newspapers of interest, the following query is used in Factiva to search for newspapers: 'gh|courir|sc|ind|ns|nrco|thesun|dmirr|grdn|theexp|mosm|daim|dt|st|t|fabp'. This query yields twenty-one newspaper titles, including: Aberdeen Press & Journal, Daily Mail, Evening Standard, London Evening Standard, Mail on Sunday, Mirror, The Courier, The Daily Express, The Daily Mirror, The Daily Telegraph, The Evening Standard, The Guardian, The Herald, The Independent, The Mail on Sunday, The Northern Echo, The Press and Journal, The Scotsman, The Sun, The Sunday Times, The Times.

Some of these titles are used as alternative names for newspapers or refer to a sister publications, i.e. the Evening Standard and London Evening Standard are alternative titles for The Evening Standard; the Aberdeen Press & Journal is an alternative title for The Press and Journal; The Daily Mirror is an alternative name for the Mirror; and the Daily Mail is a sister newspaper with the Mail on Sunday, alternatively known as The Mail on Sunday. After consolidating these groupings, fifteen of the sixteen originally selected newspapers of interest are confirmed to be included in the dataset. The National is absent from the search results, possibly due to Factiva’s classification criteria, which may not systematically categorize articles from this newspaper.

Next, using Factiva’s classification system, all articles in the database categorised under “Corporate/Industrial News” and tagged as related to Scotland are identified and retrieved. This procedure yields a total corpus of 205,630 news articles covering the period from January 1, 2006, to September 30, 2024.^[1]

Figure 1 illustrates the annual evolution of the total number of news articles by newspaper title. Notably, a significant decline in the total volume of articles is observed after 2010.This drop can be attributed to several potential factors, including changes in classification standards by Factiva, which likely resulted in a reduced volume of articles being accessed post-2010. Additionally, several key shifts in the media landscape, industry practices, and broader societal trends might have also played a role. For instance, the rise of digital platforms and the decline in print media have drastically reshaped news production and consumption. Traditional newspapers scaled back print articles as digital formats became dominant, with cost pressures and reduced advertising revenue driving a focus on fewer, more impactful stories. This shift, combined with click-driven online content, reduced long-form reporting and regional coverage.

Notes: The figure plots the yearly number of news articles by newspaper title that the Factiva database classifies as related to Scotland and under the “Corporate/Industrial News” category when using the following search query: 'gh|courir|sc|ind|ns|nrco|thesun|dmirr|grdn|theexp|mosm|daim|dt|st|t|fabp'. The total corpus consists of 205,630 news articles covering the period from January 1, 2006, to September 30, 2024.

Additionally, content priorities shifted during the 2010s, with major political events like the Scottish Independence Referendum and Brexit dominating coverage at the expense of other topics. Global events, such as the 2008 financial crisis and the COVID-19 pandemic, disrupted reporting, with attention focused on public health and government responses.

In the same spirit, Figure 1 also highlights variations over time in the distribution of articles across newspapers, particularly among Scottish publications. However, as depicted in Figure 2, the overall share of articles from Scottish newspaper titles has remained relatively stable, averaging approximately 85% throughout the sample period. This share ranged from 90% at the beginning of the period to 82% at the end, while the remaining 15% of articles were contributed by the ten selected non-Scottish UK newspapers.

Notes: The figure plots the yearly share of news articles by newspaper title that the Factiva database classifies as related to Scotland and under the “Corporate/Industrial News” category when using the following search query: 'gh|courir|sc|ind|ns|nrco|thesun|dmirr|grdn|theexp|mosm|daim|dt|st|t|fabp'. The total corpus consists of 205,630 news articles covering the period from January 1, 2006, to September 30, 2024.