We are testing a new beta website for gov.scot go to new site

Public Procurement Reform - a rapid evidence review


Chapter 1: Introduction and methodology


1.1 The aim of the evidence review is to examine existing material on procurement reform, drawing on policy and research evidence from Scotland, the UK and EU, to help inform the development of the Scottish Government Procurement Reform Bill.

1.2 The objectives are threefold: (i) to briefly outline the Scottish and EU strategic policy contexts in which the procurement reform agenda is located, underscoring the rationale for reform and its key components; (ii) to identify some developments in procurement reform in Scotland and the UK; and (iii) to examine a range of evidence including Scottish, UK and EU policy documents, independent reviews of procurement and international comparative studies and surveys of procurement policy and practice. The findings from the analysis of the Scottish Procurement Reform Bill Consultation is published separately, however, cross reference is made here to some of its main findings in an attempt to consider the implications of the wider research and the consultation response analysis, for policy and legislative development (cf Mulholland et al, 2013a & b). This evidence review combined with regulatory impact assessments (and the Starting Gate Review Process) forms part of the policy and legislative options appraisal process to help identify where intervention/action is justifiable (HM Treasury, 2003).

1.3 Stages 1 to 3 of the options appraisal process involve asking the following: Stage 1 - is there an identified need and is the proposed intervention likely to be worth the cost? What are the pros and cons of taking action and the risks of not acting? Stage 2 - what are the desired outcomes and objectives of an intervention? What full range of options might be available to enable them to be delivered? Stage 3 - what are the range of policy options available? Is there a 'do minimum' option? (HM Treasury, 2003). Utilising a range of evidence as part of the appraisal process should help identify what is necessary for the Bill, what is necessary for policy and where and how any interventions will bring additionality. Equally, drawing on a range of evidence should assist in establishing the resource and administrative implications of any legislative and policy changes. Arguably, there is a need to conduct a cost benefit analysis as part of estimating the costs and risks of intervention and action and this is beyond the scope of this evidence review.

Scope and structure of the evidence review

1.4 Chapter 2 covers the strategic policy and legislative context of the procurement reform agenda. Chapter 3 explores some of the evidence from financial data, systems capability development and audit reviews in examining developments in Scotland and the UK on procurement reform. Chapter 4 outlines some key findings from the Scottish Procurement Reform Bill Consultation analysis with cross reference to connected themes from the wider evidence. Chapter 5 explores the existing evidence structured around the six themes of the Procurement Reform Bill and Chapter 6 outlines conclusions from the review.

1.5 The themes identified in the evidence are structured around the six themes in the Scottish Government Procurement Reform Bill Consultation. Where gaps in the evidence exist, this is identified and discussed. The six themes are: (i) Public Procurement processes are transparent, streamlined, proportionate, standardised and business friendly; (ii) Making it easier for business, particularly, newer businesses, SMEs and the third sector, to access public contract opportunities and sub-contracting opportunities; (iii) Smarter use of public procurement to encourage innovation and growth; (iv) Taking account of social and environmental sustainability issues through public procurement; (v) Dealing with inappropriate conduct and poor performing suppliers; and (vi) Application and compliance (Scottish Government, 2012a).


1.6 A desk based evidence review was conducted covering a range of material including Scottish, UK and EU policy documents and legislation, independent reviews of procurement, international comparative studies, case studies and evaluations. In gathering material for Chapter Three on systems, discussions were held with five colleagues in the Scottish Government working in the area of capability and the Single Point of Enquiry (SPOE) to gain a broad understanding of the aims of the systems and processes. The review of the evidence is not exhaustive and focuses more specifically on documentation from 2000 onwards (broadly taken as the commencement of the procurement reform process).

1.7 The review does not attempt to detail the procurement processes and systems of EU Member States as this type of descriptive content is documented elsewhere (Bianchi & Guidi, V (eds), 2010; European Commission, 26 October 2012; Kahlenborn et al, 2011; OECD, 2010; PWC, 2011). Instead, the review draws on existing and emerging evidence on policy and best practice in relation to the parameters of the policy and legislative proposals within the Scottish procurement reform context. It is not a Systematic Review and, therefore, it is not comprehensive and does not attempt to critically analyse the methods used in all the studies reviewed or assess the quality of all the evidence. It is out with the scope of this exercise to attempt to critically assess qualitatively and quantitatively progress on the reform agenda by examining procurement systems, processes, capability and professional capacity and their efficiency and effectiveness. The evidence review was conducted between November 2012 and March 2013.


1.8 In terms of definitions, some discussion is made in the relevant sections on the contestation surrounding innovation, demand side innovation and sustainability.

Context setting and highlights from the evidence

Scottish and EU procurement reform policy contexts

1.9 In terms of objective (i) of the evidence review - regarding strategic policy contexts, there is congruence between the Scottish, UK and EU policy drivers and objectives. The Scottish Government Procurement Strategic Plan (2012-13) underscores its four main priorities in procurement reform:

  • embedding sustainability in all we do;
  • improving suppliers' access to public contracts;
  • maximising efficiency and collaboration;
  • delivering savings and benefits (Scottish Government, 2012e).

1.10 The key components of the procurement reform agenda can be categorised into two broad strands: (i) improving procurement systems and processes (especially e-procurement, e-commerce and standardising IT tools); and (ii) improving the profession (capacity, expertise and capability building) both requiring resources, training and cultural change.

1.11 Central to the procurement reform project at an EU level are: the themes of encouraging sustainability and community benefit clauses (environmental, economic and social); improving access to public contracts including access by the private and third sectors; and increasing collaboration at both contracting authorities and supply sides. This is set in the economic context of the need to ensure savings during a period of fiscal constraint. Evidence suggests that all EU member states (thereafter, MS) are aware of the broad reform principles and the value of utilising public procurement as a vehicle for achieving wider policy objectives. Some countries are more advanced than others in encouraging and requiring environmental, social and economic benefit clauses in public contracts (Kahlenborn et al, 2011). The older MS tend to be more advanced in their use of environmental, social and innovation clauses and requirements than newer MS. The Netherlands, Sweden, Denmark and the UK tend to be the front runners in considering environmental, social and economic benefits in procurement (Kahlenborn et al, 2011 & European Commission, 2010b: 82). However, practice varies across the EU with some countries enforcing above the EU threshold approaches and regulation to below the threshold contract values, and others devolving the management of contracts to local contracting authority level.

1.12 The OECD (2010) found that, on the whole, the majority of MS regulate public procurement above and below EU thresholds within the same Act and require that contracts below the EU thresholds are based on open and fair competition procedures, although there is some variation in practice and a good degree of discretion at MS level (cf also European Commission, 26 October 2012). The centralisation and specialisation of public procurement varies across MS (European Commission, 26 October 2012: 26). MS with no regulation still use competitive tenders below the EU thresholds (OECD, 2010: 7). The OECD also found that publication procedures and processes vary across MS but on the whole, MS publish contract notices on national portals or websites (OECD, 2010: 16 & cf also European Commission, 26 October 2012). However, in terms of the limits of current data, the OECD highlights that, to date, a systematic review, of how MS have developed and designed their legal frameworks outside the scope of the EU Directives, has not been undertaken (OECD, 2010: 5).

Procurement reform - some developments in Scotland and the UK

1.13 Regarding objective (ii) of the evidence review - some developments in Scotland and the UK on procurement reform - improvements have been made in terms of capability and capacity within procurement. There is, however, still room for improvement in terms of cultural change, systems and data development and management. These issues are discussed in Chapter 3.

Effective project and contract management

1.14 An emerging theme from the evidence that connects all the six themes is that good, thorough, rigorous project and contract management processes are crucial. Strict procedures at the PQQ stage (the screening stage before full invite to tender) could provide solutions to some of the challenges/problems regarding improving transparency, increasing access of SMEs and Third Sector and including community benefit clauses, regulation and compliance. Equally, good project and contract management could help hold suppliers to account and thorough monitoring of contract criteria could help capture outcomes of contracts.

Lack of management and evaluation data on outcomes and impact

1.15 Bringing objectives (i) and (ii) above together in considering objective (iii) of the evidence review - a considerable volume of material exists at the strategic policy level within Scotland, the UK and the EU outlining and reinforcing the policy direction of travel (see references list). Equally, there are a number of EU wide studies and surveys charting and comparing the procurement systems and approaches to managing and regulating above and below EU threshold level contracts. These studies, on the whole, are descriptive (and often MS self-reported) with varying levels of detail on the systems, allowing the reader a brief snapshot of the procurement systems but with little or no detail on the effectiveness, outcomes and impact of differing approaches. Some of the literature (policy and research) highlights the dearth of management data and evaluation data on outcomes and impact at national and EU levels (Audet, 2002; Audit Scotland, 2009 & 2012a; Edler & Georghiou, 2007; Edler et al, 2011; Edler et al, 2012; European Commission, 2010b; European Commission, 26 October 2012; Georghiou, 2007; Izsak & Edler, 2011; Kahlenborn, 2011; McClelland, 2006; McFarlane & Cook, 2008a & b & 2008; OECD, 2011; Westminster Sustainable Business Forum, 2008).

1.16 As long as ago as 2002, Audet (2002) pointed to the lack of reliable data and the existence of only a few studies within and across member states. He noted that the lack of detailed and consistent measurements by governments have created information gaps making it difficult to measure and compare public procurement activities and markets across member states (Audet, 2002: 150 - 153 & 171). Audet argues that this gap needs to be addressed and that there is a need to ensure that national data is gathered on the basis of harmonised procedures across the EU (Audet, 2002: 153). Some responses from the Procurement Reform Bill Consultation Analysis also capture this point regarding lack of data on outcomes (Mulholland et al, 2013a & b).

1.17 Where some case study material has been identified, there is more descriptive content than critical appraisal of the processes and outcomes of utilising various procurement systems and processes. Therefore, it is difficult to determine the utility and feasibility of translating and replicating policy measures from one context to another.

The need for an inter-disciplinary approach to building the evidence base

1.18 Specifically on the six themes of the Procurement Reform Bill Consultation, more evidence was uncovered on themes (2) SME and Third Sector access, (3) smarter use of public procurement to encourage innovation and growth, (4) taking account of social and environmental sustainability issues through public procurement, and (5) dealing with inappropriate conduct and poor performing suppliers. Less information was found on (1) transparency and (6) application and compliance. Although (5) and (6) overlap and theme (1) is often implicit within the other themes. Across the EU MS, the limited evidence on some of the themes of the procurement reform agenda suggests the (i) need for data development and data gathering at public sector organisational levels, in addition, (ii) to exploiting wider, cross social science and business management bodies of evidence and (iii) conducting further research.

1.19 All six themes are substantive topics and could merit further exploration individually and to examine the inter-connections between all six. The aforementioned three pronged approach could be helpful in building the evidence base to inform policy intervention/direction of travel and uncover what, if any, evaluation data exists on policy interventions especially given the challenge that public procurement faces in achieving wider socio-economic and environmental policy objectives.

1.20 The evidence search could be broadened out to include literature/established bodies of work, for example, within criminology for suggestions on themes (5) and (6) for improving regulation, monitoring and compliance and material from sociological and economic studies on employability/employment for suggestions on (4). I have touched upon a body of evidence within economics that has provided material for theme (3) on innovation but this could also be expanded. Another disciplinary area for exploration could be business management and in particular, business management on sustainable procurement and wider procurement practitioners' journals. Of course, covering this wider range of literature will be unable to demonstrate the direct impact of public procurement unless specific independent evaluations of public sector contracts and processes are uncovered in the literature. Primary research could involve conducting qualitative interviews with public sector organisations and a quantitative survey of the public procurement landscape could help shed some light on the level of progress made so far on structural, cultural and behavioural changes recommended in previous reviews of public procurement.