We are testing a new beta website for gov.scot go to new site

Public Procurement Reform - a rapid evidence review


Chapter 3: Procurement reform: some developments in Scotland and the UK

Chapter 2 outlined why reform is necessary. This chapter outlines some developments in practice in Scotland and the UK and explores where gaps and challenges still exist.

Procurement reform – some developments in Scotland and the UK

3.1 Drawing on secondary management data only, some financial evidence (between 2009-12) within the Scottish public sector context suggest that 'significant progress' has been made in the first three years (2006 to 2009) with estimated savings of £800m and £312m in 2009-10 alone (Scottish Government, 2010b: 1 & 2010a & cf Audit Scotland, 2009). The Scottish Government's Programme for Government 2012-2013 states that £1.2 billion in efficiency savings have been achieved through Public Sector Procurement Reform (Scottish Government, 2012b: 35). It outlines that the Scottish Government will continue to pursue savings through procurement activity, increase access of SMEs and the third sector to public sector contracts and will continue to build on its shared service approach to eCommerce systems, citing the increasing use of Public Contracts Scotland (PCS) portal (Scottish Government, 2012b: 35, paras 29-30). In 2010, the Cabinet Secretary for Finance and Sustainable Growth underscored the "[…] importance of procurement as a key strategic enabler of service design and service delivery […]" (Scottish Government, 2010b: 1).

3.2 The Scottish Government (2009) found that some evidence suggests that cultural change is needed within the public sector and there is the need to build sustainability into corporate cultures. Other evidence identified the issue of poor levels of data and systems for monitoring and capturing performance, spend and savings and that there is a clear need to improve data capture in public procurement (Audit Scotland, 2009: 4-5 & cf Audit Scotland, 2012a; McFarlane & Cook, 2002a &b, 2008; Westminster Sustainable Business Forum, 2008). With specific reference to local authorities, Audit Scotland (2012a) outlined why good cost data is necessary for understanding performance at local authority level. It argued the need for cost information plus a focus on monitoring and measuring outcomes for service users and communities (Audit Scotland, 2012a: 3 - 5). The Scottish Government (February 2012 Refresh) outlined some achievements regarding the Scottish Government's seven Cs including improved procurement capability, competitiveness and collaboration.

Developing systems capability and data collection in Scotland

3.3 In Scotland, a number of systems and processes for data collection in the Scottish Government have developed in recent years in response to recommendations of procurement reviews and as part of the reform programme. Some systems are briefly outlined here as part of considering the ways in which data are currently collected and to explore ways of exploiting existing data sources and to consider areas for development.

3.4 Some areas recommended by McClelland (2006) and subsequently developed in Scotland included, inter alia, the creation of Centres of Expertise, the Scottish Procurement Policy Handbook, a single public sector electronic portal, a Single Point of Enquiry (SPOE), and the development of the Scottish Pre-Qualification Questionnaire (PQQ). It was also recommended that public sector organisations should develop an advanced procurement plan (see later in this chapter reference to Procurement Capability Assessments (PCAs) conducted annually across the Scottish Public Sector and designed to capture the stage of procurement capability development of each organisation) (McClelland, 2006: 45-47 & cf the Scottish Procurement Policy Handbook, 2008a: 2 & 4).

The Scottish Procurement Information Hub

3.5 The Scottish Procurement Information Hub (thereafter SPIH) collects management information and analyses spend by organisation, type of goods, services or works procured and geographical location of suppliers (Scottish Government, 2008a: 32). One hundred and twenty two (122) public sector organisations have a licence to use the SPIH and the data published on it is only accessible by all 122 organisations, the four Scottish Centres of Expertise, the police serious crime squad and the Improvement Service. The Contracts Register is held in the SPIH, is updated daily and links with the Public Contracts Scotland database. Information on the Public Contracts Scotland database can be accessed externally by all.

3.6 In terms of information on the SPIH, the existing data allows us to identify (from data provided by the 122 organisations) public sector spend on SMEs, local spend, by geographical boundaries and by collaborative suppliers. It is also possible to analyse spend by supplier age (that is, by age of business). The data reveal that the public sector is only, very rarely contracting with suppliers less than three years old and that the larger value contracts are with older suppliers (that is, over three years old). For example, 38 out of 1300 suppliers, who are currently in receipt of public sector contracts are less than three years old (Scottish Procurement Information Hub, 2013). Equally, suppliers under three years old and micro suppliers tend to obtain lower value contracts than older and larger suppliers. The biggest areas of public sector spend are in construction and social care and anecdotal evidence suggests that these two areas could be better resourced in terms of procurement expertise and that there is possible room for improving the controls and governance of these sectors in receipt of public sector contracts. This would also support the argument that concentrating on tightening up the PQQ and specification stages, in addition, to the full project management process could, be preventative in managing risks of unsuccessful processes and outcomes for any given contract (and I return to this point later).

3.7 In terms of SME access to public sector contracts, the SPIH notes that spend on SMEs has stayed at around 45/46% over the last five years across the whole of the public sector (Scottish Procurement Information Hub, 2013). There has been a slight improvement in spend on 'micro' businesses, which is positive given that at the European level, evidence suggests that 'medium' size businesses tend to do well regarding public sector spend but the challenge that remains is for 'micro' businesses to win public sector contracts (European Commission, 2008). One limitation of the data is that it only indicates spend data, at an aggregate level, and does not hold sub-contracting data. Therefore, it might be the case that more SMEs actually access public sector contracts as sub-contractors.

3.8 A core set of Best Practice Indicators (BPIs) have been developed in an attempt to measure progress of the reform programme in a consistent way across contracting authorities (Scottish Government, 2008a: 32). The BPIs are designed to capture annual cash savings made by the organisations in the SPIH. The BPIs variously capture cash and non-cash savings, spend with collaborative and contracted suppliers, demographic data on procurement staff with appropriate procurement qualifications, the number of people in training, and the range of electronic procurement in operation. Guidance exists on a calculation method to determine cash savings.

3.9 There is now approximately seven years of data in the SPIH and it is possible to identify some patterns/trends in spend data as outlined in paragraph 3.6 above. However, it is impossible to explain why patterns are occurring and as originally conceived, the database does not attempt to capture information on impact and outcomes associated with public sector spend.

Public Contracts Scotland

3.10 Public Contracts Scotland (thereafter PCS) is an online procurement advertising portal, which was introduced in 2008 and can be accessed externally. Its aim was to raise visibility in advertising public sector procurement opportunities and reporting award notices. Although there is a requirement to advertise in OJEU for public sector procurement opportunities above EU threshold levels, there is no obligation to advertise below or above EU threshold levels opportunities in PCS. The majority of public sector purchasers do, however, use the portal as there are benefits to doing so, including reducing costs and gaining access to the supplier base. There is no penalty if public sector organisations do not advertise.

3.11 Over 400 organisations advertise on the portal and over 800 have registered as a buying organisation (although their status is not vetted and verified). The portal gives us data on suppliers and the makeup of suppliers and SMEs. Information from the portal does not reveal the value of contracts. There are over 60,000 suppliers registered on the system. Of the contracts awarded (contract opportunities and quick quotes) on PCS during the calendar year 2012, 82% of contracts were awarded to SMEs and 68% were awarded to Scottish SMEs (Scottish Government, 2013d: 3-4). These figures are aggregate data and do not indicate which SMEs (and how often) are winning public sector contracts via PCS.

3.12 For lower value contracts, the Scottish Government operates a system of Quick Quotes whereby purchasers can invite quotes from three or more potential suppliers. In the majority of cases, a purchaser then choses a supplier and awards the contract. There is no requirement to advertise Quick Quotes. Quick Quotes was introduced in 2009. The publication of award notices and award notices for Quick Quotes was introduced in 2010. This helps capture spend data but only for those purchasers and suppliers who use it. This information is pulled across to the Scottish Procurement Information Hub (SPIH) data.

3.13 PCS is not; however, a full and representative picture of the public sector procurement and supplier landscapes. It is self-reported data from purchasers and suppliers and data is not verified or checked for accuracy. Furthermore, the actual number of suppliers is not clear with the potential for double counting to occur given that individuals within organisations can register as suppliers as opposed to only allowing one organisation to register once and having one contact for the organisation. One suggestion that might help reach a more accurate figure for suppliers and avoid double counting is that the Dun and Bradstreet number, used on the portal as a unique identifier for companies, could be made mandatory[1]. There is no cost attached to suppliers obtaining a Dun and Bradstreet number.

Public Contracts Scotland (PCS - Tender)

3.14 The Public Contracts Scotland-Tender (PCS-Tender) electronic (e-sourcing) system was introduced in November 2011. The intention is to roll out the e-sourcing system across the whole of the Scottish public sector. The system is used by purchasers and suppliers to manage the procurement process and it stores all procurement tender process information and documentation including PQQ data for Routes 2 and 3 of the Procurement Journey. The aim is to build useful management information and for example, to check if purchasers are using consistent PQQ questions in a standardised template. Suppliers will be able to store their PQQ answers for future usage from June 2013. Approximately, fifty organisations have been trained so far in the e-sourcing system.

The Procurement Journey

3.15 The Procurement Journey provides online national guidance for all purchasers. It outlines guidance on three possible routes (1 to 3) dependent on level of risk and contract value with 3 being highest risk and offering more detailed guidance at this stage including information on contract and supplier management. The number of hits on the Procurement Journey can be identified, which indicates level of usage (cf Scottish Government, 2013b for a description of the Procurement Journey and PCS - Tender).

Procurement Capability Assessments

3.16 Procurement Capability Assessments (PCAs) were introduced in 2009. PCAs are assessments conducted by representatives of one of the four Scottish Centres of Expertise and involve Scottish public sector organisations being assessed annually, evidencing progress on a range of issues regarding capability (one of the core themes of the procurement reform agenda). Assessment is categorised according to four levels - 0 to 3 (Level 0 - non-conformance, Level 1 - conformance, Level 2 - improved performance and Level 3 - superior performance). These four category levels were adopted from McClelland's recommended schema (McClelland, 2006). In 2012, a total of 118 organisations were assessed. The completed score sheets are returned to the Scottish Government from the Centre of Expertise and the findings from the PCA process are reported to the organisations' Chief Executives (for their own organisation), the Public Procurement Reform Board (PPRB) and the Procurement Reform Delivery Group (PRDG).

3.17 The Flexible Framework Assessment Tool is outlined in the Scottish Sustainable Procurement Action Plan and it recommends that organisations conduct initial and periodic self-assessments against the flexible framework in order to assess levels of performance (based on 1 to 5) and areas for improvement. There is a question in the PCAs on the Flexible Framework Assessment Tool. The PCA findings are used as part of informing continuous improvement within the organisations involved. The findings from the PCA process are not published or reported externally; however, organisations within sectors do share ideas/best practice at sector specific capability events. The next step in progressing the sharing of best practice would be to encourage high scorers in the PCAs to share information on their best practice as part of helping all sectors improve their performance. At an aggregate level, Scottish Government has identified that all sectors involved in the assessment process have improved in terms of their procurement capability over the last four years (Scottish Government, 2013e unpublished).

3.18 As part of the assessment process, organisations should be gathering data throughout the year to evidence progress and improvement. The assessment itself is one day in duration. It is emphasised that it is an assessment and not an audit. An important caveat is that the information gathered is self-selected evidence from assessed organisations and it is conducted by the Centre of Expertise for their sector. Therefore, potential bias might exist in the process, in terms of the information selected to demonstrate progress and potential assessor bias might exist across sectors. In this respect, the accuracy of the data cannot be verified. One also needs to ask how success or progress is actually measured at an organisational level as it is unlikely to be measured consistently across all organisations, although perhaps across sectors, there will be more consistency. Furthermore, the assessment questionnaires have now been administered over four years but the content of the questionnaires has changed slightly over the first three years and significantly in year four (2012). This makes it difficult to compare data over the four years and to accurately measure progress as the change of questions and structure will have resulted in different responses. The PCAs methodology for assessing progress and questionnaires can be accessed online at http://www.scotland.gov.uk/Publications/2012/09/3651 (Scottish Government, 2013c).

Single Point of Enquiry (SPOE)

3.19 The Single Point of Enquiry (thereafter SPOE) was established in 2008 and is an informal service which aims to provide suppliers to the Scottish public sector with advice and guidance on public procurement regulations and practices. It can also be used to help broker resolution of disputes (cases) relating to tender exercises undertaken by public bodies in Scotland (Scottish Government, 2012i: 3).

3.20 In addition to providing on-going advice and guidance, the SPOE has seen a total of 163 cases, across a wide range of business sectors, since its introduction (Scottish Government, 2012i: 4). Data emerging from the cases processed highlights themes and patterns being raised by suppliers over time. Some issues raised over the previous four years concern all aspects of procurement tendering but three primary themes have emerged: visibility of contract opportunities; appropriateness and proportionality of assessment criteria; and effective post award feedback to suppliers. Clearly this feedback is useful in informing improvement activities and, in particular, the development of the PCS portal, standardised PQQ and the procurement journey (buyer best practice guidance) all focus on these three areas of concern.

3.21 However, key challenges for the SPOE are continued promotion of the service and definition of how it measures success. For example, do more or less enquiries to the SPOE signify success of the service and how does one attribute cause and effect to improved procurement practice? These areas are being addressed by the SPOE Champions Group as it continues to promote and develop the SPOE activity.

3.22 For example, in terms of taking action on PQQs, Rahman and Horner (2013) estimated the cost to the Scottish Public Sector of preparing and assessing PQQs for construction contracts. They calculated the cost to be approximately £1 billion per year (Rahman & Horner, 2013: 2). However, extreme caution should be exerted over the figures and conclusions reached as they are based in the findings of a survey distributed to 135 public sector construction clients with only 9 out of the 135 having responded. This is a very poor response rate. The findings are then combined with figures from another study conducted in 2006, of which the method employed and calculations deduced are not discussed and explicated (Rahman & Horner, 2013).

Where do gaps and challenges still exist?

Gaps exist in management, cost and evaluation data

3.23 Spanning the last ten years, a range of evidence suggests poor levels of data and systems for monitoring and capturing performance, spend and savings at Scotland, UK and EU levels (Audit Scotland, 2009: 4-5, Audit Scotland, 2012a; Kahlenborn et al, 2011; McFarlane & Cook, 2002a, b, 2008; Westminster Sustainable Business Forum, 2008). To date, no systematic research review has been undertaken on the procurement reform agenda at national and EU levels and its progress.

3.24 With specific reference to local government in Scotland, Audit Scotland (2012a) outlined why good cost data is necessary for understanding performance. The relevance for the procurement reform agenda is the current fiscal context of reductions in public sector spending. In this context, there is the need for a good understanding of both the cost and quality of services if public sector buyers are to demonstrate good value for money and the need to monitor suppliers' performance. Audit Scotland (2012a) noted that there is a need for cost information plus a focus on monitoring and measuring outcomes for service users and communities at each local authority level (Audit Scotland, 2012a: 3 - 5). Audit Scotland (2013a) echoed the previous point in its report on 'Improving community planning in Scotland', which drew on three recent audits set within the context of public sector reform in Scotland. Although Audit Scotland (2013a) referenced examples of good partnership working across Scotland, it found that there was a lack of evidence on the outcomes and impact of community planning partnerships. Audit Scotland (2013a) argued that it was difficult to demonstrate that community planning partnerships have had their intended effects. Linked to this point, Audit Scotland concluded that community planning partnerships have not been focused enough on identifying local priorities and taking action on priority areas (Audit Scotland, 2013a: 6). It also found that lack of consistent leadership and poor governance and accountability issues hampered the performance of community planning partnerships (Audit Scotland, 2013a).

3.25 Audit Scotland (2012a) indicated that there is the need for local authority councils to be transparent about cost information in order that communities can be informed (Audit Scotland, 2012a: 3). This is important as the community empowerment agenda grows and as government policy encourages communities to be active decision makers and implementers of service delivery (Scott et al, 2012). This could have implications for public procurement. Audit Scotland concluded that the use of cost information amongst local authority councils is fairly undeveloped, thus, "[…] we are not advocating the immediate adoption of a comprehensive and detailed costing measurement system" (Audit Scotland, 2012a: 9). As Audit Scotland (2012a) argued a focus on outcomes, needs to be supported by a good understanding of costs - what is termed as 'outcomes focused budgeting' and that there is a role for Community Planning Partnerships to understand how the collective spend of public bodies affects the delivery of their shared outcomes (Audit Scotland, 2012a: 12). Reference is made later in this evidence review (Chapter 5, Theme 5) to the argument for increasing transparency in administrative and cost data as part of a strategy to help prevent crime in public procurement.

3.26 Cuthbert and Cuthbert (2012) presented a critical account of procurement in Scotland. As noted earlier in this evidence review, whilst it is recognised that procurement reform is still work in progress, there is a broad consensus on the need for procurement reform and its current direction of travel at EU, UK and Scotland levels. Cuthbert and Cuthbert (2012) depart from that general consensus and contend that the current approach to procurement supports large, foreign businesses, hinders economic development in Scotland and fails to emphasise social, economic and environmental objectives in procurement but their argument is based on very limited evidence. Their reference to the procurement "[…] process not working satisfactorily in Scotland's interests" and "[…] that apparently irrational decisions are often taken, with firms with local expertise and locational advantage being passed over for large enterprises which may not be located in Scotland at all" do not appear to be based on objective evidence (Cuthbert and Cuthbert, 2012: no page number). Equally, Cuthbert and Cuthbert seem to take a protectionist view, - that is, that all Scottish businesses should be always prioritised and favoured despite the fact that Scotland and the UK exist within the single EU internal market and that EU rules on Procurement and State Aid are designed to avoid distortion of competition.

3.27 It is unclear, methodologically how Cuthbert and Cuthbert (2012) gathered their evidence. The material used is partial with specific examples and strong statements being used to defend their argument as opposed to being based on a balanced, range of evidence. Their review is not a full scale review of contracts awarded and therefore, is unrepresentative and selective in its presentation of cases. In this respect, extreme caution should be exerted over their conclusions.

3.28 The Royal Incorporation of Architects in Scotland (RIAS) (2011), whilst acknowledging that some progress had been made under the procurement reform agenda and underscoring its support for reform highlighted a number of areas for concern with specific reference to the construction sector. RIAS (2011) identified the following issues: (i) the increasing barriers to SME access to public sector contracts; (ii) the lack of capability and expertise in construction in the public sector; (iii) "the design quality of new buildings and their fitness for purpose"; (iv) "the pricing, reliability and value for money of services provided through the public sector procurement process" (RIAS, 2011: 3). RIAS recommended tackling the identified issues, in addition, to reducing the administrative burden on the private sector by improving the PQQ process, giving feedback to tenderers, tackling the tendency to aggregate projects, shifting to a focus on quality of outcomes as opposed to quality of procurement processes and place a greater focus on quality and whole life value for money. RIAS also called for greater public sector in-house construction expertise, however, it is unclear from the evidence provided, what evidence exists to substantiate their argument that there is a lack of public sector in-house capability in procurement. RIAS recommended the creation of an independent and expert panel to advise on these issues and how to resolve them. Although wider evidence exists on some of the issues identified, caution should be exerted over some statements and conclusions as limited and anecdotal evidence is provided to support them. For example, the claim in paragraph 2.2.3 of the RIAS (2011) report regarding "[…] Scottish procurement policies and processes which sit under the Directive and Regulations are not fit for purpose…" would need to be unpacked and evaluated in order to determine its validity (RIAS, 2011: 6).

Leadership, partnership working, adapting and developing performance and management data

3.29 Audit Scotland (2013b) offered an overview of the ways in which local government in Scotland is responding to challenges and change in the current economic context. In the context of public sector reform, it presented four key recommendations for local authorities: (i) develop good leadership and governance through building positive working relationships and ensuring financial accountability and monitoring service performance; (ii) work in partnership especially in the area of community planning to improve outcomes; (iii) adapt with ongoing changes to public policy and service provision and rigorously appraise options for delivering services; and (iv) develop performance and management information (and promote self-evaluation within organisations) to help assess impact and outcomes as part of service improvement and assessing value for money (Audit Scotland, 2013b: 4). Audit Scotland noted,

"CPPs (Community Planning Partnerships) must demonstrate that they are improving local outcomes but this can be difficult. It requires good local data and meaningful comparisons to track progress over time" (Audit Scotland, 2013b: 14).

3.30 Reference is made in Audit Scotland's (2013b) report to a summary of legislative changes that might impact on local government, however, no reference is made to the forthcoming Procurement Reform Bill (Audit Scotland, 2013b: 8, Exhibit 3 & 12). Brief reference is made to Procurement Capability Assessments (PCAs), which are conducted annually and Audit Scotland reported that the PCA score for councils has been improving and is now at "[…] 48% per cent, just short of the Scottish Government's target for all sectors of 50 per cent by the end of 2012" (Audit Scotland, 2013b: 23).

3.31 The Scottish Government Procurement Reform Delivery Plan set two targets in relation to Procurement Capability Assessments - (i) that all relevant organisations should be in the conforming category by the end of March 2012 and (ii) that 60% of relevant organisations should reach improved or greater status by the end of March 2012 and that all/100% of organisations should achieve this by the end of March 2013 (Scottish Government, 2012k, para. 6, bullet 3). Current analysis shows the following for organisations by sector in the Improved category or above: 82% of organisations nationally; 100% Central Government Family; 81% Health Family; 100% Higher Education/Further Education; 100% of Full PCAs; and 56% Local Authorities (Scottish Government, 2013e unpublished: 9 and cf with paragraphs 3.16 to 3.18 above describing procurement capability assessments).

Evidence from the management of ICT projects in Scotland

3.32 Audit Scotland (2012b) conducted an audit of three Scottish public sector ICT programmes and highlighted some lessons learned for public bodies running ICT programmes in the future. The main lessons learned regarding effective management of ICT contracts include: (i) the importance of having skilled staff with project management skills; (ii) the importance of good governance arrangements; (iii) the need for staff with risk management knowledge and expertise and knowledge of the subject to avoid over-reliance on suppliers' knowledge; (iv) the need to conduct a whole life costing of any given project; and (v) the importance of monitoring and evaluating ICT contracts - their progress, outcomes and impact (Audit Scotland, 2012b). The themes identified in this Audit Scotland (2012b) report are similar to those themes highlighted in this evidence review and can be useful for thinking about areas for continual development in the broader programme of procurement reform.

Evidence from Commissioning social care in Scotland

3.33 Audit Scotland (2012c) provided an overview of the local government landscape on commissioning social care. It is set within the wider context of socio-economic and demographic change in Scotland, public sector reform and the challenges posed by these changes when operating within a mixed economy of service provision in the social care sector. Audit Scotland (2012c) highlighted a number of areas that need attention to ensure good strategic commissioning of social care services. These included: (i) strong leadership and commitment to a joint long term strategic planning for commissioning social care; (ii) strategic vision; (iii) the importance of cultural and behavioural change; (iv) the right skills, capability, capacity and risk management expertise in both procurement and social care; (v) partnership working; (vi) a focus on performance and improvement and as part of this, the use of analysis and evidence to develop services based on users' needs and monitor impact; and (vii) the importance of user engagement in commissioning social care to ensure needs based analyses (Audit Scotland, 2012c: 12-13, Exhibit 5).

3.34 Audit Scotland also concluded that many local authorities and NHS Boards had not fully implemented commissioning strategies although guidance existed on commissioning and procurement (Audit Scotland, 2012c:16). Audit Scotland identified a weakness in procurement capability arguing that local councils perform worse than other sectors in their Performance Capability Assessments (PCAs) with social care procurement being weaker than other areas of procurement activity (Audit Scotland, 2012c: 21). Audit Scotland argued that the reason for this was,

"This is often because social care has not been following corporate policies and procedures or the policies do not take sufficient account of users' personal dependence on the services" (Audit Scotland, 2012c: 21).

As outlined above, the themes are similar to the themes outlined in this evidence review - for example, Audit Scotland identifies the need to understand due diligence checks before awarding contracts (for example, at the PQQ stage), the importance of good project management skills and the importance of monitoring, gathering evidence and assessing the impact of services on people's lives (Audit Scotland, 2012c: 30). This reinforces the argument for the need to continue with the procurement reform agenda as it is still work in progress. Other linked issues raised include the need to involve users and carers in the commissioning of contracts for service provision in order to ensure that services are user based and user friendly, the need to improve the support to carers and to develop consistent and comparable measures to improve the evidence base. The themes set out in this report are very similar to those themes discussed in the aforementioned Audit Scotland reports and all set in the wider context of public sector reform in Scotland.

Developments in England and Wales

3.35 The National Audit Office (NAO) (2013a) 'Improving Government Procurement' provided an overview of the procurement reform strategy, governance and accountability procedures and progress to date with the specific focus on the UK government. The NAO (2013a) provided a critical account of some recent procurement developments, the newly created Procurement Government Service (thereafter GPS) and Whitehall Departments. It recognised that some positive progress had been achieved, for example, the creation of the spend analysis tool is an improvement in data collection. However, limitations remain with some smaller arm's length bodies not providing data and problems exist with cleansing data and identifying individual suppliers (NAO, 2013a: 19 & 46-47, Appendix Three on Data coverage).

3.36 One of NAO's (2013a) main findings was that, despite the policy drive to centralise the procurement function in the formation of the GPS and to mandate Whitehall Departments to use central contracts, government departments are often still operating independently in terms of procurement and gathering and calculating spend data at a departmental level. This makes it difficult for the centralisation of public procurement to take full effect and for comparable and accurate estimates of spend and efficiency savings to be calculated. It also results in government not maximising the potential for savings through centralised procurement. Having said that, the NAO stated that it had confidence in GPS's reported £426 million savings for central government in 2011-12 (NAO, 2013a: 2).

3.37 Other findings outlined in the NAO's (2013a) report included: (i) poor quality data collection in terms of spend on SMEs making it difficult to measure whether the government is meeting its target to achieve 25 per cent of spending with SMEs by 2015; (ii) lack of analysis or consultation with departments in setting targets and checking how departments would move towards central contracts; (iii) operational issues exist with GPS's management of central contracts and tensions exist between operational and cultural norms at both GPS's level and within and across departments making the standardisation, simplification and centralisation of procurement processes difficult to implement and the freeing up of procurement resources difficult to achieve; (iv) lack of foresight and planning with respect to capability and skills required following the reforms; (v) inadequate mechanisms exist for all departments and GPS to hold each other to account regarding expenditure and operational risk (and given that the mandate to use central contracts is not enforced and no sanctions exist, this does not encourage behavioural change at a departmental level); (vi) linked to this is the issue that governance structures have grown organically with the purpose and remit of GPS groups and boards being unclear; and (vii) the aforementioned is "[...] exacerbated by gaps and inconsistencies which remain in data management systems at the centre of government" (NAO, 2013a: 8).

3.38 The NAO (2013a) concluded that whilst there has been some progress, for example, common goods and services are more centralised, SME participation has increased and the GPS is an improvement on its predecessor, there is still much room for improvement. The NAO (2013a) argued that the Cabinet Office will need to lead the way in the fostering cultural and behavioural change across Whitehall Departments to help tackle the problems that remain, namely, "[…] ineffective governance structures, unrealistic targets, incomplete data, and weaknesses in the management of the central contracts" (NAO, 2013a: 8). The NAO (2013a) recommended improving the definition of roles and accountability, in addition, to Cabinet Office setting "more sophisticated" procurement targets and strengthening the quality and consistency of data that it collects "[…] to improve its oversight of procurement expenditure, and to ensure the data is useful to departments" (NAO, 2013a: 9).

3.39 The NAO (2013a) also recommended that GPS "[…] should provide robust evidence to departments, to give ongoing assurance that the central contracts provide value for money" and that "[…] GPS should be accountable to departments for underperformance, and should address a number of weaknesses in its management of the central contracts" (NAO, 2013a: 10). With regards to Departments, NAO argued that departments should comply with the mandate to sign up to central contracts, that is, they should "[…] remodel their procurement functions to adapt to a centralised model" and should improve the quality and consistency of their procurement data (NAO, 2013a: 10). In order to monitor progress of the reform strategy, the NAO recommended that government improve its management information and develop effective targets (NAO, 2013a: 17 & 19).

3.40 The NAO (2013b) reviewed five UK government reform initiatives on ICT as part of the ICT strategy and procurement reform and reported shorter term and longer term savings through ICT contracts (NAO 2013b: 4). The NAO (2013b) found that negotiating with the largest ICT suppliers as a single customer has helped departments reform their supplier relationships (NAO, 2013b: 7). Equally, it found that leadership and cross profession collaboration have been important enablers in the progression of the reform initiatives (NAO, 2013b: 9). Despite some reported savings, the NAO (2013b) identified some outstanding issues still requiring further development including: (i) the need to build a stronger evidence base on ICT expenditure and improve accurate data capture; (ii) the need to address the insufficient resource for the ICT spend control process; (iii) tackling the complex governance arrangements; and (iv) the need to improve capacity and capability across central government (NAO, 2013b: 9). The lack of accurate data on ICT spend results in a lack of accurate baselines for comparative analysis of spend across all ICT suppliers and also makes it difficult to evaluate and report the impact of ICT initiatives beyond solely ICT savings (NAO, 2013b: 8 & 10). Locating ICT and procurement reforms within wider public sector reform, the NAO argues the need for strong leadership in civil service reform beyond the ICT profession (NAO, 2013b: 11).

3.41 Drawing on the evidence from across the EU and the information available from current systems, there is a need to gather more data in a consistent and comparable manner across the public sector and the need to mandate the collection of that data. Progress has been made in the creation of different systems but more could be done to strengthen those systems to allow for the more systematic collection of reliable data, which would allow for a more detailed picture of procurement reform progress, the procurement landscape and the performance of organisations engaged in it.