We are testing a new beta website for gov.scot go to new site

STSRF - meeting 2 2010-11

Third sector Research Forum - 2nd Quarterly Meeting

14 September 2010 10.30am- 12.30pm

Conference Room B

St Andrews House

Kay Barclay, SG (Chair and minutes)

Geoff Pope, SG

Linda Boyes ACOSVO

Dee Fraser, ESS

Jacqueline Killeen Big Lottery Fund

John Lee VDS

Louise Meikleham OSCR

Stephen Osborne, University of Edinburgh

Alasdair Rutherford, VDS

Fiona Robb HIE

Ruchir Shah SCVO

Antonia Swinson, SSEC

Sandy Watson SE


Kathy Cameron, COSLA

1. Welcome

The forum welcomed new member, Professor Stephen Osborne from University of Edinburgh Business school to the forum

Minutes from the previous meeting were approved and updates confirmed.

2. Project Updates from Forum members

2.1 Louise Meikleham, OSCR

· Resilience of Charities The three month project commissioned by OSCR has recently been completed by Axiom Consulting (findings were discussed at last forum meeting); OSCR now plans to use this survey as baseline data.

· OSCR is keen to explore impacts of current policies and practices on the charities sector as a whole (to help inform resourcing and corporate planning)

· OSCR is in the early planning stages for research to help establish the future focus and extent of OSCR's activities to improve compliance with the Charities and Trustee Investment (Scotland) Act 2005;

· How the charity sector will respond depends a lot on the relationships charities have with funders. As there are already research sources on charity income analysis OSCR will look at the information already available and then - explore in more detail how charities are responding to current circumstances/ how will they be affected? i.e. will there be more/fewer/same number of charities. Louise asked the Forum if they were aware of other similar research on charities income.


GP asked if this project would look at trends over time/ include analysis of past funding streams; and emphasised the need to explore current policies for funding; to better understand who is being funded; implications of funding cuts and implications for organisations becoming charities. He drew attention to Scottish Government work on Public Sector contractual spend using Spikes Cavell data.

RS - suggested Sensitivity Analysis as a useful methodological approach; this measures the degree of sensitivity of different parts of the sector; this methodology has been discussed between SCVO, NCVO, NICVA and WCVA and between SCVO and Carnegie to measure impact of recession. However, there is a lack of resources to enable it to put into practice.

Action Point - All to give further consideration to the potential use of Sensitivity Analysis as a tool; RS to forward relevant documents to LM.

Jackie Killeen- will feed back to the Funders Forum

2.2 Antonia Swinson, SSEC

· Results available from the latest IPSOS Mori Poll (conducted at six monthly intervals) on awareness / attitudes to social enterprises. Show increasing awareness and understanding of the term "social enterprise" and decreasing ignorance over time. Also show more than 50% of respondents interested in community led energy projects. All in all, encouraging results.

· Survey of Food and Drink Social Enterprises SSEC are managing a survey into food and drink social enterprises which has been funded by Scottish Funding Council and conducted by Queen Margaret University. The aims of the study are to map the number of enterprises in food and drink sector and key facts such as numbers of employees, numbers of volunteers, income, state of business, sector with greatest number of enterprises trading, portion of whole food and drink sector. Respondents are being contacted via SSEC intermediaries and Survey Monkey used. Results expected at end of November 2010.


· There was a query on whether or not the results (compared with previous years) were statistically significant. Antonia believed that the findings would be statistically significant but would need to check the details.

Action point: Antonia will check with Ipsos and feed back.

Action point- Antonia to share the questionnaire with Alasdair Rutherford at VDS and Kay Barclay at Scottish Government to input to wording of questions.

· LM drew attention to issue of trend of very low awareness of charities amongst young males and wondered if this might be similar in relation to social enterprises and more widely. Likewise AR noted that they have data showing that young people lack awareness of volunteering and of that their parents volunteer; and LB noted that ACOSVO have similar data about low levels of volunteering amongst young people. It would be useful to pull together data on gender/age demographics of awareness of third sector / volunteering.

Action point- All members to check with their relevant groups/ forums and submit relevant research/ data to Kay who will follow this up;

2.3 Dee Fraser, Evaluation Support Scotland:

· Harmonising Reporting

ESS has been working with Scottish Funders' Forum to make reporting less burdensome for voluntary organisations. Key themes include helping organisations to report against outcomes, how organisations can demonstrate social value; making use of existing resources to measure outcomes and how to communicate learning; Key problem is that organisations lack or do not collect the data for outcomes evaluation.

ESS have collected a fair bit of data on outcomes from small organisations (which may be of potential use in third sector outcomes work); of greatest use is data relating to interventions and outcomes


· There may be potential links to Sue Northrop's work (later on the agenda) on third sector approaches to measuring outcomes;

· LM noted that OSCR is driving to ensure the minimum statutory requirements are met in terms of data collection and submission

· The issue relates to the challenges faced broadly of third sector demonstrating social impact (a theme being addressed by TSRC and which was discussed a great deal at the recent NCVO research conference)

· GP raised the potential of SROI as a means of measuring social value

· SO suggested it may be useful to avoid getting heavily into quantitative indicators and overly focus on quantifying social value, and that qualitative approaches may be more useful (but quant data tends to be favoured by Ministers as it is regarded as more robust)

· RS drew attention the relevance of "relational capital" research that SO is doing; and suggested that it would be useful to have a future agenda item on "impact" in future

Action point: consideration to a future seminar on evaluating impact; Dee to provide a summary of findings from ESS work at organisational level

2.4 Fiona Robb, HiE

· HIE Social Impact Measurement (SIM): At 6 month stage of pilot year. Data collection has been slow, however, growth plans tend to be completed during the latter part of the year. Continuing support from ESS on formative evaluation of model.

· Demonstrating Value: Pilot agreed with CEIS for 2 social enterprises in HighlandL Fyne Futures and STrathpeffer Pavilion

· 040 Policy Retreat 7-8 September: HIE supported pilots and attended conference. 040 pilots have generated useful information about the extent to which social enterprise is a viable model for basic older people's services. HIE starting to shape a sectoral approach to supporting social enterprise.

· Cultural Maps: Currently collating data for a range of maps demonstrating cultural activity in HIE area.

2.5 Alasdair Rutherford, VDS

· Development of Volunteering Support (Big Lottery funded project) as part of a Volunteering Improvement Programme. The project will explore needs, barriers and interventions involving focus groups and a literature review.

· VDS research team- Helen Harper due to return from maternity leave in October; Alasdair moving own; John Lee will be coordinating research in the meantime

2.6 Jackie Killeen, Big Lottery and Funders Forum

· Funders Forum: Capturing the Value of small scale grass roots voluntary activity; focus on activity that is "at risk" including "under the radar" activity that is not captured by SOAs.

· BIG have reopened their main funding portfolio -with focus on self evaluation; a support contract will be available at the end of 2010 to enabling funders to evaluate their effectiveness;

· 2 Small knowledge exchange partnerships (one for older people; the other on young people especially those leaving care) which aim to bring together grant holders, policy makers and academics

AR suggested that Mike Woolvin's work on information volunteering would be relevant;

Action point: AR provide JK with link to Mike Woolvin.

2.7 Stephen Osborne, Edinburgh University

Four PhD studies under his supervision are:

1) Developing Capacity in Contingency Planning (third year PhD student)

2) Co-production amongst Asylum seekers (Kirsty Wallace) - exploring how third sector organisations engage with asylum seekers, examining the nature of co-production; potential to relate to public-social-partnerships.

3) Philanthropic Foundations - capacity for promoting innovation

4) Social networking sites- volunteering and time-sharing sites

2.8 Kay Barclay, Scottish Government

1) Scottish Government: Success Factors for Social Enterprises;

· Qualitative research with case study enterprises (initial list of 25 narrowed down to 11 in depth, included 5 BME enterprises)

· Report completed and publication imminent

· Key success factors included clear and shared social mission; strong and inspiring leadership; close alignment with stakeholder/market needs; product of value; effective relationships; operational excellence; entrepreneurial streak/business acumen; culture of learning; sustainability; strong handle on finances

· Conclusions

· Need to achieve the right balance between social goals and making a profit: must be a successful business first and foremost

· They are very susceptible to market conditions/ government actions

· Important to create conditions that are conducive to social enterprises to flourish

2) Third Sector Longitudinal study;

Wave one field work completed (end of the first year of the project); all eight case studies done; all three thematic focus groups completed for the year (covering themes of health/ social care, equalities, employment/regeneration)

Analysis and Report

· Use "Framework" approach to longitudinal analysis ( matrix system enabling cross sectional and longitudinal analysis)

· Wave One Report due at end of September. Publication of this report intended in January 2011.

Next steps:

Next Research Advisory Group at end of October 2010; Workshop (Feb 2011); Archiving data; policy seminar in March 2011;

3) Evaluation of Investment and Business Support Evaluation

Evaluation of funding and support provided through the Enterprising third sector action plan

Key findings

  • Broadly the right approach to delivering funds and support
  • More than 1700 applied and 1000 received support
  • Funds and support well received and positively evaluated
  • Lack of cohesion across the package
  • Especially business support elements - not well integrated/ some lack of awareness
  • Strong support for element of choice over business support in terms of type, amount and provider
  • Scope to improve cohesion through a one door entry point, a short business health check at application stage, better signposting to different package components and clear consistent branding of the package
  • Some positive impacts included raising profile of their org; increasing income generation and achieving longer term financial stability

Report publication date: 21 Sep 2010 (update the report was published on 23 September)

4) Third Sector Quantitative Data Work

This is a collaborative project involving a sub group from the Research Forum. A draft of the paper written by SG analysts discusses main sources of data; Some shared areas of need identified. SG is keen to work in partnership with forum to overcome data gaps.

Key findings so far

  • Identified key themes: shape and size of the sector; economic contribution; employment data; volunteering; added value of third sector; social enterprises; public sector spend
  • Key data gaps - Basic quant data - number of third sector organisations; income; contribution to economy; number of employees and volunteers
  • A number of data gaps (especially around economic contribution)- expect to be filled by new approach to measuring the national indicator (parallel project - to pilot a new methodology; although this is unlikely to be taken on as approach to measuring the indicator, this should enable us to start to gather more robust suite of data- this is discussed in greater detail see item X on minutes)
  • Projects to calculate public sector spend is also underway

Other key data gaps:

Volunteering - Scottish Household Survey is main source of data; robust source on prevalence and frequency of volunteering; BUT only asked of half the sample- so smaller sample size means findings cannot be broken down by LA; request to SHS to increase to full sample size

Social enterprises - very difficult to isolate organisations on existing datasets (such as IDBR) so not able to separate out in the indicator calculation; also problems of definition; no formal regulation of social enterprises

Possible options

· SVA Managing Intelligence- one stream of larger project which aims to support local interfaces to manage and store their data in a consistent way: may give more localised data on numbers of volunteers / characs of organsiations (but not clear how robust, certainly more comprehensive than current systems)

· Consideration given to a mapping exercise and survey of social enterprises

· Developing SCVO Panel Survey.

Next steps - Kay to organise another meeting to discuss options for developing the data (once report is completed).

3. "Third Sector Contribution to Outcomes management- an introduction" (Sue

Northrop, Scottish Government)

Sue Northrop from OCR sought interested parties to input to work that she is doing to support Community Planning Partnerships' delivery of local outcomes. She is keen to explore key role and contribution of third sector to the evaluation of outcomes and asked the Research Forum if they could provide examples of good practice amongst the third sector organisations; help to develop points of contact and identify key themes.


· GP drew attention to potential relevance of pilot public-social-partnership case studies that are currently underway involving a consortium of organisations inputting into the design of services; Impact measurement is running alongside; the role of the sector in the delivery of outcomes is an area of great importance.

· RS noted the role of campaigning and delivery organisations basing decision on robust evidence (and not in response to panic); a link with the co-production can also be made

· FR noted the existence of a great deal of information and a need for a central management point in order to feed back to CPPs in a more effective way.

· AS noted that membership organisations face challenges in demonstrating what they are doing and that it is critical that the sector feeds into local decision making

· Speaking to the Third Sector Interfaces would be a key way forward

Action point: Sue would like to explore the issue in greater detail with a sub group key stakeholders from the Scottish Third Sector Research Forum initially by email. Those that are in a position ton contribute should respond directly to Sue Northrop at sue.northrop@scotland.gsi.gov.uk

Action point: GP will provide link to interfaces to enable first hand experience to be obtained

4. "Piloting a new methodology for National indicator 18 (social economy turnover): difficult to define organisations" Geoff Pope

Geoff Pope presented Kay Barclay's paper on the proposed methodology for calculating the third sector indicator (paper attached). In sum the IDBR is the most robust source of data and the methodology proposed will involve developing a database of third sector organisations in order to cross match onto the database.


It was generally accepted that the methodology was a robust one. The main problem was with definitions of the third sector (see later).

· In terms of the methodology, RS noted the challenges of obtaining a good match and referred to earlier similar work by NCVO where match rates were very low. He questioned how SG would deal with the problem of match rates varying year on year (eg it was likely that the match rate could improve with each year) which could affect the accuracy of comparison over time. Would an estimate of the turnover of unmatched organisations be provided? If so, how would this be calculated?

· He suggested that it would be useful to provide a second set of results based on turnover data from other organisations in order to obtain a maximum turnover figure. This would enable a range of possible figures of between "no less than X and no greater than Y" to be provided.

· It was also clarified that this approach only provided data for the larger organisations (and that the approx 75% of third sector organisations with turnover less than 25K would not be included accurately as their data is imputed on IDBR); however this was accepted, given that 90% of third sector income is generated by the 10% of largest organisations which appear on IDBR.

Action point: The points were acknowledged and will be checked with the statisticians in Scottish Government responsible for the indicator, to clarify the approach that will be taken to deal with unmatched organisations.

Update: Response from statistician in OCEA, Scottish Government

  • It's likely that we can tolerate some exclusions without estimating the non-matches and adding them in as the focus of the indicator is to measure change over time. However if the match rate is very low, we may need to think again.
  • To overcome the issue of improving match rates from one year to the next, a back series for the indicator will be revised each year when the new data are released. This is already done for other matching exercise where we have a core list that is updated each year.
  • A range of figures would not be a good idea for the indicator itself - given that we are concerned with changes over time and the fact that we won't have a 100% match. However, that doesn't mean to say that we couldn't consider producing different estimates based on the use of different categories as we work towards the indicator definition and these findings could supplement the indicator data.

Issues around definitions

· The key problems are around the definition of third sector and there was a lot of discussion around the "difficult to define" organisations that had been identified

· As a forum, members concluded that the rationale for excluding organisations should simply be that they are not the target of recent government policy and for that reason should not feature in the indicator calculation.

· It is useful to note that the definitions are being used here specifically for calculating the national indicator rather than provide data for the third sector as a whole.

· It was also noted that "charities sector" and "third sector" are not one and the same and therefore there is not necessarily a problem in distinguishing some organisations that have charities status from the third sector definition.

· Deciding which Cooperatives should be included and which excluded was problematic and somewhat arbitrary as in theory all could be argued to be providing social benefit.

· Likewise, excluding churches and organisations "advancing religion" is problematic as many organisations categorised as such deliver essential services.

· Leisure trusts- need to probe and explore this area a bit more as they may hold differing statuses / "independence" can be interpreted differently

· AR noted a problem with the wording in the report regarding "consulting third sector representatives" as a presumption has already been made as to who is included in the third sector definition and who is not.

· Supporters trusts- agreed they could be included for the purposes of the national indicator publication.

· There may be a need for more consultation with key groups.

Action point: Kay and Geoff to update paper and work to clarify definitions.

Update: further points for consideration have been received from OSCR and will be taken into account before a final decision is reached.