We are testing a new beta website for gov.scot go to new site

Evaluation of the Scottish Adult Literacy and Numeracy (ALN) Strategy - Final Report

Listen

EXECUTIVE SUMMARY

1 This report analysed the views of adult literacy and numeracy learners from 9 geographical areas of Scotland and the views of tutors from the same areas in order to assess the impact of participation in adult literacy and numeracy ( ALN) provision on individuals' lives and any wider benefits as perceived by the learners and tutors. It used both quantitative and qualitative data. Learners were interviewed twice in order to assess differences that had occurred in their views as a result of their participation in ALN. Six hundred and thirteen learners were interviewed in the first interview and 393 learners were re-interviewed one year later. Seventy-eight tutors were interviewed once to assess their views of the impact of the ALN strategy on their learners.

Findings from the Learners' Interviews

2 The impact of participation in ALN on learners' lives has been assessed through an analysis of their perceptions of the provision they received and its impact of their personal, family, work, education and public lives. Assessment of the quality of the ALN provision is based on the benchmarks identified in the 'Literacies in the Community; Resources for Practitioners and Managers' (2000) evaluation framework. The findings are based on a comparison between the 393 learners interviewed in both rounds of the research who were drawn from 9 Partnership areas that represent the geographical diversity of Scotland. Learners were interviewed that had taken part in programmes based in over a hundred different institutions with a variety of programme arrangements and types of providers. The views of all these learners were analysed and compared quantitatively and two hundred qualitative responses were also analysed and compared. There were small differences in the characteristics of the learners in the two samples but these were not statistically significant. So this is a representative cross sample of learners from Scotland and the comparisons between the 2 samples are valid.

Learner characteristics

3 The personal and programme characteristics of the 613 learners who took part in the first interview and the 393 who took part in the second interview were compared. The sample was predominantly: female (62% in both rounds), White British (92% 1 st; 88% 2 nd), receiving tuition in a non- FE setting (70% 1 st, 78%, 2 nd) and taking part in dedicated provision (68%, 1 st, 70%, 2 nd).

4 There were marginal differences between the two samples as follows: fewer were under the age of 21 (17% 1 st, 22% 2 nd), fewer were studying in integrated programmes (32% 1 st, 30% 2 nd); fewer were studying in F.E. colleges (30% 1 st, 22% 2 nd).

5 Of the 393 learners who were interviewed in both rounds: fifty (13%) had experienced positive changes in terms of moving into full time or part time employment or moving into FE provision from community-based provision. Twenty (5%) had experienced negative changes by moving from employment into unemployment or from FE into unemployment.

ALN Provision

Engaging in Provision

6 Research shows ( e.g.OECD and Statistics Canada, 2000) that adults with low literacy and numeracy skills do not necessarily seek tuition so it was important to discover what had motivated learners to start their programme. It was found that learners were mainly motivated by a desire for self-improvement and the development of their skills. The most important barrier to participation was about personal sensitivity such as the learners' age, lack of confidence, problems to do with meeting new people or how friends would react. The second most important barrier was a concern about ability, such as not being able to cope with learning or taking up education again. The fear that it might be like school again for people that had not enjoyed school and expected that their learning programme might use similar methods was the next most important barrier. Barriers that were within the control of learning providers such as the look of the building or staff being unwelcoming and the lack of facilities such as a crèche were much less significant.

7 The next stage in engaging in provision was enrolling on a programme. Most learners were encouraged to do this by a variety of people and events. The highest proportion was encouraged to start by unofficial people (family, friends, work-mates or casual acquaintances). This was followed by self-encouragement and then by people holding some position (doctors, social workers, job centre, youth club, employer). They received information on the programme from people at the centre that they enrolled at and the majority had no difficulty in starting their programme. Learners found the people that dealt with them were very helpful, felt that they were made welcome and important, found the information they were given useful and, if they met a tutor before they started, got advice about what would be best for them.

8 The factors that would make joining programmes easier broadly clustered into better publicity and the process of joining the course. Publicity, learners suggested, should change public perceptions about the image of ALN learning in order to make it more positive. Learners were particularly appreciative of the 'Big Plus' media campaign particularly as it had used 'real' learners talking about their own difficulties. When joining the course learners emphasised the importance of the first point of contact being knowledgeable, friendly, welcoming and not patronising. Pre-course guidance was perceived as important and those learners that had opportunities to meet tutors and other students before they started their course found it very helpful.

Learning, Teaching and the Curriculum

9 Learners were asked to give their views on what they had been learning, how it had been taught and what they thought of the staff. Overall responses were very positive with more than 90% satisfaction on the majority of indicators in both rounds of interviews. However, there were slight increases in negativity, which were statistically significant, between the 2 rounds of interviews in relation to:

  • The course being enjoyable
  • The learning programme being well-structured
  • The staff being encouraging
  • Learners having their confidence boosted
  • Having enough feedback on progress

10 These differences were differently experienced by the overall sample. There were statistically significant decreases in enjoyment by learners attending FE provision, female learners were significantly less likely to find staff encouraging, older learners were significantly more dissatisfied with their tutors and younger ones were significantly more likely to report that they did not get enough feedback. These slight increases in negativity appear to be due to learners raising their expectations of learning, teaching and the curriculum over time. Many ALN learners who have negative memories of school are likely to be very positive about educational experiences that are learner centred and focused on their needs (see McGivney, 2001). Once these positive experiences of returning to education become accepted then they are likely to become more critical. The larger increases in dissatisfaction amongst male and younger learners and those in FE could be due to their more recent experience of school and also being in an environment where the curriculum was more structured by Scottish Qualification Agency ( SQA) assessment requirements. This means that feedback on progress was likely to be less frequent and detailed. However, learners were overwhelmingly positive about all aspects of their programmes so these slight reductions in satisfaction are not indicative of a significant problem. Indeed it is a criterion of the evaluation framework for teaching, learning and the curriculum that learners should become more critical so this may be an outcome of their experience of being encouraged to do so.

Guidance and support

11 The quality of guidance and support was weak at entry to, and during, the learning process particularly in respect of learners' awareness of the Individual Learning Plan ( ILP). The ILP is an essential feature of the learning programme because it sets out in detail the learning outcomes, the learning necessary to achieve them and the sequence that learners and tutors should follow towards their achievement. Thus it is the most effective means of identifying ALN needs and building an individual curriculum to address them and forms a key part of the guidance process in ensuring that learners are working towards achieving their own goals. This impacted at different stages of the guidance and support process as follows:

12 Entry: 37% of learners overall were unaware of having an ILP drawn up at the start of their course and of the group participating in FE settings this rose to 45% (32% non- FE).

13 On the course: 50% of learners in FE settings and 27% in non- FE settings said that they had never had a review of their ILP during their programme. In addition, quite a high percentage of learners also reported that they had never discussed what had been learnt (22%), or their skills, knowledge and understanding (21%) during their course.

14 Exit: For the small percentage of learners that had left their programmes and had an ILP, a high percentage had received a review of their learning. However, quite a high percentage of learners did not have an ILP. Only a small percentage of learners could remember being helped to move on to other learning opportunities.

15 Leaving the programme: A very small percentage of learners had left their programme because they were unhappy with the content or methods These cases were most likely to occur when learners were part of mixed groups with a broad spread of ability. Learners who had completed their original courses have continued with a range of other provision in a variety of settings with some moving from community based provision into FE.

Reflection on Teaching and Learning

16 Learners were very positive about all aspects of their experience of participating in ALN programmes. These included:

  • The learning environment including the timing and location of the course, the cost, the facilities (crèche transport, café, rooms) and the learning resources that were available;
  • The factors that contributed to a good experience of teaching and learning, including what was learnt and the way it was learnt, the tutor, the pace of the learning and the number of hours of tuition available each week;
  • The social nature of the learning including the other students and the social opportunities to meet other people.

17 However, learners' in FE settings were more likely to express dissatisfaction with:

  • The cost of their learning (8% FE, 1% non- FE)
  • What they learnt (13% FE, 4% non- FE)
  • How they learnt (13% FE, 5% non- FE)
  • Their tutor (4% FE, 2% non- FE)
  • The pace of their learning (13% FE, 5% non- FE)

18 In addition, there were some concerns, evidenced through the qualitative data and the case studies, from learners about aspects of the course organisation in terms of its inflexibility, number of learners, or content. These concerns seemed to stem from the lack of resources to provide a fully flexible learning and teaching environment.

Impact on learners' lives

Social Capital

19 Social capital refers to the processes between people that establish networks, norms and social trust that facilitate co-ordination and co-operation for mutual benefit leading to reciprocity. The combined effect of trust, networks, norms and reciprocity creates both strong communities and a sense of personal and social efficacy. The development of social capital involves the active and willing engagement of citizens within a participative community. Research suggests that developing an identity as a learner is shaped by the complex interaction of a number of factors. These include past learning experiences and the mediating effect of family influences upon them as well as the norms and values of the social networks that individuals belong to. Social capital therefore provided a relevant framework for analysis as it was hypothesised that learners might improve their social capital as a result of participation in programmes.

20 Overall respondents had very high social capital as they generally liked their neighbourhoods, were well integrated in their communities, voted, and would like some more local involvement. Despite these high levels the evidence from the quantitative elements of the research has shown an associated statistically significant increase in their social capital between the two rounds, specifically in relation to learners' levels of social activity and contact with others, and this is confirmed by the testimonies of the learners in the qualitative sections. The only decrease was in 'voting' but this was likely to be because elections had not occurred during the interview period. Although the complex operation of social capital precludes the attributing of any direct causal relationship between the two, these findings concur with the evidence of other research that indicates a link between engagement in learning and increased social capital.

Confidence

21 Confidence in ones self in the social world, and confidence in ones self as a learner interact together in complex ways and are both linked to prior experiences of learning and to social capital (Field, 2005, Schuller et al, 2004). Since growth in self confidence is the most widely documented 'soft' outcome of learning, this research sought to measure learners' levels of confidence at the beginning of their learning experience in order to chart potential changes in confidence over time. A method for measuring confidence was devised that picked out relevant scenarios for learners that were grounded in situations that they would face in their everyday lives. The scenarios asked how confident learners were when: meeting new people; making phone enquiries; joining a group of strangers; discussing things with officials; discussing things with a doctor; speaking up in a meeting; complaining about poor service; defending their position in an argument; agreeing within the family; and being interviewed. By the second interview statistically significant numbers of learners, in particular women and older learners, had become more confident about making enquiries over the phone, joining a group of strangers, speaking up in a meeting and being interviewed and their overall confidence scores had increased. In other words, their social and communicative abilities had increased during their learning episodes. Field (2005:19) suggests that interpersonal communications and connections are the core elements of social capital, and Schuller et al.'s (2004) studies of the wider benefits of learning provide strong evidence of the impact of learning on social meta-competencies that equip people with the confidence and ability to develop their social connections. The evidence from these confidence scales together with the increases in social capital already evidenced indicate that this same shift is being experienced by the participants in this research.

Effect on personal, family, work, education and public lives

22 There is evidence of a marked increase in the confidence and self-esteem of the learners between the first and second round of interviews. In our society there is a dominant discourse that regards people who are not literate or numerate as deficit individuals and learners' internalise this. When they participate in ALN provision and find that they can learn, this changes their perceptions of themselves too. As well as these psychological changes, learners have become more able to learn new skills, undertake more activities and interact more with other people as a result of their tuition. This, in itself, gives them more confidence in tackling other aspects of their lives. This newly found sense of self had been used to open doors into other worlds and activities that learners would not previously have contemplated, and they had grown further in the process.

23 Increased self-confidence: Learners' reported that increased confidence acts as a key to opening up other opportunities through a growing sense of their potential, ability and achievements, an increase in skills and an increasing range of activities that they could participate in. Responses suggest that this confidence can be separated into 3 distinct groupings; the confidence to learn, the confidence in learning, and the confidence in life that develops through learning. Confidence, self-esteem, closer family relationships and social and civic engagement each comprise elements of social capital, and as Field (2005) observes, they affect and are affected by learning. The evidence suggests that the 'virtuous circle' of social capital is operating and affecting those who have engaged in learning. It indicates that increased confidence and self-esteem are impacting on familial, social and work relationships, which in turn add to the sum of learners' social capital.

24 Expectation of change: There was a statistical association between learners' expectation of change and the actuality of this. In addition, people's lives were changing even when they did not expect them to through the benefits of learning. The data indicate that where predicted change was lowest, in relation to engagement in the community, unexpected gains were proportionately highest.

25 Positive differences: Participation in ALN had a positive impact on relationships and activities within the family especially when parents became more confident about their involvement in their children's education. It also had a positive impact on learners' perceptions of their current and future employment prospects and earnings. This research indicates that learners' growth in self-confidence leads them to seek better jobs or gain wage increases and that self-confidence is more important than qualifications in learners' own perceptions. It also indicates that the greatest discrepancy between aspirations and actuality lies in relation to employment, notwithstanding the fact that 7% fewer were unemployed and 28% reported improved job prospects between rounds. Other research (see, for example, Glasgow CLSP, 2005) suggests that as adults become more engaged in learning, their expectations of its effects shift from the naively optimistic to the realistic, particularly in relation to enhanced employment prospects. Despite this, 60% noted employment related changes after their ALN learning.

Findings from the Tutor research

26 The sample of 78 tutors was selected from different centres from those in which the learners' sample was based but in the same ALN Partnership areas. This avoided the possibility of the replies from learners influencing, or being influenced by, the participation of their tutors. It also meant that direct and inappropriate comparisons between tutors' and learners' views of programmes were avoided. These interviews took place after data from the first interview with learners had been analysed so that the issues that had been raised by learners could be included in the interview schedule. The interviews were conducted by telephone between April and September 2004 by members of the research team and, on average, took an hour. The questionnaire was based on the good practice framework, Literacies in the Community Resources for Practitioners and Managers (2000), in order to assess the quality of tuition provided against the framework. The questionnaire also provided the opportunity for tutors to reflect on: the learning programme itself, planning, resources, staffing and management within the organisation, their own professional development, partnership working, and the impact of the strategy on themselves, their organisation and learners alike. This is not a representative sample of tutors and so the findings cannot be generalised across the whole of Scotland. Nevertheless, the findings are indicative of some of the positive impacts and of some of the issues that need to be considered further if the ALN strategy is to be responsive to the views of tutoring staff.

Positive outcomes of the ALN Strategy

27 Overall the tutors' perceptions of the impact of the ALN strategy were that it has generally been well received and had impacted positively in the following areas:

  • The number and range of learners participating in ALN. These included previously under-represented groups such as young people, engaging with learners in different locations such as the workplace, and an overall increase in referrals from a range of organisations.
  • The tutors' approaches to teaching and learning. This area included better training and support for tutors on approaches to learning and teaching, the availability of more learning resources for a wider range of groups and more awareness of a range of approaches to learners with special needs ( e.g. dyslexia).
  • Funding and resources. This area is strongly related to resources for teaching and learning but also includes better publicity especially through the 'Big Plus' media campaign. Additional funding had led to better teaching accommodation that was open for more hours and was more accessible.
  • The experiences of learners. This area included more positive attitudes to ALN learners as a result of the initiative and more awareness of ALN issues on the part of providers such as employers and community learning and development staff. Positive student experiences also resulted from a higher priority being given to ALN provision through the additional resources detailed above.
  • The local and national profile of ALN. This area encompasses all the other positive aspects of the initiative including the improved quality of provision, the range of participating learners, the wider awareness of ALN amongst employers, public officials and education providers.

Further improvements required

28 Areas of the ALN Strategy where there could be further improvement were:

  • Fostering links with, and encouraging transfer to, other learning opportunities. An important aspect of the ALN strategy is to encourage the development of lifelong learners through learners moving on to, or between, other opportunities. Most of the tutor sample were either not aware of, or did not have responsibility for, helping their learners to move on. There needs to be a greater awareness of the value of links with other learning opportunities by tutors, and better information about them, as tutors are the people from whom learners are most likely to seek advice.
  • Guidance and support was an area that tutors identified as sometimes problematic. These difficulties included not having enough support to provide the right kind of guidance and also not having sufficient education and training opportunities to move learners onto. This was particularly problematic in rural areas or where learners had learning difficulties that required specialist provision, as this was not available outside of the ALN programmes.
  • Exit pathways were also identified as a weak aspect of provision by a number of tutors. The difficulties mostly related to roll-on roll-off provision that meant learners could join and leave at any time so they could continue on a programme without ever having a formal exit process for learners that were leaving. There were also problems of finding provision that learners could move on to as outlined above.
  • Communication with learners by management. The perception of the tutor sample was that management did not often consult directly with learners in ways that were appropriate. If the ALN strategy is to meet the good practice criteria then managers need to be more active and imaginative about their interactions with learners.
  • More attention to the needs of English as a Second or Other Language ( ESOL) tutors. It appears that, whilst more provision is now available for ESOL learners, there was a lack of resources specifically for this group and a lack of staff development for their tutors. It should be noted however that a new PDA in ESOL literacies has been introduced subsequent to the interviews.
  • Access to good quality and appropriate staff development and support for part-time staff and volunteers. Although opportunities for staff development and support have increased, it does not appear to be reaching the volunteer and part-time tutors that work on a sessional basis. Quite a high proportion of the negative comments on the impact of the strategy reflected the dissatisfaction of sessional tutors with the expectation that they would give up unpaid time to take part in training. There were also negative comments about the lack of access to the type of training that tutors felt would be most useful to them. It is important that these front-line staff, that are delivering the learning programme, have appropriate support and staff development.

Conclusions

What are the barriers and pathways into learning for ALN learners?

29 The most important barrier to be removed as identified by learners is the stigma attached to being a literacy/numeracy learner and the clearest pathway should be better publicity both locally and nationally. This implies that publicity about ALN should be directed at changing the negative public image of ALN learners and making it more positive. A key emphasis should be on how people can improve their own skills and in so doing become more capable and self-confident.

What are learners' and tutors' perceptions of the quality of learning and support they have received?

30 Learners were very positive about the quality of the teaching and learning they received and tutors were equally positive about the impact of the ALN strategy on who, what and how they taught. However, guidance and support needs to be improved particularly by ensuring that the ILP is used appropriately at all stages of the learning process.

What are the outcomes and impact that learning has on individual learners?

31 There have been a range of outcomes that have impacted on individual learners but the dominant one is increased self-confidence. Our respondents reported that increased self confidence was experienced as a growth in abilities, feeling better about one-self generally and feeling better about oneself in relation to others. Increased self-confidence acts as a key to opening up a wide range of other changes resulting from the confidence to learn, the confidence in learning, and the confidence in life that develops through learning.

What are the possible implications for the wider social benefit and economic activity from such findings?

32 There is an extensive research literature that demonstrates the link between low literacy and numeracy skills and economic and social status. Adults with low skills are more likely to be unemployed, living on low incomes, experiencing poor health and early morbidity. There is also a strong relationship between educational inequality, income inequality and lack of social cohesion in terms of societal trust and community safety. Given these negative indicators any positive changes in outcomes for learners as a result of ALN participation will contribute to wider social and economic benefits. The data have shown that engaging in learning enhances social capital and this increases economic and social activity leading to wider benefits for the individual, their community and society. The effect of education in raising people's sights is experienced more widely as a positive influence on the cultural norms that encourage others to do the same. Learning and its benefits are dynamic in the sense that benefits gained in one domain such as education impact on functioning in other domains, such as family and community. Many parents detailed the variety of ways in which their participation in ALN had helped them to do a better job as a parent. The positive changes respondents reported in their attitudes to education and family life are likely to result in benefits for the wider family and community as well as the individual concerned. These findings illustrate the impact that participation in ALN has on wider social and economic activity and shows the importance of providing good quality teaching and learning to enable this group of people to sustain and progress in their learning.

33 Research shows that adults who have returned to learning after an unsuccessful school career are more vulnerable to failure at this stage than other, previously successful, returnees. It is vital, therefore, that the Scottish Executive's commitment to 'Closing the Gap' between the disadvantaged and advantaged is carried through in providing an adult literacy and numeracy strategy that is world class. This research, by providing an evaluation of the current strategy based on learners' views, makes an important contribution to this process.

Recommendations

34 The following recommendations arise from the findings from both the learner and tutor surveys. They are based on the key areas of teaching and learning that require improvement because they do not meet the highest benchmarks for teaching and learning specified in Literacies in the Community: Resources for Practitioners and Managers (2000). These recommendations should be addressed by the various bodies responsible for ALN - Scottish Executive, Learning Connections and the ALN Partnerships - working together to bring about change.

  • More and better publicity that will change the image of ALN and make it more positive would encourage more learners to participate. This publicity should build on the success of the 'Big Plus' campaign.
  • More resources, which would enable programmes to be more flexible in terms of their timing, location and content.
  • Better guidance and support for learners: Tutors need to have more training on the use of the ILP and to use it with learners.
  • Better exit guidance and more opportunities for moving learners on to other provision: Tutors need more training in providing guidance and ALN Partnerships need to provide a greater range of learning opportunities.
  • Greater access to good quality and appropriate staff development and support for part-time staff and volunteer tutors.