Learning Connections Adult Literacies Phase-Two Pathfinders: Evaluation Framework

7. Recommendations

In this section we set out a number of recommendations based on the lesson learned and the conclusions of the Pathfinders' Evaluation Project. These are intended to provide practical advice which can be used in the future by Learning Connections in its role promoting and supporting development and innovation.

7.1 Future Projects: While formally beyond the scope of the evaluation project we would recommend that future literacies projects supported by Learning Connections should a) generally be smaller and b) should be linked to existing agency's work and do not rely on the establishment of a separate project infrastructure and recruitment of new staff. This would avoid the relatively long lead in time the Pathfinders experienced, the difficulties of recruitment which some, but not all the Pathfinders had and finally, the understandable effort that goes into considering and obtaining longer term funding that inevitably arises when projects with new staff focus on, towards the conclusion of their initial funding agreements.

7.2 Project Evaluation: We would recommend that as well as undertaking final (summative) evaluations, Learning Connections also pilot alternative forms of formative evaluation. As we say above, we felt that the approach used in the second stage Pathfinders was perhaps over-reliant on a documentary reporting system. We would recommend an approach which was more "process" oriented. For example, integrating periodic review workshops into the project plan.

7.3 Evaluation Toolkit: We would recommend that there would be value in Communities Scotland incorporating the toolkit into its web site and making this more widely and generally available. We would also recommend, that the toolkit could be reviewed with a view to making it more adaptable to a wider range of projects. We are of the view that the evaluation criteria set out in the framework are appropriate for a much wider range of community learning and development projects, and with the addition of a few more criteria ( e.g. as we say above "sustainability"), could provide a useful and flexible resource for other projects.

7.4 Project Management: We would recommend that there would be value in devoting more time at the outset of any project using external consultants to clarify roles and expectations of both parties more clearly. As we comment, we did not feel that we experienced major difficulties, but it would have been helpful if the respective roles of the Project Managers, the Project Co-ordinators and the consultants were a bit clearer.

7.5 Web Sites: While we understand the reluctance of Communities Scotland IT section to operate within the CS web sites, sites which are not under their direct editorial control, we would recommend that a more flexible approach could be adopted. For example CS could provide consultants with guidelines on site content, layout, format and language which would ensure that it conformed to CS's standards.

7.6 Sharing Experience: A key objective of this project was to encourage and support the sharing of experience among projects during their lifetime. This proved more difficult to organise than we had suspected. We would recommend that the responsibility to participate in this was make more explicit and (almost obligatory) in any future pilot programmes.

7.7 Dissemination Events: We understand that the difficulties we experienced with low turnout seems to be a common experience with other similar events sponsored by Communities Scotland. We would recommend that there may be a number of solutions to this. We also appreciate that for a number of reasons it may not be acceptable to Communities Scotland to organise these events on a "commercial basis". In this case, we would have been happy to explore with Learning Connections the scope for organising the dissemination event on a fee charging basis, with Communities Scotland underwriting the event, but with the consultants generating income from the event and thus reducing the consultancy fees charged. It is also our experience that participants are much less likely to be "no shows" at events where they have paid a fee.

We would also recommend that Learning Connections review its database of contacts in Adult Literacies and Community Learning and Development. We received comments from some of the conference participants that they were aware of potential participants who had not received invitations and while no data base can be fully comprehensive, we would recommend that the current data base is reviewed and that a process for regular updating is put in place.