Publication - Independent report

Developing the Young Workforce (DYW) regional groups: formative evaluation

Published: 15 Nov 2018
Directorate:
Fair Work, Employability and Skills Directorate
Part of:
Education
ISBN:
9781787813380

External evaluation to examine the early operations of four of the most established Developing the Young Workforce (DYW) regional groups. The study was carried out between December 2017 and April 2018.

100 page PDF

1.1 MB

100 page PDF

1.1 MB

Contents
Developing the Young Workforce (DYW) regional groups: formative evaluation
Annex E: Survey questions

100 page PDF

1.1 MB

Annex E: Survey questions

Survey of DYW Regional Group Board Members

Profile

Q1. Which of the following best describes the organisation you work for?

  • Private sector employer
  • Employer representative body
  • School
  • College
  • Local authority
  • Charity / third sector
  • Other public sector body
  • Other (please specify)

Q2. Approximately how many hours per month do you spend on activities relating to the operation and governance of the [NAME OF REGION] DYW Regional Board? This includes attendance at all Board and sub-group / committee meetings.

  • Number of hours

Engagement with DYW Regional Group

Q3. How important are the following factors as motivators for your ongoing membership of the [NAME OF REGION] DYW Regional Group?

RESPONSE OPTIONS: Not important / Moderately important / Very important / Don't know

  • Alignment to your corporate objectives
  • To put something back into the local community
  • To address skills shortages in the region
  • To help develop the future pipeline of skills for the region
  • To improve links between employers and education
  • To improve the employability skills of young people
  • To improve the work-readiness of young people
  • To improve the job outcomes of young people
  • An opportunity for personal development
  • Other (please specify)

Q4. To what extent are your expectations from membership of the [NAME OF REGION] DYW Regional Board being met?

  • Expectations are not being met
  • Expectations are being partly met
  • Expectations are being fully met
  • Don't know / not sure

FILTER: If expectations are not being fully met:

Q5. Can you explain in what way your expectations are not being fully met?

  • Open Question

Q6. What would need to change to enable them to be met?

  • Open Question

Governance and operation of DYW Regional Group

Q7. To what extent do you agree with the following statements relating to the [NAME OF REGION] DYW Regional Group?

RESPONSE OPTIONS: Strongly disagree / Disagree / Agree / Strongly Agree / Don't know

  • I am clear on the strategic objectives of the Group
  • I am clear on the priority areas of activity for the Group
  • The Regional Board takes the lead on key decisions for the Group
  • The Regional Board signs off on key decisions for the Group
  • The sub-committees of the Board take the lead on key decisions for the Group
  • The DYW Executive Team take the lead on key decisions for the Group
  • There are good lines of communication between the Regional Board and the Executive Team
  • There are good lines of communication between the Regional Board and its Sub-Committee(s)
  • There are good lines of communication between the Executive Team and the Sub-Committee(s) of the Regional Board

Q8. To what extent do you agree with the following statements relating to the DYW National Group?

RESPONSE OPTIONS: Strongly disagree / Disagree / Agree / Strongly Agree / Don't know

  • I am clear on the strategic objectives of the DYW National Group
  • The National Group provides the right level of guidance on which activities the Regional Group should be prioritising
  • I am clear on how the Regional Group contributes to the wider DYW policy agenda
  • There are good lines of communication from the National Group to the Regional Groups
  • There are good lines of communication from the Regional Groups to the National Group
  • The Link Member model works effectively

Q9. Overall, how effective do you think the current governance arrangements are for the [NAME OF REGION] DYW Regional Group?

  • Very ineffective
  • Not effective
  • Effective
  • Very effective
  • Don't know

Q10. FILTER IF NOT EFFECTIVE / VERY INEFFECTIVE:

Q11. How do you think the governance arrangements for the [NAME OF REGION] DYW Regional Group could be improved?

  • Open Question

Q12. How effective do you think the current performance monitoring and reporting arrangements are for the [NAME OF REGION] DYW Regional Group?

  • Very ineffective
  • Not effective
  • Effective
  • Very effective
  • Don't know

FILTER IF NOT EFFECTIVE / VERY INEFFECTIVE:

Q13. How do you think the performance monitoring and reporting arrangements for the [NAME OF REGION] DYW Regional Group could be improved?

  • Open Question

Effectiveness and impact of DYW Regional Group

Q14. How would you rate the effectiveness of the [NAME OF REGION] DYW Regional Group in relation to:

  • Creating a single point of contact for employers to engage with education
  • Creating a single point of contact for schools / colleges to engage with employers
  • Co-ordinating employer engagement with education
  • Encouraging more employers to offer work experience placements
  • Improving the quality of work experience placements
  • Encouraging more employers to deliver work inspiration activities
  • Improving the quality of work inspiration activities
  • Encouraging more employers to take on apprentices
  • Encouraging more young people to pursue vocational career pathways
  • Encouraging employers to change their recruitment practices
  • Encouraging the education sector to engage with employers

RESPONSE OPTIONS: Not effective / Effective / Very effective / Don't know

Q15. How far has the culture within schools and colleges in the region has changed as a result of engagement with the DYW Regional Group?

  • A lot for the worse
  • A little for the worse
  • Not at all
  • A little for the better
  • A lot for the better
  • Don't know

Q16. How far has the behaviour of schools and colleges in the region has changed as a result of engagement with the DYW Regional Group?

  • A lot for the worse
  • A little for the worse
  • Not at all
  • A little for the better
  • A lot for the better
  • Don't know

Q17. How far do you think the culture of employers in the region has changed as a result of engagement with the DYW Regional Group?

  • A lot for the worse
  • A little for the worse
  • Not at all
  • A little for the better
  • A lot for the better
  • Don't know

Q18. How far has the behaviour of employers in the region has changed as a result of engagement with the DYW Regional Group?

  • A lot for the worse
  • A little for the worse
  • Not at all
  • A little for the better
  • A lot for the better
  • Don't know

Success factors / Areas for improvement

Q19. How would you rate the [NAME OF REGION] DYW Regional Group in terms of:

  • Clearly communicating their aims and objectives to others in the region
  • Engaging senior leadership within schools / colleges
  • Engaging senior leadership within employers
  • Adding value to the existing landscape of provision
  • Tailoring their offer to meet the needs of employers
  • Tailoring their offer to meet the priorities of schools / colleges
  • Responding to local needs / priorities
  • Getting all of the relevant partners on board
  • Reaching disengaged young people
  • Reaching young people with additional support needs
  • Demonstrating impact

RESPONSE OPTIONS: Very poor / Poor / Good / Very good / Don't know

Q20. Of those elements of the [NAME OF REGION] DYW Regional Group that you think are working particularly well, what are the success factors?

  • Open question

Q21. Of those elements of the [NAME OF REGION] DYW Regional Group that you think are not working well, how do you think they be improved?

  • Open question

Q22. How effective do you think each of the following would be in helping to improve the operation of the DYW Regional Groups?

RESPONSE OPTIONS: Not effective / Effective / Very effective / Don't know

  • Access to shared services
  • National DYW marketing campaign
  • A more streamlined and consistent set of KPIs
  • More opportunities to share best practice
  • Access to a bank of toolkits and resources
  • Guidance on measuring impact

Survey of schools / colleges

Profile

Q1. Which of the following best describes your organisation?

  • Early Years Establishment
  • Primary School
  • Secondary School
  • Special Educational Needs School
  • College

Q2. Approximately how many students are in your school / college?

  • Number

Q3. Which of the following best describes your MAIN job role?

  • Headteacher / Deputy Head Teacher
  • Primary classroom teacher
  • Secondary classroom teacher / PT
  • Lecturer
  • Guidance teacher
  • Careers adviser
  • DYW Co-ordinator
  • Support staff
  • Other (please specify)

Q4. What is your role in relation to DYW? (tick all that apply)

  • Engagement and communication with DYW Regional Group
  • Managing programme of DYW activities
  • Co-ordinating delivery of DYW activities
  • Managing relationships with employers
  • Identifying students to participate in DYW activities
  • Other (please specify)

Q5. On average, how many hours per week do you spend on DYW-related activities?

  • Number

Engagement with DYW Regional Group

Q6. How important are the following factors as motivators for your school / college to engage in DYW activities?

RESPONSE OPTIONS: Not important / Moderately important / Very important / Don't know

  • To improve the work-readiness of your students
  • To improve the job outcomes of your students
  • Opportunity to get additional resource for your students
  • To improve the employability skills of your students
  • To meet the requirements of the Career Education Standard
  • To meet the requirements of the Work Placements Standard
  • To meet other organisational objectives
  • To address a gap in existing provision
  • As an opportunity for staff development
  • Other (please specify)

Q7. Has your organisation experienced any of the following barriers / challenges to engaging with DYW activities?

Select all that apply

  • Lack of staff time to support delivery
  • Lack of budget to support delivery (including to cover transport costs)
  • Lack of senior leadership support and value placed on outcomes for DYW
  • Challenges with time and flexibility within the school / college timetable
  • Focus on other policy priories, including attainment
  • Lack of relevant suitable offers from employers
  • Lack of interest from students
  • Other (please specify)

Q8. How many employers has your organisation engaged through the [NAME OF REGION] DYW Regional Group in each of the following categories?

Insert numbers next to each OR select "don't know"

  • Strategic engagement with leadership within schools/colleges – for example through the establishment of a formal partnership, collaborative working on programme / curriculum design and development (including Foundation Apprenticeships)
  • Involved in the planning and delivery of activities – such as mentoring, curriculum development, project-based work, enterprise activities or work experience placements
  • Attendance and participation at events / activities – such as careers fairs, business breakfasts, classroom or work place visits
  • Don't know

Q9. Has your engagement with the DYW Regional Group led you to deliver more and / or better quality employer engagement activities for your students?

RESPONSE OPTIONS: Yes – delivered more of these / Yes – delivered better quality / No change

  • Careers and skills fairs
  • Careers talks
  • CV workshops
  • Mock interviews
  • Mentoring with an employer
  • Employer-delivered workshops on employability skills
  • Enterprise activities / competitions / challenges
  • Employer-delivered classroom learning (on career pathways / opportunities)
  • Employer-delivered classroom learning (curriculum)
  • Co-design of the curriculum
  • Networking with employers
  • Work place visits for students
  • Work place visits for teachers
  • 1-2 week work placements
  • Flexible work placements
  • Part-time working
  • Work shadowing
  • Other (please specify)

Q10. How useful did you find the school / employer partnership guidance (produced by Education Scotland) for planning your work with employers?

  • Not at all useful
  • Not useful
  • Useful
  • Very useful
  • Not aware of this

Q11 – Q12 – ADDITIONAL QUESTIONS FOR AYRSHIRE ONLY

Q11. Has your school / college sourced any additional investment through the Ayrshire DYW Innovation Fund?

  • Yes
  • No
  • Don't know

FILTER, IF YES:

Q12. How would you rate the quality of the activity that was delivered through the Innovation Fund?

  • Very poor
  • Poor
  • Good
  • Very good

Q13. How would you rate the impact of the activity delivered through the Innovation Fund on young peoples':

Response options: Negative impact / No impact / Positive impact / Don't know

  • Confidence
  • Commitment
  • Communication
  • Customer Service
  • Courtesy
  • Customer Care

Feedback on DYW activities

Q14. Overall, how would you rate the quality of the activities delivered following your engagement with the [NAME OF REGION] DYW Regional Group for your students?

  • Very poor
  • Poor
  • Good
  • Very good

Q15. Overall, how would you rate the impact of the activities delivered following your engagement with the [NAME OF REGION] DYW Regional Group on your students?

  • Large negative impact
  • Some negative impact
  • No impact
  • Some positive impact
  • High positive impact

Effectiveness of DYW Regional Group

Q16. How would you rate the effectiveness of your DYW Regional Group in relation to:

RESPONSE OPTIONS: Not effective / Effective / Very effective / Don't know

  • Creating a single point of contact for employers to engage with education
  • Creating a single point of contact for schools / colleges to engage with employers
  • Co-ordinating employer engagement with education
  • Encouraging more employers to offer work experience placements
  • Improving the quality of work experience placements
  • Encouraging more employers to deliver work inspiration activities
  • Improving the quality of work inspiration activities
  • Encouraging more employers to take on apprentices
  • Encouraging more young people to pursue vocational career pathways
  • Encouraging employers to change their recruitment practices
  • Encouraging the education sector to engage with employers

Success factors / Areas for improvement

Q17. How would you rate the [NAME OF REGION] DYW Regional Group in terms of the following elements:

RESPONSE OPTIONS: Very poor / Poor / Good / Very good / Don't know

  • Clearly communicating their aims and objectives
  • Adding value to the existing landscape of provision
  • Tailoring their offer to meet the needs of employers
  • Tailoring their offer to meet the priorities of schools / colleges
  • Engaging senior leadership within schools / colleges
  • Engaging senior leadership within employers
  • Responding to local needs / priorities
  • Getting all of the relevant partners actively supporting the DYW agenda
  • Reaching disengaged young people
  • Reaching young people with additional support needs
  • Demonstrating impact

Q18. Of those elements that you think are working particularly well, what are the success factors?

  • Open question

Q19. Of those elements from Q14 above that you think are not working well, how do you think they could be improved?

  • Open question

Survey of employers

Engagement in DYW activities

Q1. How important are the following factors as motivators for your organisation to engage in DYW activities?

RESPONSE OPTIONS: Not at all important / Moderately important / Very important

  • To meet Corporate Social Responsibility objectives
  • To put something back into the local community
  • To improve the work-readiness of young people
  • To raise awareness of the career opportunities available in your industry
  • To develop the future supply of skills for your industry
  • To address skills shortages within your industry
  • To source potential future recruits for your company
  • To provide an opportunity for staff development
  • Other (please specify)

Q2. How would you describe the nature of engagement that your organisation has established with schools / colleges through the [NAME OF REGION] DYW Regional Group?

Select all that apply

  • Strategic engagement with leadership within schools/colleges – for example through the establishment of a formal partnership, collaborative working on programme / curriculum design and development (including Foundation Apprenticeships)
  • Involved in the planning and delivery of activities – such as mentoring, curriculum development, project-based work, enterprise activities or work experience placements
  • Attendance and participation at events / activities – such as careers fairs, business breakfasts, classroom or work place visits
  • Don't know

Q3. Which of the following activities have you / your organisation participated in or delivered as a direct result of engagement with the [NAME OF REGION] DYW Regional Group?

Select all that apply

  • Careers and skills fairs
  • Careers talks
  • CV workshops
  • Mock interviews
  • Mentoring
  • Delivery of workshops on employability skills
  • Enterprise activities / competitions / challenges
  • Employer-delivered classroom learning (on career pathways / opportunities)
  • Employer-delivered classroom learning (curriculum)
  • Co-design of the curriculum
  • Networking with school / college students
  • Work place visits for students
  • Work place visits for teachers
  • 1-2 week work placements
  • Flexible work placements
  • Part-time working
  • Work shadowing
  • Other (please specify)

Q4. Approximately how many hours has your organisation spent on activities organised through the [NAME OR REGION] DYW Regional Group over the past 12 months?

  • Enter number
  • Don't know

Q5. Has you / your organisation experienced any of the following barriers / challenges to engagement in DYW activities?

Select all that apply

  • Staff time off-the-job to participate
  • Aligning schedules with school / college timetables
  • Regulations around engaging young people (including health & safety)
  • Lack of senior management support within your organisation
  • Lack of suitable opportunities for you to engage with
  • Challenges engaging staff in schools / colleges
  • Lack of experience of working with young people
  • Lack of experience of working with schools / colleges
  • Other (please specify)

Feedback on DYW activities

Q6. Overall, how would you rate the quality of the activities that you / your organisation has engaged with through the [NAME OF REGION] DYW Regional Group?

  • Very poor
  • Poor
  • Good
  • Very good

Q7. To what extent are your organisation's expectations from engagement in DYW activities being met?

  • Expectations are not being met
  • Expectations are being partly met
  • Expectations are being fully met
  • Don't know / not sure

Q8. Would you recommend engagement with the [NAME OF REGION] DYW Regional Group to other employers in your area in future?

  • Yes – I already have
  • Yes – I haven't yet
  • No
  • Don't know

Q9. Have you been able to influence the teaching within the schools that you have worked with through the DYW Regional Group?

  • Yes
  • No – despite trying
  • No – have not tried
  • Don't know

Effectiveness of DYW Regional Group

Q10. How would you rate the effectiveness of the [NAME OF REGION] DYW Regional Group in terms of the following?

RESPONSE OPTIONS: Not effective / Effective / Very effective / Don't know

  • Creating a single point of contact for employers to engage with education
  • Creating a single point of contact for schools / colleges to engage with employers
  • Co-ordinating employer engagement with education
  • Encouraging more employers to work experience placements
  • Improving the quality of work experience placements
  • Encouraging more employers to deliver work inspiration activities
  • Improving the quality of work inspiration activities
  • Encouraging more employers to take on apprentices
  • Encouraging more young people to consider vocational career pathways
  • Encouraging employers to change their practices
  • Encouraging the education sector to change its practices

Impact of engagement with DYW Regional Group

Q11. What is your organisation doing differently as a result of engagement with the DYW Regional Group?

Select all that apply

  • Offered more work experience placements
  • Offered higher quality work experience placements
  • Offered more work inspiration activities
  • Offered higher quality work inspiration activities
  • Started recruiting young people
  • Recruited more young people
  • Started recruiting apprentices
  • Recruited more apprentices
  • Gained Investors in Young People accreditation
  • Other (please specify)

Success factors / Areas for improvement

Q12. What, if anything, do you think are the success factors enabling the DYW Regional Group to work well?

  • Open question

Q13. What, if anything, do you think are the key barriers / challenges holding back the effectiveness of the DYW Regional Group?

  • Open question

Profile

Q14. Which of the following best describes the industry that your organisation operates in?

  • Agriculture, forestry or fishing
  • Utilities
  • Manufacturing
  • Construction
  • Retail, hotels or restaurants
  • Transport or communications
  • Financial services
  • Public services, including health and education
  • Other services (including care sector)

Q15. Approximately how many people does your organisation employ in total?

  • Number

Q16. Approximately many people does your organisation employ within [NAME OF REGION]?

  • Number

Contact

Email: Adrian Martin