Chapter Five Conclusions and recommended Next steps
5.1 This chapter draws conclusions for the engagement tools and materials, and the use of these to support the engagement process, from the results of the focus groups and literature and practice review. The scenarios and prototype tools developed in the study and described in chapter 3 were shaped by our interim conclusions.
5.2 The focus groups were, in effect, a test of the claims embodied in those scenarios and prototypes; claims that the selected tool genres were appropriate for engaging young people in the RWM debate, given the available evidence for 'what works' (or does not). On that basis this final chapter:-
- Makes recommendations about the planning of the engagement process, mainly in terms of the appropriate use of an e-engagement 'toolkit' package 16 , and involving stakeholders in creating appropriate content.
- Draws conclusions on what the roles of moderator and facilitator entail in online debate, to parallel the usual attributes of these roles in face-to-face work with young people.
- Discusses how evaluation criteria and indicators of success for an e-engagement should be elaborated from those used in the study.
- Makes recommendations for next steps.
5.3 The subsections of this chapter take each of these points in turn. The first point is addressed in two subsections, firstly to draw conclusions on the use of the e-engagement tools and secondly on the broader preparations for the engagement process.
UsIng the E-Engagement Tools Package
Scope of the findings
5.4 We approached the study believing that the RWM debate presented difficult challenges for e-engagement. The Internet is a 'self service' medium offering benefits of convenient access to policy-making information and exchanges between government and the public. However this is premised on trust in the information and its providers, and interest in the policy concerned. Previous cases of online and offline engagement on RWM issues indicate that where success has been achieved, this has been despite initial lack of awareness and mistrust of information, or lack of interest among young people. For example from recent UK experience: -
- The Scottish Council Foundation's focus group with young people for the MRWS consultation (Elrick et al, 2002) found that participants thought it essential the public were enabled to become better informed on the subject. However the Nuclear Industry and UK Government were the least trusted sources of information (the Scottish Parliament and Community Groups were the most trusted). More generally the report concluded that: -
"Participants placed a high premium on accurate information from trustworthy sources, but were concerned about how far various sources could be trusted. Lower levels of trust were placed in government and the nuclear industry as sources of information, with a higher degree of trust placed in environmental NGOs" (ibid. p. 2).
- The evaluation of the RISCOM II Schools website concluded that it had been successful but needed incentives to attract student involvement, and active promotion to (and within) schools. Awareness of RWM issues was initially low, and although students considered raised awareness of those issues to be one of the main benefits of taking part, the majority wanted more information (O'Donoghue, 2003).
- The PASCALEA consultation on the Atomic Weapons Establishment's environmental strategy, which used a website, focus groups, stakeholder workshops, interviews and public meetings, received little response to the website and generally faced significant concerns from participants about communication, trust, openness and transparency of information (Lancaster University, 2002).
5.5 Our awareness of the challenges that RWM represents for e-engagement has been heightened by the study. On the one hand, there was interest and often enthusiasm in using e-tools when the RWM debate was embedded in the activities of organised groups. On the other hand, we encountered very little appetite for sustained involvement in the debate by individual young people acting on their own accord.
5.6 The study shows, however, the value of electronic tools for those young people with an interest in using ICT; namely that it can address their need to see information drawn from various sources and perspectives, give them feedback on the views of their peers. Moreover an incentive to become actively involved is provided by making it the focus for ICT use integrated with the face-to-face activity of organised groups.
5.7 The design and evaluation of electronic tools should give a high priority to ease-of-use and accessibility, to minimise barriers to using them. However the main barriers to young people making effective use of the tools in the RWM debate are much more likely to be the organisational ones of coordinating the provision and use of the tools and their content. It will require the active support of the education and community sectors to provide young people with realistic opportunities to use e-tools to take part in the debate.
5.8 RWMexperts and decision-makers should be directly involved in the online debate. In planning the e-engagement there is also a need for non-government stakeholders, who may themselves be young people, to provide input and take on the following roles:-
- Facilitators to lead face-to-face discussion.
- Moderators to perform a similar role of supervising the online discussion.
- Contributors of position statements and background facts some of whom should be willing to be interviewed for online publication of their views or experiences.
5.9 The study results do not rule out the possibility of using videos or printed materials to support school or youth group discussion. However we do not consider these a realistic alternative to electronic tools, most importantly because they would be a far less effective medium for gathering young peoples' views, and allowing young people to share views with their peers in other localities. For raising awareness of RWM issues, materials that were completely print-based would be unlikely to provide the depth or variety of information that the subject demands. It would be costly to distribute printed materials and/or videotapes widely, and they would quickly become outdated as the debate progressed. Paper and video materials may be effective in support of face-to-face discussion, but can be distributed using the e-tools, to be printed locally or (in the case of video) played from CD- ROM or online.
5.10 The young people who took part in the study were, it should be noted, already members of groups engaged in face-to-face discussion of current political issues, and the results may therefore not be representative of young people who (a) do not find face-to-face discussion appealing; or (b) do not find the internet appealing. The next stage should establish the extent of any such bias. We should also note that one-off focus group discussions were not sufficient to make explicit or test all of the assumptions embodied in the prototype tools and scenarios.
5.11 Our conclusions on the engagement process are informed by the literature and practice review as well as the focus group comments. They follow below beginning with conclusions about, firstly, the fit between the tools and the purposes we identified for them, then more specific points on using each of the tool types, finally concluding with general points applicable to the e-engagement as a whole.
Using appropriate tools for the purpose
5.12 The e-engagement tools will need to cater for an audience with very diverse interests, backgrounds, perspectives, and linguistic and technical capabilities. For that reason we have not sought a single 'best tool for the job'- rather we considered the tool types as alternative ways of presenting and interacting with the content. That is, the tools should access a common underlying database of facts and opinion so that the consultation organisers and partner organisations can focus primarily on what needs to be communicated rather than how. However as the very terms "discussion board" and "frequently asked questions" indicate, some tools are better than others for specific purposes, and the study has helped to identify the appropriate balance.
5.13 The focus group comments have implications for the three 'overall purposes' we gave to the tools, i.e. to help the target audience of young people in Scotland to: -
Find out about Radioactive Waste Management: By finding out what sorts of questions are typically asked, formulating questions, and finding answers to those questions from background facts and opinion provided by technical experts and stakeholder groups.
Decide their own Point-of-View: By appreciating the current range of opinion about the issues and options that are the focus of the ongoing debate, including those expressed by the previous users' of the toolkit.
Have their Say: By expressing and exchanging ideas or views on the issues and options raised, the decision-making process or its outcomes.
5.14 The comments provided a better understanding of which tools would be appropriate for what, as shown in Figures 5.1 and 5.2 below, and elaborated in the text that follows.
Figure 5.1 E-engagement tools fit to purpose
Figure 5.2 E-engagement tools fit to context
Finding out about RWM
5.15 The most critical question on which the success of the engagement process depends, is "why is this relevant to me?" This was brought home to us in the closing discussion of Focus Group 2 and it was clear that the 'storytelling' approach that was liked by all 3 groups was critically important to communicating the relevance of the topic. In Group 3, the level of interest shown by East Lothian school pupils living in proximity to Torness nuclear power station suggests that young people living near such sites already understand the importance of the debate.
5.16 This is especially important because there is a clear need to provide factual information but the effectiveness of search mechanisms, used for example with Frequently Asked Questions, depends on the user having formulated a question that can be answered from the information stored. This presumes both an interest in the subject and some knowledge of it. We found no evidence to contradict the finding of the MRWS Scotland consultation that awareness and knowledge of RWM issues is generally low, and interest depends on appreciating the relevance and importance of those issues.
5.17 The responses indicated that although unbiased factual materials are obviously important, and the FAQ tool is an intuitive way to access them, at least as much importance should be given to 'storytelling' materials, using the Blog and Video Interview formats to give personalised accounts of how lives are affected (for better or worse) by RWM issues, and that this critically depends on the involvement and commitment of stakeholder groups to providing suitable material.
Deciding a point of view
5.18 This has 2 aspects:-
- Seeing arguments for & against a position: in organised (face-to-face) group settings the issue map tool is well suited to facilitated discussion of the pros and cons of the options available for addressing the issues under debate. Where individuals are acting alone or in a group, the Discussion Board format is well suited to following similar threads of online discussion.
- Appreciating other perspectives: the emphasis here is on understanding, rather than simply knowing that different points of view exist. The literature review and the focus group comments indicate that the video interview and diary ( Blog) formats are more suited to that.
Having a say
5.19 This has 3 aspects: -
1. Commenting on the information made available: We can distinguish between, on the one hand, information in the form of facts and opinion that is sanctioned, if not explicitly commissioned, by the 'owners' of the engagement process. On the other hand, the online engagement seeks young people's comments and these will be available to inform the views of others contributing to the debate. Although the boundary between them should not be fixed, since the materials should be updated to reflect the ongoing debate, some tools are better suited to the former than the latter (considered below). Specifically:-
- Young people want to be able to respond to factual material and know where it comes from, which supports the case for enabling users to rate the relevance of Frequently Asked Question material and send comments or questions on it to its authors (via an administrator).
- Authors should be able to respond to comments made by young people on the material they have written when this is based on personal experience or in the form of a position statement, in the Blog and Video Interview formats.
2. Exchanging points of view: the Discussion Board format is well suited to this purpose and there are many examples in the research and practice literature.
3. Rating a discrete set of options: There is an overlap in purpose and function between the game format and a questionnaire/ poll, in that both are structured approaches to eliciting responses to a range of pre-identified options, which together with any comment responses to open-ended questions, are generally not disclosed to other participants except as statistical totals. It is equally possible to embed closed questions in a discussion board format, and this could also be considered as an enhancement to the specification. Our focus group comments suggest that the main emphasis should be on: -
- Questionnaires as a 'quick and easy' option.
- Games as the 'fun way to learn and take part' option.
- Discussion board for 'other people to get back to you'.
Engagement diary as Blog
5.20 The familiarity and appeal of the blog 'diary' format suggests it may be the most appropriate means to publish details of future engagement events, e.g. online interviews, chats, or face-to-face events and response deadlines. An engagement diary/blog should also provide feedback on the response to the engagement process from the Executive or CoRWM.
Stakeholders as Bloggers
5.21 The blog format would be well suited to providing stakeholder groups with an opportunity to show how they are involved in RWM or affected by the issues raised by it. Widening the selection of 'bloggers' to include 'non-experts', for example young people living or working in nuclear communities, could better communicate the relevance of RWM to young people more generally. But to allow anyone the capability to create a blog (as suggested in group 1) would raise difficult moderation issues. Moderators would need to ensure that the blog posts were relevant, language appropriate and so on. Since the blog genre is associated with diary-style subjective personal experience, blog moderation is likely to involve difficult questions of what is or is not appropriate content for a government sanctioned publication, and would become impractical if unlimited access is given to any user to start a blog. It would be preferable to invite members of stakeholder groups to maintain blogs.
5.22 Biographical details allow the reader to understand the blog author's background, level of expertise, relevant interests and positions on the issues under discussion. In so doing, they may help readers to work out to what extent they should trust the information presented.
5.23 Where the blog authors are young people under 16 years, parental consent would be essential, and a facilitator would need to advise on the disclosure of identifying details.
5.24 Photographs or digital artwork may add to the appeal of the blog format, and provide the user with more insight into the views and experiences the blog author wishes to convey than text alone. However these require more effort to produce than text, and are unlikely to be forthcoming without a coordinated effort to encourage blog authors to provide them. Although the focus groups did not suggest it, this might take the form of a 'best blog' competition aimed at schools and youth groups involved in digital photography or arts-based consultation.
5.25 Video presents an opportunity for representatives of stakeholder groups to state their case, in response to questions that have been asked by young people. The video interview format communicates non-verbal information, and may have more appeal as a medium for complex arguments that would otherwise be communicated in text. However video and audio are prone to compatibility and downloading issues over the Internet, and then may only be accessible in the form of transcripts. Also the interviews overlap with Frequently Asked Question content since both have a question-answer format, so effective content management is needed to ensure that question-answer content is presented consistently.
Live panel: streamed video
5.26 The live (streamed) video format could provide 'effectively face-to-face' interaction with representatives of stakeholder groups on a question-answer panel. However the participants' comments about this tool and scenario did not suggest that it offered enough value to justify the likely high costs. This is particularly so when considering that the number of people able to connect to a live event at any particular time is likely to be small, and given the alternatives of pre-recorded video interviews and live chat-room technology (see below).
5.27 Personal video or 'webcam' technology is not widely implemented or accessible in schools or at public internet access points. The scenario did not imply that anyone other than the question-answer panel members would be visible on-screen. However even then it may be inordinately difficult to link up a small number of experts and stakeholders in different sites, without encountering technical problems with sound reproduction and video quality, the consequent delays to a schedule which may need to fit with school class timetables, as well as the likely difficulties the audience of young people in schools or community groups would face in accessing the streamed video streaming over the internet. Therefore we have not recommended it for inclusion in a toolkit.
Live panel: chat
Access control and moderation
5.28 The chat genre appeals because it is associated with real-time and anonymous interaction, i.e. the norm is that chat users can use any name they like to identify themselves in the chat room, and interact with each other 'live' (as opposed to the exchange of messages over days/ weeks that is the norm with email and discussion forums). However this is unlikely to be permitted in school or local authority community education contexts, without a moderator to filter out disruptive users or people outside the target group (whether with malicious intentions or not).
5.29 There should therefore be a mechanism to moderate the end-users' questions to the panel. This may partly be achieved by channelling publicity for the events to schools and community groups, so that it is only the targeted groups of young people who take part. However a password mechanism would be necessary to ensure access could only be made via authorised schools or community groups.
5.30 That would not rule out disruptive behaviour, and teachers may be dissuaded from participating without a mechanism to prevent it. It would not be realistic to expect facilitators or moderators to check every chat message from young people in their charge. This leaves two alternatives:-
- Only facilitators or moderators can take part in the chat, i.e. young people ask questions through them. This would be likely to rob the medium of much of its appeal.
- A more acceptable solution may be to use separate windows for un-moderated messages from end-users, and the interaction between moderator (asking questions taken from the un-moderated window) and the panel. Appropriate filter controls would be needed, and 'anonymous' login so that only screen names are shown.
We recommend the second of the above options.
Coordination of panel
5.31 Panel members unfamiliar with chat room technology may need some tuition and familiarity with chat room jargon and usage conventions. We would expect this to be feasible through a short online demonstration and rehearsal with telephone support before the event.
5.32 The 'live' nature of chat and the probability that the panel members would not be in the same location make it highly desirable for panel members to communicate by conference phone call at the same time as using the chat tool to interact. This would also allow the panel moderator to explain any chat room jargon used by young people, and to ensure that panel members explain any jargon they use themselves.
5.33 A list of question-answer panels or other chat events (if more than one is planned) should be included in an engagement event blog, identifying the names of participants, and providing links to biographical data or personal Blog. There should also be links to a summary of each past chat event and it would be one of the tasks of the panel facilitator/moderator to compile a summary.
5.34 Chat's could be publicised to the most likely participants, e.g. Modern Studies or Personal & Social Education pupils, through partner organisations with web-based newsletters or email lists. The publicity would target the teachers and youth workers who could facilitate young people's involvement and are more likely to be reachable than young people themselves. Potential partners might include the Scottish Guidance Association, Modern Studies Association, or Dialogue Youth ( COSLA/ Young Scot).
5.35 The focus groups confirmed the appeal of game formats, showing that even the 'look and feel' of a game is enough to convey an element of fun. However a strong gaming element is needed to keep young people engaged. There are many game genres and typical 'hooks' used to engage attention such as point scoring and advancement through levels, although these are not easily applied to 'serious' debate.
Publicising games to schools
5.36 Young peoples' access to a game tool on school computers may be affected by controls that schools enforce, including the use of internet filters that block keyword searches or access to specific sites. A game-based tool could be publicized as a 'quiz' or 'simulation' to avoid automatic blocking on name alone. However publicity efforts could also aim to have links to the tool included on educational games sites.
Improved gaming features
5.37 Given the appeal of the format it seems likely that it would be taken up by individuals outside the school context if it included 'game play' features in keeping with their expectations of game genres.
Story telling communicates relevance
5.38 The use of interviews, especially in video, was found to be an effective way to communicate the relevance of the issues to young people.
Game authoring- high resource demands
5.39 The need (in addition to other authoring tasks) to devise a means for game players to progress through a series of challenges, in order to 'win' or at least complete the game, and to support that with high quality graphics, indicates that development of this tool would be relatively demanding in terms of authoring skills and resources.
5.40 A quiz format similar to that of the NRPB's " KU World" may be relatively straightforward to devise and implement, but is oriented more to learning facts than appreciating a range of views or contributing to a debate. A balance needs to be drawn between, on the one hand, presenting both sides of an argument to which there is no correct answer and, on the other, providing a mechanism for earning points in order to win a game or complete a quiz : -
- Where the facts regarding particular RWM issues are contentious enough to rule out a correct answer, or the aim is to encourage the 'players' to consider other opinions, an appropriate solution might be to award quiz points for finding out information on the range of views expressed on the issues ( e.g. "Which of these groups supports keeping waste above ground…?")
- Points could be awarded for contributing comments, according to transparent criteria and judged by a moderator. Since there would inevitably be a delay (of hours or days) between adding a comment and the award of points, this could also be an incentive for end-users to revisit the game. There would also be the added benefit of aiding the evaluation of the contributions to the debate.
5.41 Animated graphics could help end-users to envisage the more abstract mechanisms involved in RWM options, e.g. to convey what "partitioning and transmutation" might look like, as well as giving a more 'fun' feel to the interaction.
Simulation and role playing
5.42 It would in principle be possible to devise very sophisticated simulations of RWM decision-making based on risk models, extrapolating the consequences of choices perhaps across thousands of years in a fictional word. The likely expense of that may not be justifiable. However it would be desirable to allow game players to adopt roles of stakeholders, and it may be possible to include a role playing element into a simpler quiz format, e.g. "answer questions as if you were in charge of a nuclear plant", to convey why particular stakeholder groups might hold the views they do (with the obvious risk of stereotyping).
5.43 Some comments highlighted the association with particular graphic styles with different target age ranges, for example older participants thought the KU World game was meant to appeal to younger users (children, early teens). This highlights the need to assess the tools with younger age groups and tailor them accordingly.
5.44 Using the discussion board allows young people express an opinion, and to form that opinion from reading what their peers have to say. If deliberation is taken to mean that opinions should also be based on the background information made available for that purpose, then the use of the discussion board with other tools more suited to that should be promoted.
Target Modern Studies & PSE classes.
5.45 The probable greater appeal of the discussion board to young people with an interest in debating current affairs issues (and the general remarks that school was the most likely context for engagement) indicate that the support and involvement of teachers of Modern Studies and Personal & Social Education should be sought, in using the discussion board to facilitate debate between classes or between schools.
Use in group setting
5.46 Comments suggest that the discussion board's appeal would depend on prior interest in formulating a point of view based on learning about the issues, and an incentive to exchange views with others not present in person. This corresponds with the finding of the Radiale project that forum use is higher if it is facilitator-led, i.e. preceded by face-to-face discussion.
5.47 The comments suggest a questionnaire/ poll tool could be presented to meet needs for:-
- Fast download, since questionnaires were thought likely to be used by individuals from home.
- Ease of responding, since completing a series of questions was seen as a relatively self-contained activity.
- Anonymity, since the format is associated with giving a confidential response to a survey, rather than a response that may be read by others as in a discussion board/forum.
5.48 Some focus group participants saw closed questions as too limited and potentially manipulative. We would recommend that closed questions should always be complemented by open ones to allow comment. Questionnaires should also be promoted alongside other tools that support the public exchange of views.
5.49 Questionnaires provide an 'easy option' for individuals to express their views, and their structuring around a series of (more or less) closed questions that may be answered with a mouse click aids navigation and the user's sense of interactivity. Links to background information were seen as a support to learning, although the prototype did not force users to follow these links before responding.
5.50 However questionnaires typically do not allow respondents to share their views online, except in the sense that a breakdown of the running totals of responses to each option in a closed question may be shown (although the prototype did not provide this function).
5.51 The online questionnaire format presumes some other means for a face-to-face group (or by extension a larger target group) to form and exchange their views. In a facilitated group setting it would therefore be more appropriate to use it as the last of a series of activities focused on learning and dialogue, culminating in discussion of the questionnaire responses if, as our participants suggested, the group could give a single response.
Issue maps hold group attention
5.52 The Issue Map tool had a lot of appeal to Group 2 and 3 participants as a tool for use in facilitated groups, although not for individual use outside such groups. The structuring of information around arguments/evidence for and against decision options was thought to engage attention, and to support learning and participation.
Integration with handwritten comments
5.53 There was a useful suggestion in Group 3 that stickers could be used in face-to-face discussion. These could be designed to correspond to the icons used in the tool to differentiate between issues (or ideas), and arguments (or options). Participants might then use these after writing comments on cards, to allow a facilitator to quickly sort them for editing or comparison with a pre-existing map.
Role in group process
5.54 It seems more appropriate to view an 'issue map' as the outcome of a face-to-face group having a collective say (as reflected in the research literature) rather than for separate individuals to build a map entirely online. Maps could however be posted online to represent the outcome of a group discussion, and comments invited for consideration when the group next met.
5.55 This reflects participants' comments, which suggested it was clear to them that this tool was better suited to understanding others' points of view and discussing them in the context of face-to-face discussion, rather than through online discussion. The prototype maps did not make it explicit who put forward what point, as the basis for an exchange of views. Nor did they directly support that exchange of views. This was considered a negative point, as participants thought it desirable to be able to contribute online to the debate represented in a map, but with the contribution added by a facilitator.
5.56 This mechanism could also be used to support: -
- Peer education, i.e. for a representative of one group to present that group's deliberations to a second group, and invite comments to be made online afterwards.
- Self-study, the issue map format was thought similar to 'mind maps' which are commonly taught as a self-study aid.
5.57 The needs for facilitators' training and support in learning how to use the tool are an important aspect of any follow-up research and development.
5.58 Some participants thought the maps unlikely to appeal to younger users (children, early teens). This highlights the need to assess the tools with younger age groups.
Initial training/support for facilitators
5.59 The prototype was not thought usable without some tuition. This suggests that facilitators are likely to need some initial training. Some of the focus group comments indicate the skills that a facilitator would need to deploy when using the tool with a group, i.e. although the maps show a range of views it is obviously not possible to explore all of them at once. Facilitators should in any case be skilled in helping a group to articulate its views, and the explicit mapping of points of view should support that task, at the risk of simply assuming that a map of the most common arguments corresponds with the range of views present (or forming) in the group.
Preparing for the E-engagement
Youth reference group
5.60 As we mentioned earlier in the chapter, both the e-engagement tools and the engagement process itself need to cater for the diverse interests, backgrounds, perspectives, and linguistic and technical capabilities of young people. This is emphasised by much of the research and practice literature on youth consultation, and should be given real weight.
5.61 In particular, a 'one-size fits all' product may detract from the success of any future stakeholder engagement exercise, since differences in language and culture (if not technical skill) are likely to exclude children on the one hand and older people on the other. Groups such as the Children's Parliament and Age Concern Scotland should be consulted on how the tools may best be used to support the participation of younger and older age groups in the RWM debate.
5.62 A Youth Reference Group should be established to enlist the support of young people in each of the relevant stakeholder groups, to represent a cross section of the target audience. The study had envisaged convening this group from volunteers recruited in the 3 focus groups. Although volunteers were forthcoming from these groups, there proved to be insufficient time to organise the Youth Reference Group, because of the initial difficulty in recruiting the focus groups and the impact of this delay on the project timetable. This did not affect the validity of the focus group outcomes, since the participants had the opportunity to comment on the draft reports, but there were inevitably some matters that would have benefited from the further discussion that a subsequent meeting would have provided, such as the most effective publicity routes.
5.63 The Youth Reference Group would advise the project management on matters including: -
- The conditions of use policy, and editorial decisions. Given the complex nature of RWM it is necessary to emphasise the issue of trust in the information provided. Young people also need to trust the engagement processes and the engagement owners. One mechanism to support this is for information to be sourced from and accredited to stakeholders holding variously 'pro' and 'anti' nuclear positions, and subject to a liberal editorial and moderation policy that is owned by the Youth Reference Group.
- Clarity of the decision-making process. It is important that this is communicated in terms that are understandable to young people and a reference group may be invaluable as a sounding board.
- Publicity. A reference group should be consulted about the appropriateness of particular publicity routes, to complement professional advice.
5.64 Any engagement process involving young people in Scotland is likely to "cross-cut" with on-going initiatives such as the Scottish Youth Parliament, Young Scot, Dialogue Youth and Highland Youth Voice, whose input to a Youth Reference Group would be highly desirable.
Steering the authoring process
Involvement of stakeholders
5.65 To produce material that spans the range of positions and perspectives on RWM will require the input of professional stakeholders groups, i.e. local and national government, non-governmental and civic organisations, and the nuclear industry. The conclusions of the Managing Radioactive Waste Safely consultation (Elrick et al, 2002) are relevant here, i.e. that participants felt that 'all sides' should provide expert advisers. We should also note the finding that the public trusted environmental groups most as a source of information (Rodger, 2002).
5.66 There is a particular need to involve schools, whether as direct participants or indirectly as a publicity route and as venues for Public and Stakeholder Engagement. Young peoples' knowledge of RWM issues gained in school, and their experience of debating the impact on their own communities, is likely to be distributed across a wider range of teaching subjects than in England. Although we are not in a position to recommend particular areas of the curriculum, likely contenders are Personal and Social Education, and Modern Studies. This may 'cross-cut' with the interests of (for example) the Dialogue Youth initiative in working with schools to promote learning opportunities in active citizenship. In any case, the direct involvement of teachers or youth participation workers will be essential to ensure the materials are appropriate for that purpose.
5.67 Incentives for continued involvement in the engagement need to be in place. The need for a continuous process emerged as a public demand from the MRWS consultation. This implies sustained involvement of young people and others prepared to create and update relevant materials. The example of the US Environmental Protection Agency's funding of 'environmental fellowships' may be transferable to the Scottish context in the form of support for placement students on relevant higher education courses, although this would need further investigation.
5.68 Valuable lessons may be drawn from the Radiale project (Radioactive Dialogue in a Learning Environment), which produced web-based tools and piloted them in 6 secondary schools, in 2002-3. This was coordinated by Lancaster University with the support of DEFRA, the Health and Safety Executive, and the Environment Agency, each of whom were represented on a Steering Group. A key feature of the project was the adoption of an instructional design methodology, to coordinate the work of a team of content authors. The team brought together schoolteachers with subject specialists from industry and an NGO. The team's activities were moderated by Lancaster University staff and structured around: -
- A 3 day residential meeting to work intensively on the materials
- A period of individual work on specific subject areas (2 weeks was felt sufficient).
- A further 2-day residential meeting.
5.69 The authoring work involved 12 contributing authors, and 3 staff involved in technical production, with a further 2 specialists engaged over short periods for specific tasks ( e.g. animation). As we have already remarked, the current study has not focused on specific areas of the school curriculum. However the scope of the materials is likely to be similar and require a similar amount of effort.
5.70 Several of the tools are well suited to materials that are structured in a question-answer format: -
- Frequently Asked Questions
- Video Interviews
- Chat room- panel discussion
5.71 From the end-users point of view these tools should be considered as alternative routes to the answers to their questions, i.e. they should not need to guess which tool will answer which kinds of question. The toolkit specification enables the appropriate links to be put in place, but there is also a need to manage the production of question-answer materials.
5.72 Question-answer materials will also overlap with background information drawn from existing sources. Material that is structured around issues and options can readily be structured around questions and answers (respectively) simply by re-wording. That may be appropriate where the options are in dispute, or are to be debated at a later stage in the engagement process. In other words it may be confusing to present information structured around 'issues/options' if the options are historical positions on long-running controversies, but are not actually options for the current debate. (This is the corollary to the point made earlier that content management may best use an 'issues and options' structure because of their longevity).
5.73 Stakeholder group representatives interviewed on video are likely to also be appropriate as members of a panel, to answer questions live by chat room. There is also an overlap between the video interview and blog formats in that both need introductory biographical material, and interviewees may also be suitable blog authors ('bloggers'). Furthermore, video interviews transcripts may be included in a blog with appropriate links from FAQs, and may stimulate discussion on the Discussion Board.
5.74 The authoring process for a pilot project would need further planning and investigation to identify partner organisations who may be willing to provide materials for re-use, or people to collaborate on producing materials, take part in interviews, or moderate online discussion (etc.).
5.75 As a first step at least the 10 steps outlined below are likely to be involved: -
1. Consider which issues should be the focus of discussion and which should be presented as background to the discussion, i.e. as question-answer content. On that basis:-
a. Identify issues and options that should be used to structure an issue map and discussion board.
b. Identify 'frequently asked' questions and outline the scope of the answers.
2. Investigate what materials may be sourced from partner organisations and used with copyright clearance, what existing online materials may be linked to, and what may need to be produced.
3. Consider any elements that may best be conveyed in the form of graphical animations, and which core background information and controversies can best be presented in a game format, e.g. where: -
- Facts or opinions/positions are well established and are critical to understanding the discussion options;
- Information concerns abstract processes that would otherwise need expert knowledge to visualise.
4. Consider issues where more detailed personal statements would add value, for example where: -
- The relevance to young people is not obvious, and may be better conveyed by accounts of personal experiences.
- There are policy process issues that the target audience is unlikely to understand enough to ask appropriate questions, and may be better communicated in live question/answer sessions
- The issue is particularly controversial, it is difficult to convey facts without allegations of bias, and there are various legitimate positions to accommodate.
5. Identify and recruit from partner organisations: -
- Contributors to interviews, blogs, and live panels from stakeholder groups.
- Online moderators and facilitators for the discussion board, and other sources of online comment.
6. Place interview schedule in blog event calendar at least 2 weeks in advance. Invite questions from the target audience, by email to moderator, and plan interview questions accordingly.
7. Carry out interviews, on video where appropriate, then produce the materials as follows 17 : -
a. Encode video interviews in an appropriate digital format,
b. Edit into question/answer clips, and: -
c. Transcribe the clips.
8. Set up personal blogs; add biographical details, and links to/from video interviews.
9. Set up the video interview clips and, depending on content, add transcripts to FAQs or as Blog posts, with links to/from video interviews.
10. Analyse responses to FAQs and Blog, and consider new FAQs, interviews etc. ( i.e. back to step 1).
Using the tools in facilitated groups
5.76 The e-toolkit specification outlined in this chapter provides for a "group registration" option, to enable a group to make either individual or joint responses in the name of the group. In a facilitated group setting, there is a need to consider the most appropriate tools to use to help the group familiarise themselves with the issues and make a considered response.
5.77 Further work involving teachers and youth workers would be necessary to develop a session guide. This would advise on the most appropriate combinations of tools to use at different stages of a class/group discussion. As a starting point, we suggest: -
1. Facilitator leads brainstorming to identify areas of uncertainty and formulate questions. The e-engagement website home page, selected Video interviews or animated Game sequences could be used to introduce the topic and purpose of the e-engagement, as a stimulus for discussion.
2. Facilitator (or individuals using shared computers) finds answers to questions using Frequently Asked Questions.
3. Facilitator uses Issue Map to lead discussion about the pros and cons of the discussion options.
4. Responding to the discussion, the facilitator suggests a range of Video Interviews to choose from, to help the group appreciate particular positions.
5. Individuals use shared computers to read Blogs representing stakeholder views, then search the FAQs to answer their own specific questions.
6. Returning to the Issue Map, the facilitator leads discussion of the group members' points of view, and they decide whether to contribute to discussion as a group or as individuals.
7. Individuals use shared computers to read/contribute to Discussion Board, or give their assessment of a fixed range of options using a Questionnaire or Game, or the facilitator does this on behalf of the group as a whole.
Facilitation and Discussion Moderation
5.78 This section outlines the main principles and practice relating to moderation and facilitation of online discussion (in for example Blogs, Chat room or Discussion Board). The section is based on the authors' knowledge of best practice and previous experience of coordinating online consultations.
5.79 The management of e-engagement tools that enable discussion in public ( i.e. visible to other Internet users) requires a range of skills to deliver a number of different roles. However, it is by no means necessary that the same person carries out all these roles - it is possible to allocate specific roles to different people according to interest, time and availability.
5.80 To begin with it is useful to define the core roles: -
- Moderation may be defined as presiding over and imposing due restraint to ensure all comments posted online remain within reasonable limits, as set out in a Conditions of Use statement.
- Facilitation may be defined as the act of assisting or making easier the progress of the debate.
5.81 Collectively these roles aim to ensure a fair and impartial debate by enforcing viable rules and (in the case of facilitation) a more 'hands on' and visible role. This involves ensuring that the discussion stays focussed upon the topic, introducing new issues or material, providing additional information, stimulating participation and sustaining interest by asking and answering questions, making interventions or clarifying debate to support the deliberation process as necessary.
5.82 Many of the roles and tasks involved in managing e-engagement tools are simply directed towards creating a positive environment in which young people can engage in a good online debate. Characteristics of such an environment might be described as:
- Making newcomers feel welcomed and all contributors valued.
- Enabling young people to make 'contact' online with other young people (and conversely ensuring that physical contact details are not disclosed).
- Supporting ongoing collaboration that rewards individual effort by providing an experience that is greater than the sum of the individuals' contributions.
- Providing an environment for authentic conversations.
- Enabling a feeling of ownership.
- Coordination that enables young people to identify for themselves where the conversation is going and how to avoid conflicts.
- An online space where everybody involved builds social capital by collaborative action to improve each other's knowledge.
5.83 The overall goal is active citizenship and active discussion: young people from different backgrounds having conversations and arguments about a variety of issues and treating each other's opinions with respect. Online facilitation, like its face-to-face counterpart requires enthusiasm and attention to the participants, to create the right conditions for building momentum and encouraging participation.
5.84 Taking the above into account it would be preferable though not essential for the moderators and facilitators to be young people with some experience of group discussion (online or offline) in a youth forum or similar context. Potential partner organisations would need to be approached to help recruit young people to do this, and provide sponsorship to fund initial training and compensation for the young peoples' time and expenses. It seems likely that the role would appeal to students ( e.g. of Communication Studies) seeking work experience in the area.
Welcoming and providing help
5.85 It is important to welcome young people into a debate, particularly 'lurkers' (those who observe, but do not necessarily participate). Participants can be welcomed by encouraging them to be actively involved, letting them know the benefits of joining the discussion and offering support for overcoming any problems they may have. An attractive opening message should be used, which gives young people an incentive for taking part and encourages them to post regularly and interact with each other. It is useful to be clear about the overall objectives and outcomes of the consultation from the very beginning so that participants are clear that they are not just socialising and taking part in idle chatter, rather focused debate is encouraged.
5.86 Welcoming is an important social act, but unless the consultation organiser has access to individual private email addresses this is difficult to accomplish. If large numbers of young people are taking part in a debate then welcoming messages could flood the discussion space. Therefore, if sources for contacting participants directly are not available and if the discussion is a popular one, then a single opening message to welcome all participants may be the only viable option.
5.87 It may be useful after welcoming participants to encourage them to post a short piece about themselves in a section (thread) dedicated to this purpose. This would be like an introduction that gives a brief biographical sketch of each young person and their interests. The idea is to provide a space where participants and latecomers can come to get an overview of who is taking part. It can help with socialisation and the establishment of a productive group. However care needs to be taken that young people do not disclose personal details that would identify how to make physical contact with them.
5.88 It should not be assumed that all young people are completely comfortable with computers and using the Internet, many may need assistance in learning how to communicate online e.g. with navigation, how to post and how to reply. Participants should be made aware of all the relevant avenues for receiving help; for instance, how to get in touch with the facilitator and where to look for written online guidance. It may take some time for participants to get to grips with the online environment; therefore, patience and encouragement are necessary. However, it is also important not to overstate the personal help that a facilitator can provide as assumptions can be made in this area. Facilitators should be clear about their role and its limits.
5.89 Since young people tend to ask the same questions at different stages it is useful to have a FAQ (frequently asked questions) section that the coordinator can personally update as needed to avoid repetitive messages.
5.91 It is important to ensure that young people understand how the personal information they enter will be used (such as address information), and who will have access to it, for how long a time period. Consultation responses that can be attributed to individuals are potentially sensitive, and appropriate legal data protection and ethical guidelines must be followed. Without clear guidance, some young people may be discouraged from posting messages that express personal feelings, thus limiting the range of responses any discussion forum might receive.
Conditions of use
5.92 It is important to have a clear statement of what can, and cannot, be entered as responses to the e-engagement. Any 'conditions of use' statement must be clearly understandable to both moderators (who must be able to enforce these unambiguously) and young people alike, and made clearly visible. A plain English example could be:
A word of warning! Please be respectful of others and mind your manners! We will remove comments if we think they contain offensive or abusive language, if they say things about people or organisations that may not be true and could damage them, or if they are advertising.
Please take care of yourself by not to giving out any personal information, don't give anyone's email address, home address or phone number, don't take any risks! Use of this site means you acknowledge and accept these Conditions of Use.
5.93 Clearly, conditions of use will regularly require re-considering (if not re-drafting) to be responsive to practice and need. If questions are regularly being raised regarding the 'suitability' of messages, then the Conditions of Use may require strengthening. Similarly, if too many messages are contravening the Conditions of Use, it may be that either these are not visible enough to users or they are unrealistically stringent.
5.94 Moderators should apply rules in a fair, consistent and impartial manner but avoid being dogmatic. There is a fine line between freedom of speech and the need for rules against offensive behaviour. It is necessary to judge how productive individual comments are towards the overall debate, whilst trying to remain consistent.
5.95 It is relevant to re-state here the two main forms of moderation: pre-moderation and post-moderation. With pre-moderation all participant comments are first sent to a moderator who decides whether to accept them, and post them to the forum as publicly visible, based on the conditions of use. In the case of post-moderation all comments go straight to the discussion forum and are publicly visible. The moderator then monitors comments on a regular basis (typically once in every 24 hour period), removing any that breach the Conditions of Use.
5.96 For e-engagement with young people post-moderation is generally used. Although it clearly has a higher risk factor than pre-moderation, it does enable an immediate experience for the young person and thus enhances the appeal. Typing a message and clicking 'send' or 'post' on a web site results in the user's message appearing seconds later. The act of making visible is an important part of free public speech. Pre-moderation not only introduces a time delay, but also can suggest that speech isn't public, or free, despite what the e-engagement description might say. This may be particularly so where there is a history of distrust in government information or action.
Level and mode of interaction
Keeping track of the debate
5.97 It is recommended that messages are checked and if necessary replied to at least once every day (five days a week), depending on time-scales and the intensity of the debate. It is often found that the response rate of a facilitator is reflected in that of the young people. If the facilitator is silent, not only will participants be quiet too, but they may disappear altogether. On the other hand, too many responses from the facilitator ( e.g. replying to every message) may create an unmanageable level of messages for young people to read as well as placing the facilitator at the centre of attention.
5.98 In order to sustain young peoples' e-engagement a regular level of activity is necessary. Timing is important when responding to messages; reply immediately, acknowledge and reply later, wait and reflect before replying or don't reply at all. It is advisable to reply straight away to messages that are straightforward, others may require more thought, and during the time it takes to respond other young people may have already done so. Ignoring someone's message is generally not very encouraging to young people. In any case facilitators' messages should be succinct, brief and friendly.
5.99 How the facilitator interacts during a debate is important as they can help young people to deepen their thinking, increase understanding of complex issues, mediate disagreements and enable participants to reach conclusions. However, the facilitator's involvement must always be overt, rather than covert. They must declare their interest in the debate, have a publicly visible name and not intervene without identifying themselves as the facilitator for that discussion. Facilitators should not work inside discussions as covert operatives. Equally, the facilitator cannot be seen to take any particular side or approach in the overall course of a discussion. Though for discursive reasons they may play devils-advocate or idiot savant, through the overall course of a discussion they must be equitable to different points of view. Where possible, the facilitator should avoid strong statements of opinion.
5.100 Facilitator involvement can be controversial, especially in formal consultations - since it can be argued that the facilitator has unduly influenced opinions. It is important to balance facilitator involvement so that they are not 'central' to discussions, are not always involved in every discussion and are not involved in starting many new topics themselves. The facilitator intervenes by response, rather than instigates by example.
5.101 Disagreements are common online and it is important to reply sensitively to upset or disgruntled participants. Remaining neutral whilst dealing with such situations can be difficult. One approach to deflate conflict is to first restate the participant's point of view in order to acknowledge it, before making a suggestion to consider alternative viewpoints, or challenge them. It may be best for the facilitator to avoid making assertions or being absolute by speaking from a personal perspective or making it clear whose perspective the coordinator is speaking from. It may be helpful to end such messages with an open question, and a request for confirmation or other's views on the matter.
Gathering and providing information
5.102 During a debate information can also be used to spark new interest and new lines of argument (this is in addition to information already provided on-site). Information can be typed out or sources can be referred to e.g. web addresses ( URL's) and why they may be interesting.
5.103 However, the facilitator should not be the sole (or main) provider of information. A major reason cited for not posting to online discussions is the feeling that there is nothing new to contribute to what is already there. Consequently, it is best for the facilitator to encourage other young people to provide or refer to information they have sourced rather than rushing to answer all questions raised. Asking questions and encouraging participants to source and share facts can achieve this. To create further debate from new information young people can be encouraged to indicate whether or not they agree.
Weaving and summarising
5.104 The Hansard Society adopts the terms weaving and summarising18 to refer to two e-facilitation skills. Weaving is the process of gathering together elements of various messages in order to present a new idea, focus the discussion and show participants that they have been heard. This is usually achieved by selecting various quotes from the discussion (crediting the author) and meshing them together in a way that creates a new and possibly unforeseen perspective. This can help young people to reflect on what they have written and possibly see it in a new light or a new line of argument.
5.105 Summarising is the act of drawing together the main points of a discussion. In a consultation it is a very useful tool for acknowledging participant's contributions and allowing young people to feel a sense of accomplishment by being recognised. Summaries at regular intervals throughout the e-engagement can provide a useful resource for latecomers, as well as acting as a recapitulation of the debate for participants. This can help them to reflect on what has happened so far and re-focus the discussion. Important areas that have not yet been covered may become apparent and new debate can be sparked. Some conclusions can be made in summaries, which can be agreed upon or negotiated further. Summaries help to finish off and round up a debate; they also provide a useful record for compiling final reports for partner organisations.
Evaluation Criteria and Indicators
5.106 In this section we consider how evaluation criteria and indicators of success for the e-engagement should be elaborated from those used in the study.
5.107 As mentioned in the Outline Specification earlier in the chapter, we have considered 6 main evaluation criteria to assess the e-engagement from the perspective of the 'owners' of the engagement process (the Executive), and its target audience of young people and others in Scotland. These were: -
- Ease of use
- Appeal or enjoyment
- Satisfaction that the tools have met their purpose(s).
- Take-up by a representative cross-section, in demographic terms, of young people in Scotland.
- Informed contributions to the debate on managing radioactive waste by young people in Scotland;
- Better appreciation of the views of young people in Scotland on managing radioactive waste.
5.108 Further work is needed to check that these criteria are an accurate reflection of how young people and other stakeholder groups would judge the success of an e-engagement process. However we can propose some candidate data sources. These are based on prior experience and on the facilities that we have specified for the toolkit.
5.109 The sources of data that we recommend are: -
1. Satisfaction ratings: from (a) users of the toolkit who may rate their satisfaction online; and (b) responses to questionnaires administered to targeted users in evaluation workshops set up for that purpose.
2. Comments from semi-structured interviews with targeted users: these can be analysed to identify whether and how the tools address barriers to e-engagement faced by those who would ordinarily not take part in the debate, or conversely may be excluded by lack of technical skill or Internet access.
3. Usability data: observations of a pilot system in use by the targeted users, made in evaluation workshops where young people (or other stakeholders) are invited to use the tools package to carry out the activities it is designed for. Accurate recording of what the participants do with the tools and say about them allows analysis of the nature and severity of problems encountered.
4. Registration data: users who register in order to take part in the e-engagement are asked to provide demographic data, allowing a comparative analysis with national or local statistics.
5. Content analysis: Analysis of the quality of the online contributions to the debate, which can be rated by the debate moderators or facilitators, according to the extent to which they are informed by the toolkit content.
6. Usage data: the automatic recording of web pages visited is a standard feature of web server software, and we have specified various features to support access to the relevant data and its analysis. For example, a simple measure of the appeal of an online tool is calculated by dividing the number of times action is taken ( e.g. a comment posted) by the number of visits to the corresponding page.
Recommendations for Next Steps
5.110 To conclude the study we reflect on its limitations and the need for further work to address those limitations and take the e-engagement process forward. We believe the current study provides a sound basis to further develop and pilot an e-engagement package, complemented by further research. We therefore recommend the following: -
1. Before further studies are undertaken a Youth Reference Group which is broadly representative of young people in Scotland should be established to help enlist the support of their peers, including relevant stakeholder groups. The input of on-going initiatives such as the Commissioner for Children and Young People, Scottish Youth Parliament, Young Scot, Dialogue Youth and Highland Youth Voice would be highly desirable.
2. The tools need to be developed to the same degree of functionality. In this study some were presented in fully working form, while others were presented as one or two sample screens intended to illustrate a working tool.
3. Extensive discussion of the content and evaluation of a pilot e-engagement is needed, to secure the necessary partnerships with government, industry and non-government stakeholders (local as well as national). These would focus on developing the authoring process, support for group facilitators, evaluation criteria and success indicators, and incentives for young peoples' continued involvement in the engagement process.
4. Further scenario-based workshops need to be conducted alongside any pilot e-engagement. In this study the opportunities were limited both by time and in their realism, in that neither the tools nor their content were fully developed. Our workshops were held in school, community group and university settings that we would expect the tools to be used in, but the early stage of development meant that participants' responses were not based on the experience of using the tools as in a real-life situation.
5. There is a strong case for using field study methods to observe and analyse a pilot e-engagement package being used in schools and/or community group settings. A focus on actual behaviour in context, as well as views expressed in individual discussion or group workshops, would enhance the validity of the findings. This would depend on further development and piloting of the e-toolkit.
6. A 'one-size fits all' product may detract from the success of any future stakeholder engagement exercise, since differences in language and culture (if not technical skill) are likely to exclude children on the one hand and older people on the other. Although similar tools may well be appropriate, the needs of these groups should be considered separately. In particular, different age groups are likely to use different approaches to find information and their varying needs require further research.
7. The tools and content should nevertheless be designed to cater for the widest possible age group, by finding an appropriate balance between accessibility and wide appeal. Accessibility means using plain English written for a 9-12 reading age, as well as complying with design guidelines for people with disabilities. Translations to minority ethnic languages may also be desirable. There is a trade-off between accessibility and using the multimedia elements of computer games to enhance the general appeal of the tools. The trade-off should be managed by using transcripts and still images imaginatively, with the guidance of the Steering Group.
8. The organisational mechanisms needed to support the continued involvement of content authors, moderators and facilitators should be explored so as to better understand the personnel resources required.
9. Given rapid change in technology the selection of tools for e-engagement should be reviewed and there is a particular need for further research on the use of the issue map format. Although an established technology, this has not been previously applied in e-engagement with young people. Also, various other technologies should be considered but would require more substantial investment to pilot and the involvement of a commercial service provider. These include for example interactive digital TV, and voting by text message.