The impact of Science Literacy delivery methods - what works? Single mechanism analysis Working Paper Workshops | Group 1. Events, meetings, performances V1.0 | 29 December 2018

1. Network for Information and Digital Access
The impact of Science Literacy delivery methods - what works?
Single mechanism analysis report
Workshops | Group 1. Events, meetings, performances
V 1.0 | 29 December 2018
WORKING PAPER
Team members
Valentina De Col | Lead Researcher from June 2018
Nipuni Jayarathne | Young Scientists Society, Sri-Lanka
Kamran Naim | Executive Director, NIDA
Carol Priestley | Senior Advisor, NIDA
Nadun Thilakarathne | Young Scientists Society, Sri-Lanka
Carol Usher | Manager, Publicity & Dissemination, NIDA
Attribution 4.0 International (CC BY 4.0)
Co-financed by Evergreen Education Foundation (EEF) and NIDA

2. ii
Executive Summary
1. Introduction
1.1 This report presents a synthesis of the proven impact, strengths and weaknesses of
workshops in delivering science literacy.
1.2 This individual analysis is situated within the framework of a broad study of science literacy
aimed to establish what has been proven successful in the field; with the objective to
promote and adapt good practices and fill gaps in knowledge about ‘what works’.
1.3 The full study identified 42 single-mechanism approaches, 2 composite approaches and 1
related approach. ‘Workshops’ is categorized within Group 1 relating to ‘Events, meetings,
performances’, with some overlaps within Group 2, ‘Education and training – including
online’.
2. Methodology for resource discovery and analysis
2.1 From October 2017 to May 2018, the research team surveyed existing resources through
retrieval via research databases, subject databases, open access repositories and through
contact with interested organisations, institutions and individuals.
2.2 The resources were divided into impact assessments (IAs) and descriptive resources. For the
purposes of analysis, only those published during the years 2013 -2018 were utilised. Each
resource was read in detail, significant data was extracted and entered into a specifically
developed database. An example of the database mask is included in Appendix A.
2.3 Although the total number of resources located was not designed to be exhaustive or
definitive, the resources captured in this research are limited to those available in the
English language and to translations that had already been made from other languages into
English.
3. Overview of results
3.1 Over 2,100 IA studies and descriptive resources were identified in the full research process,
of which 30 relate specifically to ‘workshops’; of which 11 were published between 2013-
2018.
3.2 The subject coverage included agricultural science, food chemistry, health promotion and
medicine, and statistics. The countries included in the studies were Japan (2), New Zealand
(2), United States of America (2), but also with examples from Australia (1), Germany (1),
Honduras (1) and South Africa (1).
3.3 The delivery models involved were equally implemented between formal education (50%)
and non-formal education (50%). The target audiences were 57.1% education and training
28.6% workers and 14.3% population groups. The audiences were reached through
educational institutions (42.9%), health facilities (28.6%), ‘others’ (14.3%), non-
governmental organisations (7.1%) and ‘various’ (7.1%).
3.4 The approaches to conducting assessment within the resources were found to be primarily
mixed-method, followed by qualitative. The most common data collection approaches
involved written or online surveys.

3. iii
4. Discussion
4.1 Examples of the use of ‘workshops’ as a science and health delivery mechanism were
observed in a diversity of contexts, worldwide. These workshops utilized different tools (flip
charts, powerpoint presentations, and peer-delivery methods) and varied in duration from a
couple of hours, to one-day, or five-lessons.
4.2 Fewer than half of the studies investigated the adoption of ‘Workshops’ for scientific, but
not specifically health-related topics (including information literacy, biology, agricultural
literacy and statistics education). The remaining studies related primarily to interventions in
health literacy.
4.3 The reviewed IA’s cited positive impacts on awareness, knowledge and understanding in all
the health promotion interventions.
4.4 The studies cited increased interest and engagement through workshops, as well as
developing a positive influence on the emotional and intrinsic components of students’
interests in science due to the characteristics of authenticity, reference to everyday life and
understandability, etc.
4.5 The studies further demonstrated attitudinal changes amongst workshop participants:
helping to increase positivity and confidence in their respective professional practices.
4.6 No explicit behavioural changes were reported in the reviewed studies.
4.7 A number of studies cited workshop participants adopting skills learnt in a variety of
contexts, including nursing and health and information literacy during the follow-up period.
4.8 Other diverse impacts were reported relating to science education, including increased
student confidence in information literacy, improved teacher confidence, and increased
excitement and enjoyment of science content. Additionally, studies reported improved
readability and content of participants’ written materials, improved performance by
students in terms of checklist scores and reading level of written instructions, and increased
likelihood of using methods imparted during the workshops.
4.9 The evidence on the use of a workshop format to deliver scientific and health concepts
highlighted a range of strengths in varying contexts; not many references to weak points for
this mechanism were cited.
4.10 With regard to feasibility, some studies cited time for implementation of content as a
concern. In others, the participants willingness to cover their own costs for travel to attend
workshops week after week was considered an indication of the positive perceived impact of
the workshops. Resource availability and practical limitations were considered important
factors for designing more effective workshop interventions.
4.11 Authors suggested that workshops could be optimized by running concerted multi-workshop
series designed in advance and spread over time. Others suggested the need for programme
evaluation modules to be developed to provide summative as well as formative evaluations.
Other studies recommended an increased use of visual aids, particularly among rural
communities, as well as increasing the number and frequency of workshops and time for
general discussion and group-based activities.

4. iv
5. Conclusions
5.1 ‘Workshops’ are used to support the delivery of a variety of scientific subjects and examples
of their application were found to be widely used both in formal and non-formal education.
5.2 The target sectors of this mechanism were also diverse, from schools and universities to
local communities and health workers.
5.3 The evidence suggests that workshops are a versatile mechanism for delivering science and
health-related topics and are shown to have been successfully adopted as a training
platform in a number of diverse settings, subject areas and resource contexts.
5.4 Workshops offer many possibilities of modification and adaptation, for example, in terms of
ways of delivering the concepts and interactivity with the audience(s). The adaptability of
the ‘workshop’ format and the possibility of its combination with other mechanisms, e.g.
videos or e-learning, might represent an interesting advantage, also in terms of costs and
coverage of the intervention.
5.5 Empowering communities and professionals in order to raise consciousness on different
topics appeared to be a common goal for several of the interventions.
5.6 To date, the number of impact assessment and evaluation studies of this mechanism is
limited and more substantive conclusions that support and illustrate the full potential and
versatility of workshops may be forthcoming in the future.

5. v
CONTENTS
Executive Summary ......................................................................................................................... ii
Mechanisms, groups and approaches ........................................................................................ 4 1.
Methodology for resource discovery and analysis ..................................................................... 5 2.
Search method ....................................................................................................................... 5 2.1.
Data extraction for the analysis ............................................................................................. 6 2.2.
Limitations of the resource discovery .................................................................................... 6 2.3.
Overview of results ................................................................................................................... 6 3.
Total number of resources discovered .................................................................................. 6 3.1.
Scientific subjects .................................................................................................................. 6 3.2.
Countries involved in the studies ........................................................................................... 7 3.3.
Educational delivery models .................................................................................................. 7 3.4.
Target sectors ........................................................................................................................ 8 3.5.
Delivery institutions ............................................................................................................... 8 3.6.
Approach to data collection ................................................................................................... 9 3.7.
Sampling technique and sample size ..................................................................................... 9 3.8.
Discussion ................................................................................................................................. 9 4.
Contexts of use ...................................................................................................................... 9 4.1.
Impacts ................................................................................................................................ 12 4.2.
4.2.1. Awareness, knowledge or understanding ............................................................................ 12
4.2.2. Engagement or interest ........................................................................................................ 12
4.2.3. Attitude ................................................................................................................................ 12
4.2.4. Behaviour ............................................................................................................................. 13
4.2.5. Skills ...................................................................................................................................... 13
4.2.6. Others ................................................................................................................................... 13
Strengths .............................................................................................................................. 14 4.3.
Weaknesses ......................................................................................................................... 16 4.4.
Costs and feasibility ............................................................................................................. 16 4.5.
Suggestions for improved methodologies and for future studies ....................................... 17 4.6.
Conclusions and overview ........................................................................................................ 17 5.
APPENDIX A: Example of data input mask ..................................................................................... 19
APPENDIX B: Selected bibliography ............................................................................................... 21

6. 4
Mechanisms, groups and approaches 1.
During the first part of the Desk Research phase of this project (i.e. Task 1), the research team
identified 42 single-mechanism approaches, 2 composite approaches and 1 related approach that
were relevant to the delivery and dissemination of scientific information. The list of single
mechanisms was further organised into 7 thematic groups, as presented in Table 1.
The subject of this report is ‘Workshops’, included in Group 1 relating to ‘Events, meetings,
performances’, but with some instances that could be included within Group 2, ‘Education and
training – including online’.
Single mechanism approach Group
Exhibitions, Expo, Festivals, Movies, Picnics,
Science Fairs, Seminars, Talks, TED Talks, Theatre,
Workshops
1. Events, meetings, performances
Colloquia, Courses, Curricula, E-learning, Webinars
2. Education and training – including online
Animations, Books, Brochures, Cartoons, Comics,
Games, Graphics, Posters, Publications, Radio,
Reports, TV, Videos
3. Traditional publishing and journalism –
print and broadcast
Competitions, Experiments, Makerspaces, Mobile
classrooms, Mobile laboratories
4. Activities and services
Blogs, E-books, E-zines, Mobile Apps, Podcasts, Social
media, Websites, Wikis
5. Online interactions
Composite approaches
Multiliteracies
Multimodalities
Related approach
Citizen Science
Table 1. Organisation of the delivery approaches of science literacy adopted in this research.
For the purposes of this study, the definition of ‘Workshop’ broadly refers to ‘a meeting at which
a group of people engage in intensive discussion and activity on a particular subject or project’1
; the
definition can also be extended to encompass ‘Training Workshops’, which implies ‘a type of
interactive training where participants carry out a number of training activities rather than passively
listen to lectures or presentations’2
.
1
“Workshop”, Oxford Dictionary, Accessed October 22, 2018, https://en.oxforddictionaries.com/definition/workshop
2
“Training workshop”, Wikipedia, Accessed October 22, 2018, https://en.wikipedia.org/wiki/Training_workshop
3
The research and analysis methodologies will, however, be available from NIDA in English, French and Spanish in order
that others may utilise and/or translate and adapt, replicate and extend the coverage.
2
“Training workshop”, Wikipedia, Accessed October 22, 2018, https://en.wikipedia.org/wiki/Training_workshop

7. 5
Methodology for resource discovery and analysis 2.
Search method 2.1.
From October 2017 to May 2018, the research team carried out an extensive process of resource
discovery to survey existing works and impact studies that could provide valuable evidence on the
impact of the identified science delivery approaches and mechanisms.
The search was carried out by retrieving documents and articles from a wide range of sources,
including research databases, Google Scholar, ResearchGate, subject databases and open access
repositories. The use of non-boolean keyword combinations returned a consistent number of
relevant results from prominent academic journals and online library databases (e.g. ERIC, Frontiers,
JCOM, MedLine/PubMed, Nature, NCBI, Wiley Online Library, PLOS, SAGE, ScienceDirect, Springer,
Web of Science). Moreover, the findings were complemented by relevant resources such as theses
and manuscripts retrieved from university repositories, reports and case studies from different
organisations and NGOs. In addition, contact was made with researchers via the ResearchGate
community and single individuals from NIDA’s Facebook and LinkedIn pages who expressed interest
in the research and directly contributed by providing annotated bibliographies for their fields of
expertise.
The resource discovery was performed by combining the mechanism name with 1 of the 5
keywords/synonyms for ‘impact’ and 1 of the 10 literacies and sub-sector literacies identified by the
Team, one combination at a time. The search strategy is exemplified in Table 2.
Approach Terms for impact Literacy and sub-sector literacies
[mechanism name]
e.g. theatre
Impact
Impact assessment
Assessment
Performance measurement
Outcomes
Agricultural Literacy
Chemistry Literacy
Climate Literacy
Computer Literacy
Earth Science Literacy
Food safety Literacy
Health Literacy
Nutrition Literacy
Science Literacy
Statistical Literacy
Table 2. Search strategy using keywords combinations.
This method generated a total number of 50-word combinations (to illustrate one single
example: ‘theatre impact science literacy’) for each of the science delivery mechanisms investigated.
The articles and materials selected for the analysis span between 2013 and 2018 and were
initially sorted into two main groups: one containing impact assessments that provide a qualitative,
quantitative or mixed method (both qualitative and quantitative) research approaches to data
collection; and a second including different typologies of descriptive resources, e.g. reviews, guides,
handbooks, reports. Resources were organised by mechanism and the principal metadata (e.g. title,
author, date, scientific subject) saved on a Microsoft Excel® spreadsheet prior to database import.

8. 6
Data extraction for the analysis 2.2.
The identified impact assessments were subsequently uploaded to a Microsoft Access® database,
developed by the Team to collect relevant information from each study. An example of the database
mask for data entry is included in Appendix A. Each article was read in detail, and significant data
were extracted, entered into the database and used as core information to carry out the analysis.
Limitations of the resource discovery 2.3.
The resource discovery was limited to resources available in the English language, and studies in
other languages were only included where translations had already been made to English3
. Another
intrinsic limitation may lie within the search methodology, particularly on the keyword
combinations, and may explain a low number of articles for some of the mechanisms investigated.
Moreover, the total number of resources located is not meant to be exhaustive or definitive. It is
a work in progress that attempts to offer a synthesis of examples spanning across different literacies
and sub-sector literacies, with no geographical limitations, with the aim to contribute to the
understanding of science, its applications, and to the promotion of science literacy.
Overview of results 3.
Total number of resources discovered 3.1.
Over 2,100 impact assessment studies and descriptive resources were identified in the full research
process, of which 30 relate specifically to ‘Workshops’. However, for the purposes of analysis a
decision was taken to concentrate on the 11 IA studies and descriptive resources published between
2013-2018, as presented in Table 3, to provide currency of information.
Resources (2013-2018) Number
Impact assessments 10
Descriptive resources 1
Total no. of resources analysed 11
Table 3. Total number of resources analysed.
Scientific subjects 3.2.
The main subjects of the impact assessments are synthesized in Table 4. The systematic
categorisation of science branches was retrieved from Wikipedia4
and customised for the purpose of
the research.
3
The research and analysis methodologies will, however, be available from NIDA in English, French and Spanish in order
that others may utilise and/or translate and adapt, replicate and extend the coverage.
4
“Branches of science”, Wikipedia, Accessed January 26, 2018, https://en.wikipedia.org/wiki/Branches_of_science

9. 7
Main subject area Detailed subject References
Applied / Healthcare
Health promotion
Goto et al. 2014; Sookhoo
2014; Goto, Lai, and Rudd
2015; Sebbens et al. 2016;
Morgaine et al. 2017
Medicine Bloom-Feshbach et al. 2015
Interdisciplinary
Statistics North, Gal, and Zewotir
2014
Life / Biology
Wegner and Strehlke 2015
Agricultural science Edwards 2016
Physical / Chemistry Food chemistry
Baroutian and Kensington-
Miller 2016
Table 4. Main scientific subjects of the resources analysed.
Countries involved in the studies 3.3.
The countries where the studies have taken place are listed in Table 5 and can be visualized on the
world map in Figure 1.
Countries No. of studies for each country
Japan, New Zealand, United States of America (US) 2
Australia, Germany, Honduras, South Africa 1
Table 5. Number of impact assessment studies for each country.
Figure 1. Geographic distribution of the impact assessment studies [Map generated with traveltip.org].
Educational delivery models 3.4.
The educational delivery models employed in each study are presented in Table 6. The Team
summarised specific models of delivery, whether formal, non-formal or programmatic (namely,

10. 8
embedded in a programme) and included, for convenience of future discussion, a distinction
towards examples related to health literacy. The relative percentage displayed in the Table is
expressed over the total number of impact assessment analysed (n=10).
Model of delivery Sector References
Formal education
(5) 50,0%
Pure & Applied
Sciences
North, Gal, and Zewotir 2014; Wegner and Strehlke 2015;
Baroutian and Kensington-Miller 2016; Edwards 2016
Health-related Bloom-Feshbach et al. 2015
Non-formal education
(5) 50,0%
Health-related
Goto et al. 2014; Sookhoo 2014; Goto, Lai, and Rudd
2015; Sebbens et al. 2016; Morgaine et al. 2017
Table 6. Models of delivery and relative percentage over the total number of resources discovered analysed
(n=10).
Target sectors 3.5.
The target sector(s) addressed by each individual study is presented in Table 7. The categorisation
used was drawn from the ILO (International Labour Organisation) Taxonomy5
list, which was reduced
and simplified. Some articles were attributed to more than one target sector.
Main target sector Sub-divided target sector References
Education & Training
(8) 57,1 %
Early childhood education Edwards 2016
Primary education
North, Gal, and Zewotir 2014; Wegner and
Strehlke 2015; Edwards 2016
Secondary education
North, Gal, and Zewotir 2014; Wegner and
Strehlke 2015
Bachelor’s or equivalent level Bloom-Feshbach et al. 2015
Master’s or equivalent level Baroutian and Kensington-Miller 2016
Population groups
(2) 14,3 %
Local communities Sookhoo 2014; Morgaine et al. 2017
Workers
(4) 28,6 %
Professional workers
Goto et al. 2014; Goto, Lai, and Rudd
2015; Sebbens et al. 2016
Rural workers Morgaine et al. 2017
Table 7. Target sectors and relative percentage over the total number of instances.
Delivery institutions 3.6.
The delivery institutions promoting ‘workshops’ are presented in Table 8 as identified within the
wide categorisation identified by the research team.
5
“ILO Taxonomy”, Accessed January 26, 2018, http://www.ilo.org/dyn/taxonomy/taxmain.showSet?p_lang=en&p_set=1

11. 9
Delivery institution References
Educational institution
(6) 42,9 %
School Goto, Lai, and Rudd 2015
University
Goto et al. 2014; North, Gal, and Zewotir
2014; Goto, Lai, and Rudd 2015; Edwards
2016; Morgaine et al. 2017
Health facility
(4) 28,6 %
Healthcare centre
Goto et al. 2014; Baroutian and
Kensington-Miller 2016; Sebbens et al.
2016
Medical school Sookhoo 2014
Non-governmental organization
(1) 7,1 %
Not-for-profit
organisation
Wegner and Strehlke 2015
Various
(1) 7,1 %
Bloom-Feshbach et al. 2015
Others
(2) 14,3 %
Baroutian and Kensington-Miller 2016,
Morgaine et al. 2017
Table 8. Delivery institutions and relative percentage over the total number of instances.
Approach to data collection 3.7.
Of the 10 impact assessment studies, 7 used a mixed-method approach and 3 were primarily
qualitative.
For these studies, data collection approaches involved written or online surveys (6 studies),
experiments (4), interviews (3) and observations (1).
The data collection tools or scales employed included Likert scales (5 studies), questionnaires (4),
checklists (1), multiple choice response (1) and written or graphic responses (1).
The statistical approaches used involved hypothesis testing such as t-tests (3 studies) and chi-
squared (1). A few studies specified the software tool or app used for analysis: Microsoft Excel (2),
Stata (2), SPSS 21 (1) or others (1).
Sampling technique and sample size 3.8.
Amongst the sampling techniques employed in the studies under consideration, there were
convenience sampling (7 studies), random (2) and systematic sampling (1).
Sample sizes ranged from 10 to 430 (114 mean; 47 median) and 6 studies have a sample size
under 100 participants.
Discussion 4.
Contexts of use 4.1.
Examples of the use of ‘workshops’ as a science and health delivery mechanism were observed in a
diversity of contexts, worldwide. Different tools were employed, including flip-charts (Sookhoo
2014), Powerpoint presentations (Bloom-Feshbach et al. 2015), laboratory activities (Wegner and

12. 10
Strehlke 2015) and peer-delivery methods (Morgaine et al. 2017). The workshops also varied in
duration, from a couple of hours (Goto, Lai, and Rudd 2015; Edwards 2016), to one-day (Wegner and
Strehlke 2015) or five-lessons, being one lesson each weekend for a total of five weekends (North,
Gal, and Zewortir 2014).
Fewer than half of the studies (North, Gal, and Zewotir 2014; Wegner and Strehlke 2015;
Baroutian and Kensington-Miller 2016; Edwards 2016) investigated the adoption of ‘Workshops’ for
scientific, but not specifically health-related topics.
Baroutian and Kensington-Miller (2016) investigated the usefulness of an information literacy
workshop designed and integrated within a Food Processing postgraduate course at the University
of Auckland (New Zealand) to support international students with diverse backgrounds,
qualifications and learning approaches. In Germany, Wegner and Strehlke (2015) presented the
project ‘Biology Up Close’, a workshop promoted by the Bielefeld University (Germany) to pupils
from regional primary and secondary schools. The workshop offered the opportunity for participants
to gain an understanding of scientific phenomena and experience biology at university laboratories,
with the additional aim to stimulate interest in becoming scientists. Through the workshop, the
participants were introduced to an interesting, relevant branch of science by providing them with
the opportunity to conduct experiments themselves and question the items they used every day.
The workshops were planned, organized and held by teacher students, therefore helping them
concurrently to improve and reflect on their teaching skills (Wegner and Strehlke 2015). In the US,
Edwards (2016) evaluated teachers’ perceptions on the implementation and impact of professional
learning in an agricultural literacy workshop training named ‘Dig into Learning: An Agricultural
Literacy Innovation’, addressed to students from the kindergarten through to the 5th
grade. The
programme had the main objective to encourage teachers to use agriculture as a context for
teaching and learning.
North, Gal, and Zewortir (2014) presented an intervention in South Africa based on a capacity-
building workshop series in the area of statistics education, the aim of which was to improve
teachers’ familiarity and confidence with statistics topics in the classroom.
The remaining studies (Goto et al. 2014; Sookhoo 2014; Bloom-Feshbach et al. 2015; Goto, Lai,
and Rudd 2015; Sebbens et al. 2017; Morgaine et al. 2017) have, as a common denominator,
interventions in health literacy.
Bloom-Feshbach and colleagues (2015) described the case of a US skills-based health literacy
workshop for fourth-year medical students as part of an Objective Structured Clinical Examination
(OSCE), integrated into the curriculum of an internship preparatory clerkship. This evidence-based
initiative included an interactive workshop on screening tools and communication techniques to
care for patients with low health literacy and an innovative OSCE that assessed the application of
these skills during transitions of care, such as hospital discharge. The workshop was designed to be
interactive, with many opportunities for student participation and role-play. It emphasised the
importance of health literacy, advised ways to actively assess health literacy and allowed for the
practice of communication skills, e.g. the use of plain language and teach-back6
(Bloom-Feshbach et
al. 2015).
6
“The teach-back method, also called the "show-me" method, is a communication confirmation method used by
healthcare providers to confirm whether a patient (or care takers) understands what is being explained to them. If a
patient understands, they are able to "teach-back" the information accurately”.
“Teach-back method”, Wikipedia, Accessed October 24, 2018, https://en.wikipedia.org/wiki/Teach-back_method

13. 11
Two examples of healthcare promotion were reported in Japan in response to the 2011 nuclear
disaster in Fukushima. Goto and colleagues (2014), and Goto, Lai, and Rudd (2015) presented the
case of health literacy training workshops designed to augment the communication skills of
radiation-related health information to public health nurses. The workshop aimed to improve
nurses’ knowledge and skills with regard to radiation, parenting support and to provide
opportunities to share information among peer nurses (Goto et al. 2014). The primary goals of these
interventions were to change communication practices and norms among public health nurses and
improve access and understanding of health and science related information for community
residents (Goto et al. 2014; Goto, Lai, and Rudd 2015). In particular, the training program was based
on Rudd’s “Eliminating Barriers - Increasing Access Workshop” and adapted the American’s
experience to respond to the needs of public health nurses in Fukushima. It was also shaped by
findings from health literacy studies indicating that a good deal of health information was
inaccessible to the general public, in part due to the use of unexplained jargon, sophisticated
numeric concepts and scientific terms (Goto, Lai, and Rudd 2015). It was reported to be the first
initiative in Japan aimed to build health literacy skills of health service providers for supporting
community residents in gaining access to the needed health information (Goto et al. 2014).
One study from Central America (Sookhoo 2014) sought to explore the use and need for health
education information presented in a 'charla’, the Spanish word for ‘chat’ or ‘informal health
workshop’, in two similar rural Honduran communities with low literacy levels. The author aimed to
assess the preferences of rural Hondurans for health education information received in mobile
medical clinic ‘charlas’ and to examine if these preferences matched the previous health education
models implemented by a non-governmental organization (NGO). The ‘charla’ covered a diversified
range of health topics, including maintenance of water, hand, oral, and personal hygiene, proper use
of latrines, cervical and prostate cancer (Sookhoo 2014).
In New Zealand, Morgaine and colleagues (2017) evaluated the impact of an interactive, skills-
based workshop called ‘GoodYarn’, focused on raising mental health literacy in rural communities,
primarily farmers, their families and farm workers, as well as the “farmer facing” workforce. The
project utilised a peer delivery model, where the workshops were delivered by people with both a
farming or rural industry background and a personal or close personal experience of mental illness.
‘GoodYarn’ focused on building key aspects of mental health literacy specifically in the rural context,
acknowledging the particular culture as well as the stresses of living and working in a rural
environment and supporting community members that might experience geographic, attitudinal and
service configuration barriers to accessing mental health services (Morgaine et al. 2017). Along the
same lines, Sebbens and colleagues (2016) developed and evaluated a Mental Health in Sport (MHS)
workshop targeted to coaches and support staff working with elite athletes and teams in Australia.
In the interface between two different delivery mechanisms, the study of Plank, Molnár, and
Marín-Arraiza (2017) offered an example of a science video workshop as a training programme
designed to provide scientists with knowledge and tools for creating and publishing video abstracts
about their own research. This appears to be particularly interesting since, in turn, ‘videos such as
recordings of conferences, workshops, lectures, experiments and projects all have the potential to
make the knowledge gained from scientific communication more useful, by providing a deeper
understanding of the experiential aspects of the corresponding contributions published as text’
(Plank, Molnár, and Marín-Arraiza 2017).

14. 12
Impacts 4.2.
The impacts identified in the studies were organised using impact categories proposed by the
evaluation framework of the National Science Foundation7
.
4.2.1. Awareness, knowledge or understanding
Positive impacts were observed in all the health promotion interventions considered (Goto et al.
2014; Sookhoo 2014; Goto, Lai, and Rudd 2015; Sebbens et al. 2016; Morgaine et al. 2017). In Japan,
Goto and colleagues (2014) reported an improvement in the understanding of citation and
referencing management, similarly to the experience described by Goto, Lai, and Rudd (2015), who
noted an overall gain in knowledge about health literacy in nurses participating to the workshops.
In two studies taking place in rural communities, Sookhoo (2014) found that the great majority of
the Honduran patients understood that the workshops (‘charlas’) were meant to improve health in
their communities; whereas the ‘GoodYarn’ workshop in New Zealand had a significant positive
impact on participants’ knowledge and awareness about mental illness and well-being (Morgaine et
al. 2017). An improving of participants’ knowledge of the signs and symptoms of mental illness,
especially depression and anxiety, emerged as a result of the MHS workshop with Australian coaches
and sport staffs (Sebbens et al. 2017).
Furthermore, the intervention presented by Edwards (2016) reported that most of the teachers
felt more knowledgeable about agriculture concepts after attending the agriculture literacy
workshop.
4.2.2. Engagement or interest
The majority of participants in the mental health literacy intervention in New Zealand (Morgaine et
al. 2017) agreed that the workshop was engaging and interesting and more than half of the patients
participating in the health education ‘charla’ in Honduras reported their increased interest in
learning about new health topics (Sookhoo 2014).
Another study (Wegner and Strehlke 2015) in German primary and secondary teachers and
students revealed that the participation in the workshop ‘Biology Up Close’ raised pupils’ overall
interest in science. Moreover, the workshop had a positive influence on the emotional and intrinsic
components of students’ interests in science, whose components, in turn, showed to be positively
influenced by the workshop’s characteristics of authenticity, reference to everyday life and
understandability, among other things (Wegner and Strehlke 2015).
4.2.3. Attitude
Goto and colleagues (2014) reported that more than half of the nurses participating in the health
literacy workshop gained confidence in developing, assessing and revising written materials and in
involving community residents using the Marker Method, an assessment method in which nurses
7
Friedman, AJ, Allen, S, Campbell, PB, Dierking, LD, Flagg, BN, Garibay, C, Korn, R, Silverstein, G and Ucko, DA. “Framework
for evaluating impacts of informal science education projects. Report from a National Science Foundation Workshop”
(2008): 114. http://www.informalscience.org/sites/default/files/Eval_Framework.pdf

15. 13
asked the readers to mark difficult words and phrases (see Goto et al. 2014); whereas in Goto, Lai,
and Rudd (2015) participants reported that in addition to gaining confidence in assessing and
revising written materials, participation in the workshop increased the potential of application of the
skills learnt to a new community, that at municipal level.
Across Honduran rural communities, patients also reported positive attitudes on the proposed
‘charla’ about health literacy (Sookhoo 2014).
Moreover, Sebbens and colleagues (2017) noted that Australian sport coaches and staff
participating in the MHS workshop were more confident in helping someone who might be
experiencing a mental health problem; similarly, medical American students who gained confidence
in assessing health literacy and communicating with patients of low health literacy after attending
the workshop and the OSCE (Bloom-Feshbach et al. 2015).
American teachers’ participation to the workshop training ‘Dig into Learning’ also showed
positive attitudes and perceptions towards the use of agriculture as a context for teaching and
learning science, technology, engineering and mathematics (STEM) education (Edwards 2016).
4.2.4. Behaviour
No behavioural changes were clearly reported in the studies analysed.
4.2.5. Skills
Goto and colleagues (2014) reported that more than half of the nurses participating in the health
literacy workshop in Japan applied the skills they have learnt in practice during the follow-up period.
Moreover, the great majority of workshop attendees could explain health literacy needs and use the
Marker Method, while more than half could write easy-to-read text (Goto et al. 2014). Moreover,
Goto, Lai, and Rudd (2015), reported that during the follow-up period, nurses applied the skills they
had gained to develop and communicate health information in various settings, such as school
health education, parenting support, support groups for the elderly home visits, and for different
types of materials, including leaflets, oral presentations and e-mails.
In addition, Baroutian and Kensington-Miller (2016) stated that the information literacy workshop
for New Zealand postgraduates was overall helpful and effective, with an average improvement in
students’ skills to identify, locate and evaluate information.
4.2.6. Others
Other impacts were reported by different authors in science educational fields (North, Gal, and
Zewortir 2014; Wegner and Strehlke 2015; Baroutian and Kensington-Miller 2016; Edwards 2016),
including a positive influence on postgraduate students’ confidence about choosing good quality and
scholarly information sources and an overall enjoyment of the information literacy workshop about
Food Processing in Australia (Baroutian and Kensington-Miller 2016); an improvement in South
African teachers’ confidence to teach statistics, as well as a growth in their confidence that learners
were able to understand and master statistics topics in the school curriculum and handle the
learning material with ease (North, Gal, and Zewortir 2014); elementary teachers’ excitement about
the integration of agriculture literacy contents to support STEM learning (Edwards 2016); an overall

16. 14
students’ enjoyment of the biology workshop held by teachers in the Bielefeld University’s
laboratories and the willingness to participate in similar workshops in the future (Wegner and
Strehlke 2015). The latter being particularly interesting since students’ pre-test revealed that the
majority of the pupils had a lack of interest in science and a rather reluctant attitude towards science
subjects at school (Wegner and Strehlke 2015).
In health literacy, three studies (Bloom-Feshbach et al. 2015; Goto, Lai, and Rudd 2015; Morgaine
et al. 2017) pointed out: an improved readability and content of participants’ written materials in a
Japanese workshop series for nurses (Goto, Lai, and Rudd 2015); the desire of many participants that
their whole workplace/organisation could take part in ‘GoodYarn’ training (Morgaine et al. 2017),
and, better performances for the medical students who completed the workshop prior to the OSCE,
in terms of checklist scores and reading level of their written instructions (Bloom-Feshbach et al.
2015). These students were also more likely to use and normalize the teach-back method (Bloom-
Feshbach et al. 2015).
Strengths 4.3.
The evidence on the use of a workshop format to deliver scientific and health concepts highlighted
some particular strengths.
The authors of the training workshops in Japan (Goto et al. 2014; Goto, Lai, and Rudd 2015)
considered that the experience served as an important reference for future disaster management
efforts, in addition to prove the importance of providing health literacy training opportunities for
professionals to strengthen health system’s ability to access information (Goto et al. 2014). This
study also highlighted the importance and value of adaptation rather than adoption; in this specific
case, the model developed with a Western perspective (i.e. United States of America) was modified
in accordance to the cultural and specific needs of a professional group of nurses in Fukushima City
and implemented using a culturally sensitive approach (Goto et al. 2014). These modifications
incorporated tools appropriate for the Japanese language and communication of risk specific to
Fukushima (Goto, Lai, and Rudd 2015). This case study also appeared to be particularly unique and
challenging since it was adapted to face the post-disaster phase, when public health nurses are
required not only to communicate evidence on health risks of radiation exposure but also transfer
their interpretations in layman terms (Goto et al. 2014). Moreover, Goto Lai and Rudd (2015)
reported that the workshop training had prompted the Japanese nurses to adopt a health literacy
perspective when reviewing all their communication activities thereafter and reaffirmed the
importance of health literacy for the participants.
In rural Honduras, ‘charlas’ offered a patient-centered platform to discuss public health needs
and dispense valuable and basic preventative health information to target communities with low
literacy levels (Sookhoo 2014). The author underlined the unsustainability of simply providing
medical care and medication in the absence of health education within the context of international
development. In the author’s experience, the use of ‘charlas’ played a significant role in addressing
public health issues and meeting the pressing need to cultivate a deeper understanding of personal
disease risk. Furthermore, they illustrated and how environmental factors that place patients at risk
for developing diseases require innovative approaches in imparting health literacy. Moreover,
’charlas’, which rely on verbal and visual communication, were a preferred method of health
education communication among Spanish-speaking Latinos, in comparison to brochures, as they

17. 15
found the medium more interactive and the information more relevant. This method of
disseminating information proved to be a viable medium communicating health information in low
literacy settings, using an approach that was culturally appropriate. It also confirmed that by using a
patient-centered care and empowerment approach in community health interventions, women
could increase control over their health and its determinants, particularly in a culture where
‘machismo’ attitudes are strong. It was proposed that the workshops could potentially transform
gender dynamics and increase health literacy through education (Sookhoo 2014).
The US skills-based health literacy workshop intervention (Bloom-Feshbach et al. 2015), aimed to
improve students’ ability to communicate health topics with low health literate patients, was also
found to be effective. The workshop participation additionally improved students’ communication
skills in asking validated screening questions and in employing and normalizing teach-back, all
clinically important techniques for effective communication with patients for whom low health
literacy is a barrier to care.
The experience in Australia (Sebbens et al. 2017), which addressed mental health problems in
elite sport environments, was found by the authors to be potentially transferable to contexts outside
elite sport and also to be extended beyond helping athletes to helping colleagues, friends, family and
persons in the community. Regarding a similar mental health literacy topic, Morgaine and colleagues
(2017) presented the results of the ‘GoodYarn’ project that was upscaled at the national level in New
Zealand. Participants overall recommended the workshop for the way it supported the discussion of
mental health and well-being, built an informed and connected rural community and encouraged an
attitude of caring. The interactive format and pleasant style of the workshop delivery were also
reasons for its success and for recommending it to others. Various participants valued the emphasis
placed on real-life experience and scenario-based role-plays. Moreover, the high level of
concordance in workshop outcomes across delivery by various organisations suggested that
programme consistency and quality was maintained throughout the upscaling. The authors found
that the uptake of the programme by rural organisations and communities at a national level,
combined with the positive evaluation results of the whole-of-country implementation, provided
encouragement that building mental health literacy in rural populations could be an effective
strategy for mental health promotion (Morgaine et al. 2017).
The biology workshop project presented by Wegner and Strehlke (2015) proved to support both
primary and secondary pupils in direct and vivid experience of science concurrent with providing
university students with practise in designing lessons plans and conducting practical workshops with
young people. Following the study’s results, pupils experienced the university’s laboratories as an
authentic extracurricular learning environment and they felt that the interdisciplinary and relevant
contents of the workshop supported their general scientific literacy. The study further revealed a
connection between pupils’ interest in science and experiments and their interest to work in the
scientific field, although the participation in only one workshop was not found to be able to change
their overall career choice. However, as reported, pupils’ interest affected their professional
ambitions and since the workshop supported their current interest in science, it could have made
them more open to considering an internship or even studies in this sector in the future (Wegner
and Strehlke 2015). Another example of a workshop in a formal education setting was the American
agricultural programme ‘Dig into Learning’ (Edwards 2016). This was perceived by the teachers from
a rural North Carolina school district to have a positive impact on STEM education and to provide a
useful context for learning. The workshop particularly focused on the importance of integrating real-

18. 16
world, hands-on learning into the daily curriculum to promote the success of 21st
century learners
(Edwards 2016).
The information literacy training workshop for Australian postgraduate students highlighted how
it helped them to go beyond their limited knowledge to search for information, assisted them to
investigate deeper into their subject matter and be more aware of the rich store of technical and
scholarly information available outside the lecture room (Baroutian and Kensington-Miller 2016). In
particular, the students reported that the workshop improved their ability to identify databases
relevant to their discipline, use proper keywords and search terms, evaluate information based on
originality and credibility and to cite information resources based on standards and guidelines
(Baroutian and Kensington-Miller 2016). The findings from this study also indicated that the need to
support international postgraduate students’ transition, learning and practice through well-
developed information literacy training, was key to their success in the course since their main
challenges were unfamiliarity with information sources, practices and with academic language
(Baroutian and Kensington-Miller 2016).
Weaknesses 4.4.
In general, there were not many references to weak points for this mechanism.
In the workshop programme in Japan, the authors noticed that the materials chosen for rewriting
by the participating nurses were for health promotion rather than for disaster or risk-related
materials. The authors speculated that the participants might not have been comfortable addressing
more difficult materials or those with many scientific terms during the workshop (Goto Lai and Rudd
2015). In addition, the analyses revealed that there were still needs for further training in the
paraphrasing of medical and administrative terms, interpretations and explanations of ‘risk’ (Goto
Lai and Rudd 2015).
A second observation comes from the study of Sookhoo (2014). The authors observed that
although the workshops could prove beneficial to changing attitudes and improving health, if the
structure of resource-poor communities remains deplorable and health and gender inequality
remains unchallenged, the ability to undo negative health behaviours could go unchanged. The use
of an empowerment approach was therefore suggested to be essential to strengthen and build
capacity within communities and to achieve greater health outcomes (Sookhoo 2014).
Costs and feasibility 4.5.
In the experience of a workshop aimed to integrate agriculture literacy into the curriculum of an
American school district, the researcher identified that teachers were concerned about time
required for the implementation of the contents in the scheduled lessons (Edwards 2016).
Researcher and curriculum specialists, therefore, provided participants with resources, such as
lesson plans and tips, to help reduce teachers’ time and simplify the utilization of the programme
(Edwards 2016).
In another experience, that of South Africa, the authors of the training workshop (North, Gal, and
Zewortir 2014) reported that teachers participated in workshops, week after week, at their own cost
and often travelling long distances; this was seen as an indication of teachers’ recognition of the
positive impact of the lessons on their ability to teach statistics. The authors also remarked that the

19. 17
lack of resources and the practical limitations on participation in the workshop must be taken into
account in designing more effective interventions. In particular, they suggested the examination of
alternative modes of delivery, such as a ‘travelling trainer’ or distance education methods, e.g.
internet-based lectures or demonstration videos provided on DVD, still available in remote areas.
They concluded that even the lack of electricity at some schools could be overcome by offering
online lessons and demonstrations to teachers at community centers, where internet access is
available (North, Gal, and Zewortir 2014).
Suggestions for improved methodologies and for future studies 4.6.
North, Gal, and Zewortir (2014) proposed that a concerted, multi-workshop series should be
designed in advance and spread over time, ‘in order to be able not only to present relevant
information but also to examine its effectiveness with participants, allowing them to share their own
instructional experiences’. The authors of the study suggested the need for a proper programme
evaluation module to be designed in advance, not only in the form of a ‘summative evaluation
regarding changes in knowledge and attitudes, but also a formative evaluation component that looks
at barriers and blocking factors regarding teachers’ initial interest and willingness to participate,
actual participation patterns, post-workshop behaviours (e.g., did teachers implement any of the
ideas in the workshop, and if so, how and when) and their underlying causes, and so forth’ (North,
Gal, and Zewortir 2014).
Another example in formal education (Baroutian and Kensington-Miller 2016) in Australia
recommended the adoption and integration of information literacy into the curriculum of all
postgraduate qualifications in the form of a workshop.
As a result of the workshop held in rural communities in Honduras, patients recommended that
more images should be included to improve the ‘charla’ (Sookhoo 2014). This suggestion might
reflect the need for these communities to receive health information through images, pictures,
symbols and visual aids while incorporating the viewpoint of patients and, in this way, communicate
health effectively to a low-literate populace (Sookhoo 2014).
The most commonly suggested workshop improvement reported in the study of Morgaine and
colleagues (2017) was to increase number and frequency of the workshops, time allocated for
general discussion and group-based activities in order to share real-life experiences of conversations
about mental health.
Conclusions and overview 5.
‘Workshops’ appear to be a mechanism used to support the delivery of a variety of scientific subjects
(Table 4) and examples of its application were found to be widely used both in formal and non-
formal education (Table 6). The target sectors of this mechanism were also diversified, from schools
and universities to local communities and workers (Table 7).
The evidence suggests that workshops are a versatile mechanism for delivering science and
health-related topics and shown to have been successfully adopted as a training platform for
improving teaching statistics (North, Gal, and Zewortir 2014), introduced as an extra-curricular
activity in science (Wegner and Strehlke 2015) and used to support the knowledge and importance
of agriculture in the curriculum (Edwards 2016), to cite some examples. In other interventions,

20. 18
workshops were proven to bear potential as an effective vehicle to facilitate the discussion of mental
illness and mental health in farmers (Morgaine et al. 2017) and elite sport environments (Sebbens et
al. 2017).
Within the examples analysed, ‘workshops’ seem to offer many possibilities for modifications and
adaptations, for example, in terms of ways of delivering the concepts and interactivity with the
audience(s), e.g. flip-charts (Sookhoo 2014), Powerpoint presentations (Bloom-Feshbach et al. 2015),
laboratories’ activities (Wegner and Strehlke 2015), peer delivery methods (Morgaine et al. 2017),
‘charlas’ facilitated by symbols and images (Sookhoo 2014). The adaptability of the ‘workshop’
format and the possibility of its combinations with other mechanisms, e.g. videos or e-learning,
might represent an interesting advantage, also in terms of costs and coverage of the intervention.
The aspect of empowering communities and professionals appeared as a common goal for
several of the interventions (e.g. Sookhoo 2014; Goto Lai and Rudd 2015) where this mechanism was
used to raise consciousness across a wide range of topics, from basic health needs in low-literate
rural communities (Sookhoo 2014) to national preparedness for risk communication in public health
crisis scenarios (Goto Lai and Rudd 2015).
Nevertheless, it may be important to note that studies of impact assessment and evaluation of
this mechanism are still somewhat limited and that more substantive conclusions that support and
illustrate the full potential and versatility of this mechanism may be forthcoming in the future.

21. 19
APPENDIX A: Example of data input mask

22. 20
Extracted from:
Nicklas, Theresa, Sandra Lopez, Yan Liu, Rabab Saab, and Robert Reiher. “Motivational Theater to
Increase Consumption of Vegetable Dishes by Preschool Children.” International Journal of
Behavioral Nutrition and Physical Activity 14 (February 7, 2017): 16. https://doi.org/10.1186/s12966-
017-0468-0.

23. 21
APPENDIX B: Selected bibliography
Impact assessments
Baroutian, Saeid, and Barbara Kensington-Miller. “Information Literacy: The Impact of a Hands-on
Workshop for International Postgraduate Students.” Education for Chemical Engineers 14
(January 2016): 16–23. https://doi.org/10.1016/j.ece.2015.10.001.
Bloom-Feshbach, Kimberly, Dana Casey, Lucy Schulson, Peter Gliatto, Jonathan Giftos, and Reena
Karani. “Health Literacy in Transitions of Care: An Innovative Objective Structured Clinical
Examination for Fourth-Year Medical Students in an Internship Preparation Course.” Journal of
General Internal Medicine 31, no. 2 (2015): 242–46. https://doi.org/10.1007/s11606-015-3513-
1.
Edwards, Erica Brown. “Dig into Learning: A Program Evaluation of an Agricultural Literacy
Innovation.” PhD thesis, Gardner-Webb University, 2016. https://eric.ed.gov/?id=ED570279.
Goto, Aya, Alden Yuanhong Lai, and Rima E. Rudd. “Health Literacy Training for Public Health Nurses
in Fukushima: A Multi-Site Program Evaluation.” Japan Medical Association Journal: JMAJ 58,
no. 3 (September 2015): 69–77.
Goto, Aya, Rima E. Rudd, Alden Yuanhong Lai, and Hiromi Yoshida-Komiya. “Health Literacy Training
for Public Health Nurses in Fukushima: A Case-Study of Program Adaptation, Implementation
and Evaluation.” Japan Medical Association Journal: JMAJ 57, no. 3 (May 1, 2014): 146–53.
Morgaine, Kate, Louise Thompson, Katie Jahnke, and Rebecca Llewellyn. “GoodYarn: Building Mental
Health Literacy in New Zealand’s Rural Workforce.” Journal of Public Mental Health 16, no. 4
(December 18, 2017): 180–90. https://doi.org/10.1108/JPMH-07-2017-0027.
North, Delia, Iddo Gal, and Temesgen Zewotir. “Statistics Education Research Journal.” Statistics
Education Research Journal 13, no. 2 (2014): 15–27.
Sebbens, Joshua, Peter Hassmén, Dimity Crisp, and Kate Wensley. “Mental Health in Sport (MHS):
Improving the Early Intervention Knowledge and Confidence of Elite Sport Staff.” Frontiers in
Psychology 7 (2016): 911. https://doi.org/10.3389/fpsyg.2016.00911.
Sookhoo, Linda. “Challenging Low Health Literacy in Rural Honduras: The Utilization of Charlas
(Chats) for Patient-Centered Care.” Global Social Welfare 1, no. 4 (December 2014): 191–203.
https://doi.org/10.1007/s40609-014-0017-6.
Wegner, Claas, and Friederike Strehlke. “The Benefits of the German Science Project ‘Biology Up
Close.’” Nordina 11, no. 3 (2015): 304–12.
Descriptive resources
Plank, Margret, Attila Dávid Molnár, and Paloma Marín-Arraiza. “Extending Media Literacy
Education: The Popular Science Video Workshop,” 15. Wroclaw, 2017.
http://library.ifla.org/1776/1/242-plank-en.pdf.